C++ and Sqlite3: How to store date/time with milliseconds precision - c++

I´m building a C++ application that will be running in Ubuntu and will use Sqlite3 as a database.
One of my goals is to have a C++ class containing time/date fields and store then on database.
In the past I´ve used time_t as the variable type on my class and stored them as INTEGER type on Sqlite3, like:
C++ Class:
class MyClass {
time_t dateTimeInfo;
}
Sqlite3:
CREATE TABLE MYCLASS (....., INTEGER DATETIMEINFO, ...);
That approach allows me to SELECT times with different comparasion operatiors (>, >=, < , <=, ==) with no problems at all, as I´m dealing with simple number. At the user level the time_t is converted to ISO 8601 std::string´s so that I can have a human readable interface.
This works very well except that it does not support millisecods. In my current project I need to support them, so I need to make changes to this.
As far I had studies I undestand I need to use std::chrono::time_point as the class type, as follows:
C++ Class:
class MyClass {
std::chrono::time_point dateTimeInfo;
}
But I really don´t know what data type to use in Sqlite3 and if it will work the same way time_t used to...
Sqlite3:
CREATE TABLE MYCLASS (....., ???? DATETIMEINFO, ...);
My questions:
a) Is std::chrono::time_point the correct option here ?
b) What is the Sqlite3 type equivalent ? Still an INTEGER ?
c) Is this the recommended approach (no boost please, C++11) ?
Thanks for helping.

You can store the ISO 8601 string formatted times as TEXT directly. For instance, you can make the following table:
CREATE TABLE tbl(name TEXT, ts TEXT);
and insert ISO 8601 formatted strings with millisecond values as a text string:
INSERT INTO tbl VALUES ('first', '2015-07-06T10:59:46.1234Z');
INSERT INTO tbl VALUES ('second', '2015-07-06T10:59:47.5678Z');
INSERT INTO tbl VALUES ('third', '2015-07-06T10:59:48.9012Z');
At this point you can query them using comparison operators:
SELECT * FROM tbl WHERE ts <= '2015-07-06T10:59:46.1233Z';
// Returns nothing
SELECT * FROM tbl WHERE ts <= '2015-07-06T10:59:46.1234Z';
// Returns the first record
SELECT * FROM tbl WHERE ts > '2015-07-06T10:59:46.1234Z';
// Returns the last two records
As an added bonus, you get ISO 8601 formats on the way out when you interface with this database, which is what you're doing on the client side anyway.
This method makes use of the fact that within a timezone, the lexicographic ordering of the strings produces a semantically correct ordering of datetimes. If your usage scenario involves multiple timezones, using a single timezone, such as UTC time, for storage within the database helps preserve this ordering property.

Yes, std::chrono is a very good approach (C++ internally). You can convert time points from/to milliseconds using std::chrono like this:
using namespace std::chrono;
typedef time_point<system_clock, milliseconds> time_points_millis;
time_points_millis tp = system_clock::now();
// An at least 43 bit signed integral type
auto epoch = tp.time_since_epoch();
In this case you should use an 64-bit integral type to store it with SQLite because the data can exceed 32 bits.
Comment: I don't really know much about SQLite, yet, but I found this for 64-bit types: official docs.

Related

Delete record by time in InfluxDB-CXX

Reference: https://github.com/offa/influxdb-cxx
It is easy to delete record by time using CLI interface,
delete from imagetable where time='2022-11-16T19:42:41.945508272Z'
but I am not able to figure out how to do the same with influxdb-cxx. i.e. not able to access the time through C++ interface.
e.g. Tags can be accessed with function points[0].getTags() but how to access the time ?
Have already tried to access it with points[0].getTimestamp() but not able to print it in this format in C++ 2022-11-17T03:37:25.934547412Z
can anyone please help ? Thanks in advance.
In influxdb-cxx you can use InfluxDB::execute method to execute InfluxQL statements like from your example for CLI interface. Regarding timestamps, they are saved as std::chrono::time_point<std::chrono::system_clock> (source) in the library's Point class, which denotes Unix (epoch) time excluding leap seconds (which is what timestamps in InfluxDB represent). Your example uses RFC3339 notation to provide timestamp, but InfluxQL also directly understands "nanosecond count since epoch" notation for it (example). So, it isn't necessary to represent Point's timepoint in RFC3339 notation to use it in execute command (which is possible, but harder and reduntant), you can just use standard C++ chrono library functions to get nanoseconds since epoch for given timepoint. Example:
using namespace std::chrono;
auto nsEpoch = duration_cast<nanoseconds>(points[0].getTimestamp().time_since_epoch()).count();
idb->execute("delete from imagetable where time=" + std::to_string(nsEpoch));

Storing unix timestamp as an IntegerField [duplicate]

Which one is best to use, DateTime or INT (Unix Timestamp) or anything else to store the time value?
I think INT will be better at performance and also more universal, since it can be easily converted to many timezones. (my web visitors from all around the world can see the time without confusion)
But, I'm still doubt about it.
Any suggestions?
I wouldn't use INT or TIMESTAMP to save your datetime values. There is the "Year-2038-Problem"! You can use DATETIME and save your datetimes for a long time.
With TIMESTAMP or numeric column types you can only store a range of years from 1970 to 2038. With the DATETIME type you can save dates with years from 1000 to 9999.
It is not recommended to use a numeric column type (INT) to store datetime information. MySQL (and other sytems too) provides many functions to handle datetime information. These functions are faster and more optimized than custom functions or calculations: https://dev.mysql.com/doc/refman/5.7/en/date-and-time-functions.html
To convert the timezone of your stored value to the client timezone you can use CONVERT_TZ. In this case you need to know the timezone of the server and the timezone of your client. To get the timezone of the server you can see some possibilites on this question.
Changing the client time zone The server interprets TIMESTAMP values
in the client’s current time zone, not its own. Clients in different
time zones should set their zone so that the server can properly
interpret TIMESTAMP values for them.
And if you want to get the time zone that a certain one you can do this:
CONVERT_TZ(#dt,'US/Central','Europe/Berlin') AS Berlin,
I wouldn't store it in int, you should check out MySQL Cookbook by Paul DuBois he covers lot's of things in it.Also there is a big portion about your quetion.

How to correctly select date field using libpqxx?

I am trying to select date field from PostgreSQL database using libpqxx and C++.
I would use this code, but I don't know if it is legal. I have searched in the documentation but I haven't any documented way.
using time_point = std::chrono::steady_clock::time_point;
pqxx::work txn(c);
auto&& rst = txn.exec("SELECT date FROM table");
for(auto&& row : rst)
time_point date = row[0].as<time_point>();
Is this okey please? Do you know any better alternative?
I would like the same with date time and time field. Is there any difference please?
Thank you.
--
the documentation for field type: https://libpqxx.readthedocs.io/en/6.4/a01063.html#a3a55f6b44040b68e70382d9db7dea457
The answer on Github by JadeMatrix:
field.as<>() will work with any type for which a specialization of pqxx::string_traits<> exists. libpqxx comes with support for std::string, builtin numerics (int, etc.), and maybe a few others I don't remember.
Support for std::chrono:: types are missing by default, unfortunately. You can implement your own, but be warned that they will only work for TIMESTAMP WITHOUT TIME ZONE, DATE, TIME WITHOUT TIME ZONE, and INTERVAL. To correctly support … WITH TIME ZONE you will need Howard Hinnant's date library, which is what I use (there is talk of adding it to the standard library).
If you want, I can share my code, which relies on date functionality for parsing Postgres date/time strings (ISO 8601 format).

unable to get fraction of seconds using strptime()

I am receiving a datetime in YYYY-MM-DDThh:mm:ss[.S+][Z|+-hh:mm] this format. and i m trying to copy that value using strptime as shown below
struct tm time = {0};
char *pEnd = strptime(datetime, "%Y-%m-%dT%H:%M:%S%Z", &time);
But I can't copy the fraction of seconds as strptime doesn't support it in C++.
So what should i do?
I found a solution using gettimeofday(). but I am already getting date and time in 'datetime' so please help me to find soluntion for it...I can use poco also . but even their we take local time.
You can remove the "S" component from the string before parsing with strptime, then use it however you like later (it won't fit anywhere in a Standard struct tm - there's no sub-second fields). Just datetime.find('.') and go from there, or use a regexp if you prefer - both tedious but not rocket science.
In C++11 you can use the the high precision timing objects in <chrono>, see the answers to this question.

Postgresql date format

I have a web application (written with python/django) that (due a bad specification) Web Forms expecting "YYYY-mm-dd" date format and others using "dd/mm/yy" date format.
Is there a way to tell postgresql to accept dates in both formats? for example, to try "dd/mm/yy" and, if it fails, then try "yyyy-mm-dd".
That would be awesome.
From the fine manual:
Date and time input is accepted in almost any reasonable format, including ISO 8601, SQL-compatible, traditional POSTGRES, and others. For some formats, ordering of day, month, and year in date input is ambiguous and there is support for specifying the expected ordering of these fields. Set the DateStyle parameter to MDY to select month-day-year interpretation, DMY to select day-month-year interpretation, or YMD to select year-month-day interpretation.
PostgreSQL is more flexible in handling date/time input than the SQL standard requires. See Appendix B for the exact parsing rules of date/time input and for the recognized text fields including months, days of the week, and time zones.
So PostgreSQL should be able to deal with just about any date format you throw at it. Your "dd/mm/yy" format is, however, ambiguous. But, there is the DateStyle configuration parameter to help with such ambiguity.
For example:
=> create table x (d date not null);
=> insert into x values ('2001-01-10');
=> insert into x values ('Feb 2 2980');
=> insert into x values ('01/02/03');
=> select * from x;
d
------------
2001-01-10
2980-02-02
2003-02-01
That said, I'd recommend moving everything to ISO 8601 (YYYY-MM-DD) internally and handle the conversions at the edges of the application. OTOH, there is reality to contend with so you should do whatever you have to do to make it go.
If those are the only two formats possible then it may be better to explicitly allow only those, rather than rely on postgres to interpret. For example:
with w as (select '2011-12-13' as input_date union select '13/12/2011')
select case when input_date~'^\d\d\d\d-\d\d-\d\d$'
then to_date(input_date, 'yyyy-mm-dd')
when input_date~'^\d\d/\d\d/\d\d\d\d$'
then to_date(input_date, 'dd/mm/yyyy')
end
from w;
case
------------
2011-12-13
2011-12-13
(2 rows)