Similar to:
Convert historical time to GMT
This time my scenario is:
I need to convert some string times in the format "2011061411322100" into GMT. The problem is that the times are coming from another PC and is a historical time. So I am not getting the times in real time so I cannot simply get the GMT from the local time on the box that my code is running.
The times represent Start and End times. If my code is running during a time change, the time change will bre applied on the remote box where I am getting the times. So if the change happens betwwen the start and end time, then the end time will be offset.
I assume I must first convert to tm:
// add to tm struct
tm tmTime;
tmTime.tm_hour = hour;
tmTime.tm_min = min;
tmTime.tm_sec = sec;
tmTime.tm_mday = day;
tmTime.tm_mon = (month-1);
tmTime.tm_year = (year - 1900);
Then convert to time_t
time_t tmInTime_t = mktime( &tmTime );
Then use gmtime:
struct tm *gmt = gmtime( &tmInTime_t );
This will still cause large delta if time change happens between start and end. How do I fix?
Do I need to set .tm_isdst? How do I know what to set to?
You ABSOLUTELY must know what the reference timezone was on the source data. Simply saying it was local time on the source machine doesn't provide enough info.
Consider (for one example out of many): The DST rules changed in the US in 2007, and any date calculations spanning across that range must take that into account. When you start dealing with timezones across political borders things get vastly more complicated.
http://www.boost.org/doc/libs/1_47_0/doc/html/date_time.html
is an example of a robust time/date library written in C++ and works on most platforms. It will allow you to perform various 'arithmetic' operations on dates and times. It also includes support for the TZ database, which will allow you to perform operations on datetimes spanning timezone rule changes.
Related
Wrote something in .NET; it works well. Now I am trying to rewrite it as a shell extension with the Win32 API. Ultimately I want to convert FILETIMEs to and from ISO-8601 strings. This is doable without fuss, using GetTimeZoneInformation and FileTimeToSystemTime and SystemTimeToTzSpecificLocalTime and StringCchPrintf to assemble the members of the SYSTEMTIME and TIME_ZONE_INFORMATION structs into a string.
The problem, as usual when working with date/times, is Daylight Saving Time. Using GetTimeZoneInformation tells me the UTC offset that's in effect now. Using .NET's DateTime.ToString("o") takes into account the daylight saving time at the time represented in the DateTime.
Example for the same FILETIME:
Output of ToString("o"): 2017-06-21T12:00:00.0000000-05:00
Output of chained APIs: 2017-06-21T12:00:00-06:00
The UTC offset is wrong coming from the chained API calls. How does .NET's DateTime do it? How do we replicate that in Win32? Is there something like GetTimeZoneInformationForYear, but instead of for a year, for a moment in local time?
First, I use DYNAMIC_TIME_ZONE_INFORMATION structure and GetDynamicTimeZoneInformation
DYNAMIC_TIME_ZONE_INFORMATION and TIME_ZONE_INFORMATION also has a DaylightBias member:
The bias value to be used during local time translations that occur
during daylight saving time. This member is ignored if a value for the
DaylightDate member is not supplied.
This value is added to the value of the Bias member to form the bias
used during daylight saving time. In most time zones, the value of
this member is –60.
So, if the date is in daylight saving time, you need to add this DaylightBias to Bias.
In addition, you can determine whether the current date is daylight saving time according to the description in DaylightDate:
To select the correct day in the month, set the wYear member to zero,
the wHour and wMinute members to the transition time, the wDayOfWeek
member to the appropriate weekday, and the wDay member to indicate the
occurrence of the day of the week within the month (1 to 5, where 5
indicates the final occurrence during the month if that day of the
week does not occur 5 times).
If the wYear member is not zero, the transition date is absolute; it
will only occur one time. Otherwise, it is a relative date that occurs
yearly.
I'm attempting to modify the creation dates of files to the date they were released. I'm first converting a string such as "2 April 2005" into a std::tm. I then create a SYSTEMTIME as follows:
std::tm dt = from_string("2 April 2005");
SYSTEMTIME st { 0 };
st.wYear = dt.tm_year + 1900; // dt is years from 1900
st.wMonth = dt.tm_mon + 1; // dt is month index 0
st.wDay = dt.tm_mday;
st.wHour = 6; // FILETIME is based on UTC, which is 6 hours ahead
Afterwards I convert the SYSTEMTIME to a FILETIME and use that to apply the changes.
This sets the file time to 2 April 2005 12:00:00 AM which is correct. However, videos after April 2nd were being set to 1:00:00 AM and sure enough, daylight savings happened on April 3rd 2005.
How can I determine if a certain date is before or after daylight savings so I can adjust st.wHour accordingly? The goal is to have all times set to 12:00:00 AM. Preferably this would work on dates dating back to the 60s as well as current.
I tried using TIME_ZONE_INFORMATION and GetTimeZoneInformation but I only get back TIME_ZONE_ID_STANDARD.
A few things:
SYSTEMTIME is just a plain structure. It has separate fields for year, month, day-of-week, day (of month), hour, minute, second and millisecond. It is not either UTC or local time, or anything else by itself. That doesn't come into consideration until you pass it into a function.
FILETIME is another plain structure. It represents the number of 100-nanosecond intervals since midnight of 1601-01-01. Much of the docs would make you think it is always in UTC, but there are functions like FileTimeToLocalFileTime that disprove that. Thus, like SYSTEMTIME, it is up to each individual function to decide how to interpret it.
The SystemTimeToFileTime function accepts a pointer to a SYSTEMTIME that is interpreted as UTC, and returns a pointer to a FILETIME that is also in terms of UTC. There is no local time zone involved.
Don't try to adjust for local time yourself (st.wHour = 6 in your code). The hour will need to vary depending both on time zone and on DST.
You didn't see any response from GetTimeZoneInformation other than TIME_ZONE_STANDARD, because that tells you what is currently in effect - which is disconnected from the dates you might be working with.
You shouldn't be trying to figure out how to adjust for DST on your own. DST is not universally applicable, and it's not always one hour offset. Instead, use a function that converts from the time zone you care about to UTC.
Ultimately it sounds like you are asking about how to set the file time to midnight of the local time zone on a particular date. Thus, I suggest you go through these steps:
Construct a SYSTEMTIME with the date you care about and the time components set to zeros (the default).
Get the local time zone using the GetDynamicTimeZoneInformation function. You want the "dynamic" version such that any historical differences in rules for standard time and DST that Windows knows about are taken into consideration, rather than just the current set of rules.
Pass those two values to the TzSpecificLocalTimeToSystemTimeEx function. It will interpret the input time as being in the input time zone (which was the system's local time zone). The result is a FILETIME that is in terms of UTC.
Pass that value to SetFileTime, which expects the input in terms of UTC.
Also, keep in mind that not all file systems track file times in the same way:
NTFS stores the actual UTC time, so you can move files between computers and the timestamp represents the same point in Universal Time even if the computers have different time zone settings.
FAT and its variants store local time. So when you call SetFileTime, Windows translates from UTC to the local time zone and writes the result. If you then open the file on a system with a different time zone, the date will be interpreted in that time zone, resulting in a different UTC time. (This is often seen on USB sticks, memory cards, etc. when moving files from cameras to computers.)
Lastly, you said:
... Preferably this would work on dates dating back to the 60s as well as current.
Unfortunately, Windows time zones do not track historical dates that far. Microsoft's time zone policy is to track time zone and DST rules for 2010 and forward for all populated places on Earth. Although, several time zones track some historical changes from before 2010 as artifacts from before this policy was formalized. (They are accurate within a given zone, just not uniform in starting years across all zones).
If historical dates are significant to your application, you'll need a very different approach - one that doesn't use the Windows time zone data, but instead uses the IANA time zone database. (More on these in the timezone tag wiki.) Here are some ideas you could explore:
The ICU project has time zone support and a C implementation. It's a bit heavy for this one purpose, but good if you are using it for other localization aspects of an application.
Howard Hinnant (who commented on the question above) has an excellent Date library with IANA time zone support.
It might be possible to get what you need from the Windows.Globalization.Calendar UWP class. I haven't tested to see if it will use historical rules from IANA data or if it uses the Windows data. (If I get a chance to check, I'll come back and update this answer.)
Keep in mind that the IANA database only guarantees from 1970 forward. You said you needed from 1960, and though there are some zones with data for that (and some much older), there is no guarantee of correctness in that period.
I am referring the documentation of _filefirst() and _findnext() APIs here
These APIs return file information in a _finddata_t structure. I need to access file modification time from time_write element. Though documentation says that
time is stored in UTC format (It is a times stamp). Documentation doesn't clarify if this time represents local time or UTC time. It seems to me that time_write doesn't return the UTC time instead its value is influenced by the system time zone settings.
My Question is - Does time_write returns local time represented in the UTC timestamps ?
Edit1
Here I explain what actually I am trying to understand. My system is in IST timezone. Now, there is a file emp10.ibd for which windows shows
Date Created - 10/21/2016 10:51 AM
Date Modified -10/21/2016 10:51 AM
I used epoch converter to find out the the epoch timestamp for which it turn out to be as following -
Now if I retrieve the time_write element from _finddata_t structure which has been returned by _findnext() for the same file i.e. emp10.ibd. I expect the returned timestamp should be close to
Epoch timestamp 1477027260 as shown in the image above.
But I get the time_write as 1477043509
If I again use epoch converter I get the following
I am trying to understand why there is 4:30 Hours of time difference in GMT in both images shared above? IMO timestamp should have been almost same. What obvious I am missing here ?
Edit2
For those folks who were asking for sample code. Here I paste link of another post which I had asked a year ago for the same reason but scenario was little different, There I was referring to _stati64 struct. I didn't troubleshoot the problem further at that time. By now it is pretty clear that
_finddata_t and _stati64 APIs are affected by _tzset environment variable as Harry mentioned in this post while FILETIME struct is not.
Local time is UTC plus a geographical offset plus potentially a seasonal offset. A UTC timestamp has no such offsets.
In this particular case, the exact format is seconds since1970-01-01T00:00:00Z i.e. January 1st, 1970, at midnight UTC.
To troubleshoot further, next I used GetFileTime API to retrieve the
the file modification time in FILETIME struct and converted the time into UTC timestamp. I got the time according the time set on my computer. I was expecting the same.
At this point I started investigating the way we execute our program through a perl script. I found that perl script was setting the timezone to GMT-1.
Since my computer was in timezone GMT+5:30, therefore I used to get resultant +04:30 hrs of difference as mentioned in the original post.
Therefore I would like to sum up my experience as - the outcome of _finddata_t strcut is affected by the timezone set in the session but the outcome of FILETIME struct is not affected by the time zone set in the session, instead it is the time according the system timezone. Since I was retrieving one time using FILETIME struct and another using _finddata_t strcut that was causing the problem. Took me ~48Hrs to find out this interesting observation.
Why does that happen? Perhaps the answer is provided by Harry in the comment section.I am pasting the same here as it is -
changing the timezone in Perl is probably causing the TZ environment variable to be set, which affects the C runtime library as per the documentation for _tzset. It isn't a per-session setting, at least not in the way Windows uses the word "session"."
Edit1
From File Times, I read the following -
FindFirstFile retrieves the local time from the FAT file system and converts it to UTC by using the current settings for the time zone and daylight saving time.
Though I was using the NTFS file system but I believe it uses the same mechanism i.e. retrieve the local time from file system and converts it to UTC by using current settings. That's the reason I noticed the difference.
I'm reading timestamp fields from a PostgreSQL database. The timestamp column is defined as:
my_timestamp TIMESTAMP WITH TIME ZONE DEFAULT NOW()
When reading from the database, I convert it to a boost timestamp like this:
boost::posix_time::ptime pt( boost::posix_time::time_from_string( str ) );
The problem seems to be that boost::posix_time::time_from_string() ignores the timezone.
For example:
database text string == "2013-05-30 00:27:04.8299-07" // note -07 timezone
boost::posix_time::to_iso_extended_string(pt) == "2013-05-30T00:27:04.829900"
When I do arithmetic with the resulting ptime object, the time is off by exactly 7 hours. Is there something better I should be doing to not lose the timezone information?
I think you should be using boost::local_date_time, which handles time zones. There is an example in the documentation that is very similar to what you're trying to do: http://www.boost.org/doc/libs/1_41_0/doc/html/date_time/examples.html#date_time.examples.seconds_since_epoch
EDIT: Boost supports date parsing with specific formats. http://www.boost.org/doc/libs/1_40_0/doc/html/date_time/date_time_io.html#date_time.format_flags
string inp("2013-05-30 00:27:04.8299-07");
string format("%Y-%m-%d %H:%M:%S%F%Q");
date d;
d = parser.parse_date(inp,
format,
svp);
// d == 2013-05-30 00:27:04.8299-07
I originally asked this question so many years ago, I don't even remember doing it. But since then, all my database date/time code on the client side has been greatly simplified. The trick is to tell PostgreSQL the local time zone when the DB connection is first established, and let the server automatically add or remove the necessary hours/minutes when it sends back timestamps. This way, timestamps are always in local time.
You do that with a 1-time call similar to this one:
SET SESSION TIME ZONE 'Europe/Berlin';
You can also use one of the many timezone abbreviations. For example, these two lines are equivalent:
SET SESSION TIME ZONE 'Asia/Hong_Kong';
SET SESSION TIME ZONE 'HKT';
The full list of timezones can be obtained with this:
SELECT * FROM pg_timezone_names ORDER BY name;
Note: there are over 1000 timezone names to pick from!
I have more details on PostgreSQL and timezones available on this post: https://www.ccoderun.ca/programming/2017-09-14_PostgreSQL_timestamps/index.html
UPDATE: The "fraction of a second" parameter to Timestamp's constructor actually takes nanoseconds... I guessed it was hundredths of a second and my low values were rounded away. Question left for reference....
I'm struggling with Oracle's C++ library - OCCI. Summarily:
creating Timestamp objects and verifying they're good to hundredths of a second (though I'd like more!)
using stmt.setTimestamp then executeUpdate() to insert into a TIMESTAMP(6) column which should preserve microseconds
selecting the row in Oracle SQL Developer: the sub-second component is always 0-ed e.g. 14-JUL-11 06.03.27.000000000.
Problem
I need subsecond precision - hopefully microseconds! We've put a lot of work into capturing that precision in our servers and need (at least some of) it for analysis.
Details
I create a Timestamp from year/month/day hour/minute/second/millisecond, reducing the last to hundredths of a second as that seems to be what the constructor supports. (No Oracle documentation I can find specifies the interpretation, but in a fromText example "xff" clearly corresponds to a ".##" hundredths suffix in the value to convert. What's the point of TIMESTAMP(6) supporting 6 decimal places if you can't insert them?)
oracle::occi::Timestamp temp =
oracle::occi::Timestamp(_env, year, month, day,
hour, minute, second, millisecond / 10);
// re-extract the broken-down time from temp to prove it's stored successfully
int ye;
unsigned mo, da, ho, mi, se, fs;
temp.getDate(ye, mo, da);
temp.getTime(ho, mi, se, fs);
return temp;
Here, fs gets the milliseconds/10 value as expected.
I use this as in:
oracle::occi::Timestamp ts;
ts = _pImpl->makeOracleTimestamp(p->ATETimeStamp);
stmt.setTimestamp(11, ts);
Where field 11 is a TIMESTAMP(6).
Selecting the row in Oracle SQL Developer, the other parts of the timestamp column are correct but the sub-second component is 0-ed ala 14-JUL-11 06.03.27.000000000.
Any insight much appreciated!
(If relevant, using MSVC++ 2005, Oracle 10.2.0.4 sdk, SQL Developer 3.0.04 - please ask if something else might be relevant).
Thanks,
Tony
Turns out the "fractional seconds" field is nominally in nanoseconds rather than hundredths. I wish Oracle would say that in their documents! I say nominally because if it really preserved the least-significant digits then the hundredths values I had would have appeared as a number of nanoseconds and I might have immediately guessed at the problem - instead it seems values < 100 nanoseconds are lost anyway (and perhaps bigger - I haven't probed the cut-off point).
Thanks to anyone who had a look at the question or tried some research / investigation.