This question is probably better to explain with an example date, for instance the datetime now (London time): 21/08/2020 11:34 am
I'm confused about the time with 'ET' and also UTC format, and ISO format, is EDT same as ET?
Why so many different format and they are sooooo confusing. Can someone explain in a simple way, the information online is not very easy to understand unless I didn't find the right documentation.
The reason I want it to convert ET to UTC is because AWS cron job only take this format (please correct me if I'm wrong) Thanks in advance.
Yeah, you are right. AWS Cron job takes only UTC timestamp.
UTC and GMT are different ways of tracking time. EDT(ET) both are same. You can read about ISO vs UTC here, nicely explained here.
To covert one timestamp to other one, you can use this.
I'm creating a c++ project that should works on several timezone. The application receives an event, with a reference timezone, and this event is shown graphically to the user at the correct hour, in his local timezone. For example, an user working in Berlin receives an event written in Tokyo. The event from Tokyo is first converted in UTC time, then reconverted from UTC to the computer local time in Berlin, and finally shown to the user on his graphical interface.
To convert from UTC to local computer time I have several functions of the Windows API at my disposal to do the job. But in order to convert a time from another timezone to UTC, I need to get the timezone information from the Windows registry.
Now some timezone have also a Daylight Saving Time to consider. I'm able to create a recurrence rule from the Windows info without problems. However I noticed that the day the DST should occur is sometimes incorrect on several timezone. For example, the "E. South America Standard Time". With the recurrence provided by Windows, the DST start day begins 1 week sooner.
If I understood right, the recurrence rule returned by Windows for this specific timezone says "every year, on the 2nd month, on the 2nd week of the month". However this rule matches rarely with the correct date published on the internet for the time changing, whereas the dates are all correct if the rule would be "every year, on the 2nd month, on the 3nd week of the month". Furthermore, as you can see on the provided screenshot, the Windows registry data shows 2 weeks for the DST start time (highlighted in blue), but 3 weeks for the DST end time (surrounded in red), which is calculated correctly by my code. The description of the data content may be found here: http://msdn.microsoft.com/en-us/library/windows/desktop/ms725481(v=vs.85).aspx
I have several questions
Have I understood the recurrence rule correctly? (Here is what MSDN says about it: https://msdn.microsoft.com/en-us/library/windows/desktop/ms725481(v=vs.85).aspx)
Is there known issues about several timezone, especially the "E. South America Standard Time" one?
Is there a reason why a DST start date, that obviously occurs regularly every year, on the 3rd week of the 10th month, have a value set on the 2nd week?
Are timezone written in Windows registry reliable, if not, which function of the Windows API should I use to convert a timezone with DST from a date written in a different timezone than the one set on the local machine?
NOTE I have strongly verified if the data I read from the registry were correct before posting this message. I'm pretty sure that is not an error of this type.
NOTE I'm working with Windows 7, but the issue remains the same on Windows 10
As a Microsoft employee with significant involvement with Windows time zone data, please allow me to assure you that Microsoft works very hard to ensure that it releases updates to keep Windows time zone data as accurate as it possibly can be.
There are several challenges, including the timing of time zone changes as given by governments. For example, we recently posted a notice about upcoming Windows time zone changes for Fiji, Cyprus, Sudan, Tonga, Namibia, and Turks & Caicos. In some cases, we can meet the effective-dates established by these different governments. In some cases we cannot because they don't offer enough lead time.
Consider the recent case of Sudan, who told the the IANA tz mailing list on October 17th, 2017 of a change effective November 1st 2017. IANA has processed this change, and so has Microsoft. But due to the short notice, and the time it takes to process such a change in the Windows operating system, it will be a little while before there is a new "Sudan Standard Time" time zone created in the Windows time zone data. Thus, we issue interim guidance to use a different time zone for the time being, then switch back once we have data that properly reflects the full history of time in the affected region.
With regard to the Windows time zone you mentioned, "E. South America Standard Time", the time zone data is correct. If you expand the registry entry, you'll see that there is a sub-key called "Dynamic DST" which contains the year-by-year changes to the data. For some time zones, this isn't necessary at all, and for other time zones you'll find a very small amount of data because the same rule repeats year after year. But in the case of Brazil, you'll notice Dynamic DST entries for 2009 through 2040.
(click the image to see the full resolution, and note the areas I manually marked in red that change year-over-year)
The Dynamic DST entries support the Win32 DYNAMIC_TIME_ZONE_INFORMATION structure, and corresponding APIs with "dynamic" in their name, such as GetDynamicTimeZoneInformation. (They also support the System.TimeZoneInfo class in the .NET Framework.) Most of these APIs have been in places since Windows Vista / Server 2008, with some others coming in Windows 7 / Server 2012.
Note that the TZI entry that is in the parent key is copied in from the current year's dynamic rule by internal Windows processes. This supports the APIs that work with Win32 TIME_ZONE_INFORMATION structures, which have been in place since Windows 2000.
To answer the specific questions you asked:
Have I understood the recurrence rule correctly?
The specific rule you cited is for 2017 only, and it says that DST ends on the 2nd month (February), on the 3rd Saturday, at 23:59:59.999 local time, and starts again on the 10th month (October), on the 2nd Saturday, at 23:59:59.999 local time. Keep in mind that Brazil is in the southern hemisphere, so DST starts late in the year and ends early in the next year.
Also, you may wonder why it's 23:59:59.999 on Saturday instead of 00:00:00.000 on Sunday. This is an artifact of history. There have in the past been certain programs and processes that incorrectly used <= instead of < to evaluate the transition, and thus would accidently move the clock one millisecond into the next day, back out of it, then back into it again. The events generated by the day changing erroneously could then lead to further problems. Microsoft has done their best to fix those bugs, but still opts to be 1ms off instead of risk the problem popping up in some new place one day. (This only applies for transitions that occur exactly at midnight.)
Is there known issues about several timezone, especially the "E. South America Standard Time" one?
There are no known issues for that time zone, however there is a known issue that Windows time zones (even with the dynamic DST data) can only support a maximum of two time zone transitions in a single year. So, in places like Morocco that transition four times per year there are some internal workarounds in place that keep the current time zone data in sync such that "now" is an accurate representation of local time, but cannot represent all points of the year correctly at any given time.
We also don't currently have a time zone that maps cleanly for Troll Station, Antarctica. The reason is that from 2005 to 2015, this research station (population under 50) transitioned through three different offsets (UTC+0, UTC+1, and UTC+2) every year.
If your application is critically dependent on either of the above scenarios, then I recommend using an API with IANA data sources instead of the Win32 APIs.
Is there a reason why a DST start date, that obviously occurs regularly every year, on the 3rd week of the 10th month, have a value set on the 2nd week?
Yes, as mentioned above, it's due to the 1ms intentional error. DST in 2017, in the parts of Brazil that have DST, starts on Sunday October 15th at 00:00:00.000. That's the third Sunday of the month. 1ms prior is 23:59:59.999 on Saturday October 14th, which is the second Saturday of the month. This can be different every year, which is why there is Dynamic DST data.
Are timezone written in Windows registry reliable, if not, which function of the Windows API should I use to convert a timezone with DST from a date written in a different timezone than the one set on the local machine?
If you are using the Win32 APIs, they use the registry data themselves, so there is no need to work with the registry data directly. You should prefer the "Dynamic" versions of the APIs, as they properly account for the year-over-year changes in the Dynamic DST data. Sometimes these are labeled as "Ex". For example, the function you asked about is best handled by the TzSpecificLocalTimeToSystemTimeEx function.
...
That all said, if you are able to avoid using Windows time zone data in your application, I recommend doing so. Prefer IANA data sources, or those derived from them. There are many routes to working with IANA time zone data. Newer Windows APIs like Windows.Globalization.Calendar and Windows.Globalization.DatetimeFormatting.DateTimeFormatter in WinRT/UWP do indeed use IANA time zones, and that is clearly the path forward. In the standard C++ space, I highly recommend using Howard Hinnant's date/tz libraries, or those provided by the ICU project. There are several other viable choices as well.
Great to see the level of detail in Matt’s post.
On the question of "Is there known issues about several timezone, especially the "E. South America Standard Time" one?"
Nothing much more to add save that in addition to Matt’s incredibly detailed and thorough contributions to the Windows time zone data, there’s considerable vetting with different agencies, NGOs and government bodies that’s been outlined elsewhere. As an example, there are additional challenges in places like Morocco which not only transitions four times a year, but the information is subject to the government setting the official observance of DST in the region, and sometimes with little lead time, leading to additional publishing and deployment challenges. (Not to mention when a government revises a decision multiple times.)
So, for some applications with critical dependencies, an API to IANA source may be preferred, or calling the time zone data in Windows.Globalization.
All this is timely, if you’ll pardon the pun, just before we Fall Back in the northern hemisphere as well.
I am referring the documentation of _filefirst() and _findnext() APIs here
These APIs return file information in a _finddata_t structure. I need to access file modification time from time_write element. Though documentation says that
time is stored in UTC format (It is a times stamp). Documentation doesn't clarify if this time represents local time or UTC time. It seems to me that time_write doesn't return the UTC time instead its value is influenced by the system time zone settings.
My Question is - Does time_write returns local time represented in the UTC timestamps ?
Edit1
Here I explain what actually I am trying to understand. My system is in IST timezone. Now, there is a file emp10.ibd for which windows shows
Date Created - 10/21/2016 10:51 AM
Date Modified -10/21/2016 10:51 AM
I used epoch converter to find out the the epoch timestamp for which it turn out to be as following -
Now if I retrieve the time_write element from _finddata_t structure which has been returned by _findnext() for the same file i.e. emp10.ibd. I expect the returned timestamp should be close to
Epoch timestamp 1477027260 as shown in the image above.
But I get the time_write as 1477043509
If I again use epoch converter I get the following
I am trying to understand why there is 4:30 Hours of time difference in GMT in both images shared above? IMO timestamp should have been almost same. What obvious I am missing here ?
Edit2
For those folks who were asking for sample code. Here I paste link of another post which I had asked a year ago for the same reason but scenario was little different, There I was referring to _stati64 struct. I didn't troubleshoot the problem further at that time. By now it is pretty clear that
_finddata_t and _stati64 APIs are affected by _tzset environment variable as Harry mentioned in this post while FILETIME struct is not.
Local time is UTC plus a geographical offset plus potentially a seasonal offset. A UTC timestamp has no such offsets.
In this particular case, the exact format is seconds since1970-01-01T00:00:00Z i.e. January 1st, 1970, at midnight UTC.
To troubleshoot further, next I used GetFileTime API to retrieve the
the file modification time in FILETIME struct and converted the time into UTC timestamp. I got the time according the time set on my computer. I was expecting the same.
At this point I started investigating the way we execute our program through a perl script. I found that perl script was setting the timezone to GMT-1.
Since my computer was in timezone GMT+5:30, therefore I used to get resultant +04:30 hrs of difference as mentioned in the original post.
Therefore I would like to sum up my experience as - the outcome of _finddata_t strcut is affected by the timezone set in the session but the outcome of FILETIME struct is not affected by the time zone set in the session, instead it is the time according the system timezone. Since I was retrieving one time using FILETIME struct and another using _finddata_t strcut that was causing the problem. Took me ~48Hrs to find out this interesting observation.
Why does that happen? Perhaps the answer is provided by Harry in the comment section.I am pasting the same here as it is -
changing the timezone in Perl is probably causing the TZ environment variable to be set, which affects the C runtime library as per the documentation for _tzset. It isn't a per-session setting, at least not in the way Windows uses the word "session"."
Edit1
From File Times, I read the following -
FindFirstFile retrieves the local time from the FAT file system and converts it to UTC by using the current settings for the time zone and daylight saving time.
Though I was using the NTFS file system but I believe it uses the same mechanism i.e. retrieve the local time from file system and converts it to UTC by using current settings. That's the reason I noticed the difference.
I need to pull a date out of a string. Since not everyone uses the official ISO format when printing their dates, it is impractical to write a date parser for every possible date format that could be used, and I need to handle as many date formats as possible - I don't control the data and can't expect it to come in a specific format.
This seems like a problem that has probably already been solved ages ago, but my Google-fu is too weak to find the solution. :(
Does there already exist a C++ library that, given a string, will return the month, day, year, hour, minute, second, etc that is referenced in that string, if any?
Pseudocode:
string s1 = "There is an expected meteor shower this Thursday,"
"August 15th 2013 at 4:39 AM.";
string s2 = "20130815T04:39:00";
date d1 = magicConverter(s1);
date d2 = magicConverter(s2);
assert(d1 == d2);
You might use the code from here, but you need to configure a mask, that tells the code which time format is used. If you write a class routine, that takes a mask and a string and gets you out the time and is able to print in any format you like, you should be well prepared. You have to look in more detail, if it also supports Daynames and Monthnames. I got it to work in python with a module providing a function that seems pretty much the same.
For more detail:
Please look at the example 2013-08-03 again. Nobody and as follows no computer is able to tell you if this date belongs to August or April, except of having a mask telling JJJJ-MM-DD or JJJJ-DD-MM. Also this library may tell you only standard masked times. So it might lead you to August in this case. But as you said it can be any date declaration, thus it does not need to follow standards, thus it can also mean March. An other possibility is to tell you about the date from the context (e.g. a table with a column of all te same time formats by looking for the increase (which would also fail if you just look at one day per month for just one year).
Another example... if I ask you 2013-05-04... to which month does it belong? You might tell me... April. I would reply "no, to the 4th of May" and vice versa for May and 5th of April. If you tell me how to solve this puzzle with two possible solutions I would understand your downvote... please think before downvoting someone trying to help you.
I'm designing a project that will be developed in Django and I had a design philosophy question. In my app I need to track information like current week. This is related to the current week in the NFL (1-17) and can be calculated based on other models in the system (schedule and the current day for example). Since this information gets updated once a week, and will be used quite often in the app, does it make sense to store this information in a model (db table) of its own and just run the update weekly?
There is other information that might be useful to store as well (date/time of first and last games of the current week) so would a model of something like "current weeks information" be appropriate for this, even though the data can be calculated on the fly?
would a model of something like
"current weeks information" be
appropriate for this, even though the
data can be calculated on the fly?
It might be. You can calculate the date Easter falls on, but few applications do that. The calculation is far from dead simple, and any error would have to be treated as a bug fix. But if you store Easter dates in a table, any error can be fixed by anyone who can update calendar data.
It's simple to calculate USA holidays like Martin Luther King Day (observed on the 3rd Monday in January), President's Day (observed on the 3rd Monday in February), and Labor Day (observed on the 1st Monday in September). It's also pretty easy to calculate factory production weeks, which parallels your problem in some ways.
But when I'm building tables for businesses to use for scheduling, estimating, process control, and so on, I like to have the dates that are important to the business--holidays, for example--stored in a table rather than in procedural (calculating) code. The main advantage is that they can be collected, reviewed, and approved or corrected by relatively unskilled employees instead of needing a programmer.
So, if I were in your shoes, I would probably store the weeks in a table. A secondary advantage (or maybe the main advantage, in your case) is that most queries involving weeks might take advantage of indexes on the start and end dates.