Django attempts to address the timezone problem by storing dates internally in UTC and converting them to the client's timezone for display. This sounds fine and good in theory, until you realize two major things:
Many timezones can and do exist inside of the same UTC offset.
Since there are no timezone HTTP headers, we need to determine the timezone of the client manually, and this requires the use of JavaScript. However, JavaScript can only reliably determine the UTC offset of the client and may not guess the correct timezone.
With these two problems in mind, I assume a simple solution would be to ignore timezones, DST, etc. altogether and rely instead on the client's current UTC offset. On each page load, JavaScript on the client would update the client's cookie with the client's current UTC offset and middleware in Django would load that value for each request.
Here is the problem: Django makes use of get_current_timezone() which retrieves it's data from the value set when timezone.activate() was last called. timezone.activate() takes a timezone object as an argument.
Is there a way to use timezone.activate() with only a UTC offset?
The solution you describe, of getting the client's current UTC offset and sending back to the server, either via a cookie, or some other mechanism, is a common approach. Unfortunately it's flawed. Just because people do this doesn't make it a good idea.
The problem is that the offset you gather from the client is for a specific moment in time. However, you may not be working with that same moment in time on the server.
For example, you might call new Date().getTimezoneOffset() on the client, which gives you a value of 480, which is 480 minutes West of UTC, or UTC-08:00 (note the sign inversion). So you pass 480 to the server, load a date from the DB in UTC, and apply the offset. Except, perhaps the date you loaded was from several months ago, and the client's offset for that date was UTC-07:00. You have therefore applied the wrong offset, and produced a resulting value that is an hour off from what it should be.
A time zone cannot be identified by an offset alone. A time zone identifier looks like "America/Los_Angeles", not just UTC-8. This is a very common mistake. Read more under "time zone != offset" in the timezone tag wiki.
There are only two correct ways to handle this scenario:
Use a library like jsTimeZoneDetect or moment-timezone to guess the time zone of the browser, then let the user pick their time zone, defaulting to the guessed value. You can then use the selected or guessed time zone in your server-side code with Django or whatever.
Send only UTC to the client, do the conversion from UTC to local time in the browser using JavaScript. (The browser understands the behavior of the local time zone where it is running, even if it has trouble identifying it.) The catch here is - older browsers might possible convert older dates incorrectly, due to this bug. But for the most part, this is still a reasonable approach.
Related
Which one is best to use, DateTime or INT (Unix Timestamp) or anything else to store the time value?
I think INT will be better at performance and also more universal, since it can be easily converted to many timezones. (my web visitors from all around the world can see the time without confusion)
But, I'm still doubt about it.
Any suggestions?
I wouldn't use INT or TIMESTAMP to save your datetime values. There is the "Year-2038-Problem"! You can use DATETIME and save your datetimes for a long time.
With TIMESTAMP or numeric column types you can only store a range of years from 1970 to 2038. With the DATETIME type you can save dates with years from 1000 to 9999.
It is not recommended to use a numeric column type (INT) to store datetime information. MySQL (and other sytems too) provides many functions to handle datetime information. These functions are faster and more optimized than custom functions or calculations: https://dev.mysql.com/doc/refman/5.7/en/date-and-time-functions.html
To convert the timezone of your stored value to the client timezone you can use CONVERT_TZ. In this case you need to know the timezone of the server and the timezone of your client. To get the timezone of the server you can see some possibilites on this question.
Changing the client time zone The server interprets TIMESTAMP values
in the client’s current time zone, not its own. Clients in different
time zones should set their zone so that the server can properly
interpret TIMESTAMP values for them.
And if you want to get the time zone that a certain one you can do this:
CONVERT_TZ(#dt,'US/Central','Europe/Berlin') AS Berlin,
I wouldn't store it in int, you should check out MySQL Cookbook by Paul DuBois he covers lot's of things in it.Also there is a big portion about your quetion.
I am referring the documentation of _filefirst() and _findnext() APIs here
These APIs return file information in a _finddata_t structure. I need to access file modification time from time_write element. Though documentation says that
time is stored in UTC format (It is a times stamp). Documentation doesn't clarify if this time represents local time or UTC time. It seems to me that time_write doesn't return the UTC time instead its value is influenced by the system time zone settings.
My Question is - Does time_write returns local time represented in the UTC timestamps ?
Edit1
Here I explain what actually I am trying to understand. My system is in IST timezone. Now, there is a file emp10.ibd for which windows shows
Date Created - 10/21/2016 10:51 AM
Date Modified -10/21/2016 10:51 AM
I used epoch converter to find out the the epoch timestamp for which it turn out to be as following -
Now if I retrieve the time_write element from _finddata_t structure which has been returned by _findnext() for the same file i.e. emp10.ibd. I expect the returned timestamp should be close to
Epoch timestamp 1477027260 as shown in the image above.
But I get the time_write as 1477043509
If I again use epoch converter I get the following
I am trying to understand why there is 4:30 Hours of time difference in GMT in both images shared above? IMO timestamp should have been almost same. What obvious I am missing here ?
Edit2
For those folks who were asking for sample code. Here I paste link of another post which I had asked a year ago for the same reason but scenario was little different, There I was referring to _stati64 struct. I didn't troubleshoot the problem further at that time. By now it is pretty clear that
_finddata_t and _stati64 APIs are affected by _tzset environment variable as Harry mentioned in this post while FILETIME struct is not.
Local time is UTC plus a geographical offset plus potentially a seasonal offset. A UTC timestamp has no such offsets.
In this particular case, the exact format is seconds since1970-01-01T00:00:00Z i.e. January 1st, 1970, at midnight UTC.
To troubleshoot further, next I used GetFileTime API to retrieve the
the file modification time in FILETIME struct and converted the time into UTC timestamp. I got the time according the time set on my computer. I was expecting the same.
At this point I started investigating the way we execute our program through a perl script. I found that perl script was setting the timezone to GMT-1.
Since my computer was in timezone GMT+5:30, therefore I used to get resultant +04:30 hrs of difference as mentioned in the original post.
Therefore I would like to sum up my experience as - the outcome of _finddata_t strcut is affected by the timezone set in the session but the outcome of FILETIME struct is not affected by the time zone set in the session, instead it is the time according the system timezone. Since I was retrieving one time using FILETIME struct and another using _finddata_t strcut that was causing the problem. Took me ~48Hrs to find out this interesting observation.
Why does that happen? Perhaps the answer is provided by Harry in the comment section.I am pasting the same here as it is -
changing the timezone in Perl is probably causing the TZ environment variable to be set, which affects the C runtime library as per the documentation for _tzset. It isn't a per-session setting, at least not in the way Windows uses the word "session"."
Edit1
From File Times, I read the following -
FindFirstFile retrieves the local time from the FAT file system and converts it to UTC by using the current settings for the time zone and daylight saving time.
Though I was using the NTFS file system but I believe it uses the same mechanism i.e. retrieve the local time from file system and converts it to UTC by using current settings. That's the reason I noticed the difference.
This strange behavior has recently came to my attention, while I was testing my Rails app on local environment in which I use around_filter to set the timezone to registered user (the default timezone is UTC).
What I did was that I registered a new user in my app. My current time was 10pm GMT-5 (March 3), and this user's created_at time was saved to database to 4am UTC (March 4). Now, I know that this time is saved in database with the timezone settings, but here comes the problem:
I use a graph for visual representation of daily registered users, and when I called the following function to tell me number of users registered in the last few days:
from ||= Date.today - 1.month
to ||= Date.today
where(created_at: from..to).group('DATE(created_at)').count
It would say that this user was registered in March 4, while it was in fact registered on March 3 from my perspective.
My question is:
How should I call where function and group by a created_at column, so that the dates with be affected correctly (according to my timezone) ?
Or is there something else that I should be doing differently?
I'm not a rubyist, so I'll let someone else give the specific code, but I can answer from a general algorithmic perspective.
If you're storing UTC in the database, then you need to query by UTC as well.
In determining the range of the query (the from and to), you'll need to know the start and stop times for "today" in your local time zone, and convert those each to UTC.
For example, I'm in the US Pacific time zone, and today is March 7th, 2015.
from: 2015-03-07T00:00:00-08:00 = 2015-03-07T08:00:00Z
to: 2015-03-08T00:00:00-08:00 = 2015-03-08T08:00:00Z
If you want to subtract a month like you showed in the example, do it before you convert to UTC. And watch out for daylight saving time. There's no guarantee the offsets will be the same.
Also, you'll want to use a half-open interval range that excludes the upper bound. I believe in Ruby that this is done with three dots (...) instead of two (at least according to this).
Grouping is usually a bit more difficult. I assume this is a query against a database, right? Well, if the db you're querying has time zone support, then you could use it convert the date to your time zone before grouping. Something like this (pseudocode):
groupby(DATE(CONVERT_TZ(created_at,'UTC','America/Los_Angeles')))
Since you didn't state what DB you're using, I can't be more specific. CONVERT_TZ is available on MySQL, and I believe Oracle and Postgres both have time zone support as well.
Date.today will default to your system's set timezone (which by the way should always be UTC, here's why) so if you want to use UTC, simply do Time.zone.now.to_date if rails is set to UTC
Otherwise you should do
Time.use_zone('UTC') do
Time.zone.now.to_date
end
After this you should display the created_at dates by doing object.created_at.in_time_zone('EST')
to show it in your current timezone
In my Django app I've got a Task model with some date and time fields:
class Task(models.Model):
date = models.DateField()
start_time = models.TimeField(help_text='hh:mm')
end_time = models.TimeField(help_text='hh:mm')
# more stuff
I'll send some Task instances to some Android clients that will be in a time zone (TZ1) different from my server time zone (TZ2).
The start_time and end_time fields must be set to the target time zone (TZ1), i.e. if I enter '13:00' in the start_time field in the Task admin, it should be '13:00' in TZ1.
How can I set the start_time and end_time values to be TZ1 times? If I leave the values entered in the default admin I guess the times will be set to the server time zone (TZ2), right?
Then what's the best format to send these values (through JSON) to the Android clients to get the correct TZ2 time?
Now I'm using Python Datetime's isoformat(), which gives something like
2013-02-11T13:17:23.811680
but it has no time zone data...
This is not the best way to handle timezones.
The best way is to convert times to UTC as early as possible and convert them back as late as possible.
In other words, if I enter the current time here as Feb 11, 21:03, it should never be stored like that. Instead it should be changed to UTC before anything else happens.
That's so, no matter what happens with it, it's correct. If I send it to Inner Mongolia, it should stay as UTC right up until the point someone wants to look at it. Then and only then should it be converted (and for display only).
Following that rule will save you a lot of grief in any software that has to work across multiple timezones. Trust me on that, we fixed a major Telco up after they'd implemented some hideous system that sent timezones across the wire, meaning that every point had to be able to convert to and from every timezone.
Getting them into UTC as quickly as possible, and only getting them back on demand, saved bucketloads of time and money.
I am quering a (.net) web service from Excel (takes cover from the backlash), which returns a array of dates. I have noticed that any date that is on the day boundry gets shifted forward 1 hour, I have checked the web service on the server and the original date is correct.
I then used Fiddler to view the call on the client machine, and the dates which are at midnight have a Z on the end (to mark it as UTC) where as all other date times don't.
UPDATE The above statement is wrong, it is the first and last date that is marked with a Z, it doesn't matter what the time is. The rest is correct that the client only adjusts the first and last date
I take it SOAP is then changing this from UTC to local (BST) and therefore adding an hour.
My question is how can I stop the Z from being added onto these dates (or change it to local)?
I have checked the culture in IIS 7.5 and it is set to en-GB, the server is set to UK.
It's a little fiddly, and this isn't really an answer but should give you some things to try.
One way might be to remove the offset. I don't have the code to had but in psudeo-like code it might be
TimeZone tz = GlobalizationNamespace.GetTimeZone("Greenwich Mean Time");
SomeDateTimeNoOffset = SomeDateTime.AddHours(tz.GetUtcOffset(SomeDateTime));
This should might stop the service from putting a Z at the end as it has no TimeZone offset.
Another other option would be to expose the property in the Service Contract as a String
public string DateTimeNoTzInfo
{
get { DateTime.ToString('yyyy-MM-ddThh:mm:ss.ttttt') }
}