I have a variable:data_datetime = dateutil.parser.parse(data['datetime'])
I print it out ,it is 2016-03-12 01:49:43.082514+00:00
And I save into database :obj.update_time = data_datetime
But the database's time is wrong,It save 2016-03-12 01:49:43.082514+08 (It add a timezone 8 hours)
It should save 2016-03-12 01:49:43.082514+00:00 or 2016-03-12 09:49:43.082514+08
I felt very confused. How can I save the right time into database???
I have another try current_utc_time = datetime.datetime.now(pytz.utc)
and I print it out : 2016-03-12 02:01:58.631653+00:00
And when it save to databaseobj.update(id=id,update_time=current_utc_time)
It save the right time I want : 2016-03-12 10:01:58.364853+08
I don't know why data_datetime = dateutil.parser.parse(data['datetime']) can't not save the right time
Please help me,thank you
I found a difference: one is use obj.update_time = data_datetime ,another is use obj.update(id=id,update_time=current_utc_time)
If it is the root cause, How can I save the right time I want???
When you start out from Python datetime object, it gets serialized properly to the database. When you round trip from a database datetime object, parsed to a Python object, them serialized back, the timezone is getting corrupted. The dateutil package docs for the parse function (https://dateutil.readthedocs.org/en/latest/parser.html#dateutil.parser.parse) talk about ignoretz as well tzinfos parameters, and the fact that it will make naive guesses about timezones. Coupled with the fact that all the examples in those docs show a two digit timezone offset, my guess is you need to replace 00:00 with 00 in the strings you are parsing. Or, use the tzinfos parameter somehow to map the ISO8601 timezone offset style to the UTC time zone so the parser isn't making incorrect guesses on data it can't parse.
Related
Which one is best to use, DateTime or INT (Unix Timestamp) or anything else to store the time value?
I think INT will be better at performance and also more universal, since it can be easily converted to many timezones. (my web visitors from all around the world can see the time without confusion)
But, I'm still doubt about it.
Any suggestions?
I wouldn't use INT or TIMESTAMP to save your datetime values. There is the "Year-2038-Problem"! You can use DATETIME and save your datetimes for a long time.
With TIMESTAMP or numeric column types you can only store a range of years from 1970 to 2038. With the DATETIME type you can save dates with years from 1000 to 9999.
It is not recommended to use a numeric column type (INT) to store datetime information. MySQL (and other sytems too) provides many functions to handle datetime information. These functions are faster and more optimized than custom functions or calculations: https://dev.mysql.com/doc/refman/5.7/en/date-and-time-functions.html
To convert the timezone of your stored value to the client timezone you can use CONVERT_TZ. In this case you need to know the timezone of the server and the timezone of your client. To get the timezone of the server you can see some possibilites on this question.
Changing the client time zone The server interprets TIMESTAMP values
in the client’s current time zone, not its own. Clients in different
time zones should set their zone so that the server can properly
interpret TIMESTAMP values for them.
And if you want to get the time zone that a certain one you can do this:
CONVERT_TZ(#dt,'US/Central','Europe/Berlin') AS Berlin,
I wouldn't store it in int, you should check out MySQL Cookbook by Paul DuBois he covers lot's of things in it.Also there is a big portion about your quetion.
Timezones are driving me crazy. Every time I think I've got it figured out, somebody changes the clocks and I get a dozen errors. I think I've finally got to the point where I'm storing the right value. My times are timestamp with time zone and I'm not stripping the timezone out before they're saved.
TIME_ZONE = 'Europe/London'
USE_I18N = USE_L10N = USE_TZ = True
Here's a specific value from Postgres through dbshell:
=> select start from bookings_booking where id = 280825;
2019-04-09 11:50:00+01
But here's the same record through shell_plus
Booking.objects.get(pk=280825).start
datetime.datetime(2019, 4, 9, 10, 50, tzinfo=<UTC>)
DAMMIT DJANGO, IT WASN'T A UTC TIME!
These times work fine in templates/admin/etc but when I'm generating PDF and spreadsheet reports, this all goes awry and I'm suddenly have to re-localise the times manually. I don't see why I have to do this. The data is localised. What is happening between the query going to the database and me getting the data?
I bump into these issues so often I have absolutely no confidence in myself here —something quite unnerving for a senior dev— so I lay myself at your feet. What am I supposed to do?
You're interpreting this wrongly. The database stores a UTC time most of the time. If you use PostgreSQL, the database can store a time with time zone info, but for practical purposes (*) it's easiest to just think the time in your db is stored as UTC (i.e. as an absolute time that can be converted to any time zone) when USE_TZ = True. It always represents a correct point in time for which you don't need to remember or assume any timezone. And as far as I know, Django will always store the time as time-aware in UTC timezone.
So when you're fetching the time object using select in psql, you're getting back the time in your machine's local time zone (the time zone where you're running psql). If someone in "America/New_York" would run the same select query, she would see a -04 timestamp. Had the date been 2019-03-20, you'd have seen 2019-03-20 10:50:00+00 because on that date, Europe/London and UTC were the same.
When fetching the value of a DateTimeField as a python datetime.datetime object, Django always fetches the UTC value, because:
Dealing with aware datetime objects isn’t always intuitive. For
instance, the tzinfo argument of the standard datetime constructor
doesn’t work reliably for time zones with DST. Using UTC is generally
safe; if you’re using other time zones, you should review the pytz
documentation carefully.
This makes it easier to work with these datetime objects in your python code: They're always UTC times.
If you want to print these values in a PDF, use the same methods Django uses for the template rendering:
from django.utils import timezone
print(timezone.template_localtime(Booking.objects.get(pk=280825).start))
This renders the datetime in the default timezone (or if you activate() a different timezone, in the current timezone).
(*) Note: Why you should not give any meaning to the timezone saved in your db and just think about it as if it's all UTC: If you were to run servers in various timezones, you might actually end up saving timestamps in different timezones. They are still all correct (absolute timestamps) and can be converted to any other timezone. So basically the timezone used for saving is meaningless.
Please be aware that both Django and the Postgresql Database have their own timezone setting.
The Django timezone is set in the settings.py file:
TIME_ZONE = 'UTC'
USE_TZ = True
The Postgresql setting can be checked using:
SHOW TIMEZONE;
and set using:
SET TIMEZONE='UTC';
I'm not an expert on this, but I believe Django wants to store everything in UTC in the database and then convert to the Django timezone setting after it has been queried. On that basis I think you want to set the Postgresql timezone to be UTC then up to you if you change the Django setting to get the auto conversion or leave it as UTC and handle any conversions yourself in code.
I want to save a future wall-clock datetime for events in Django (I have timezone string stored separately).
I can't simply use the DateTimeField because it enforces timestamp with time zone and always saves time in current timezone. It doesn't handle DST or possible timezone changes between current date and the date of actual event.
I could use any of these options:
Pick any timezone to store timestamps and always throw this timezone away before applying actual timezone in Python.
Split timestamp to DateField and TimeField.
Store datetime as string.
Custom field that stores datetime as timestamp without time zone.
but it makes queries more difficult and seems quite weird.
Are there any better options I miss? This usecase seems quite common so I guess there is a better way to do that?
EDIT: my usecase:
Let's say my user want to book an appointment to 2019-12-20 10:00 and currently it's 2019-03-10. I know the timezone of this user (it's stored separately as string like 'US/Eastern').
If I assume that EST starts at November 3, 2019, the best I can do is to store timestamp to 2019-12-20 15:00:00+00:00 (or 2019-12-20 10:00-05:00. I don't want this because:
I have no idea if my tzdata has correct information for future datetime
Even if it currently does, I have no idea if there would be any unexpected change in US/Eastern timezone and it becomes worse when it's not US. Future DST changes are not guaranteed.
If user moves to different timezone, I'll have to recalculate every single appointment while taking care about DST.
If tzdata changes during this recalculation... let's not think about that.
I'd prefer to store future dates as naive datetime + timezone string like 'US/Eastern' and (almost) never construct tz-aware datetime for any date further than a week. Django + postgres currently forces me to use timestamp with time zone, which is great for logs and past events, but it has fixed offset (not even timezone name) so it doesn't fit for future wall clock datetimes.
For this usecase, let's say that I don't care about ambiguous times: not much users want to book at 02:00 AM.
I see a few possible solutions:
Set USE_TZ = False and TIME_ZONE = 'UTC' and use calendar times. No conversions will be done, so essentially you're just storing the calendar time and getting it back as a naive datetime. The main problem is that this setting is global, and is not a good one for many uses (e.g. auto_now).
As above, but set USE_TZ = True. As long as you express your calendar times in UTC, there won't be any untoward conversions. The problem here is that you'll be getting aware datetimes, so you'll have to take care to ignore or remove the time zone everywhere.
Use separate DATE_FIELD and TIME_FIELD. This may or may not be a good solution depending on what kind of queries you're trying to run.
Create your own field that uses timestamp without time zone. (Or perhaps it already exists?)
Note that this issue has nothing to do with past versus future. It's about wanting to use a fixed moment in time versus a calendar (or wall clock) time. The points you raised are certainly valid objections to using a point in time to represent a calendar time.
Mysql table one row =>
id = 1
time = "21:00" //datatype => TIME
name = "xyz"
while i am fetching the data
#person = #person.all
[#<persons id: 1, time: "2000-01-01 21:00:00, name: "xyz", created_at: "2014-03-19 05:13:43", updated_at: "2014-03-19 05:13:43", creator_id: nil">
#person[0].time # 2000-01-01 21:00:00
It should be "21:00" Right?
Why i am getting "2000-01-01 21:00:00" output any suggestion ??
After fetching data from query, you can format to time for displaying:
fetch all the records:
#person = #person.all
format to time.
#person.first.time.strftime("%H:%M")
for getting more information about date and time formating click here
#person[0].time returns the Ruby Time object for that value, which includes date. If you're outputting it directly to the console or browser window, Ruby is converting it to a default string representation of the Time object which, again, includes the date.
To format the time for display, you'll want to look at your options for date/time formatting using Rails' internationalization API.
try using strftime - it lets you specify format
ie something like:
#person[0].time.strftime("%m/%d/%Y %H:%M:%S")
Changing datatype from TIME TO VARCHAR :
It is not good approach. I think you should consider all the case about time, there is a lot possibility where you required TIME object to other operation like date.
If you store hours and minutes in VARCHAR then you can not perform any operation like TIME object. please think about, if you required hours or date from database in future but you have store only hours and minutes in database, you did not store (TIME Object)it, then it will be considered bad approach and failure.
you store TIME at the place of hours and minutes then It is good cause for future perspective.
If you store time object then you are store additional information with hours and minutes. If you will required Date, time, hours, minutes and even seconds then you can calculate by time formating.
If you store time then there is only one extra effort time formating that is negligible in perspective of performance.
To store TIME Object instead of hours and minutes in string format is always better.
I'm reading timestamp fields from a PostgreSQL database. The timestamp column is defined as:
my_timestamp TIMESTAMP WITH TIME ZONE DEFAULT NOW()
When reading from the database, I convert it to a boost timestamp like this:
boost::posix_time::ptime pt( boost::posix_time::time_from_string( str ) );
The problem seems to be that boost::posix_time::time_from_string() ignores the timezone.
For example:
database text string == "2013-05-30 00:27:04.8299-07" // note -07 timezone
boost::posix_time::to_iso_extended_string(pt) == "2013-05-30T00:27:04.829900"
When I do arithmetic with the resulting ptime object, the time is off by exactly 7 hours. Is there something better I should be doing to not lose the timezone information?
I think you should be using boost::local_date_time, which handles time zones. There is an example in the documentation that is very similar to what you're trying to do: http://www.boost.org/doc/libs/1_41_0/doc/html/date_time/examples.html#date_time.examples.seconds_since_epoch
EDIT: Boost supports date parsing with specific formats. http://www.boost.org/doc/libs/1_40_0/doc/html/date_time/date_time_io.html#date_time.format_flags
string inp("2013-05-30 00:27:04.8299-07");
string format("%Y-%m-%d %H:%M:%S%F%Q");
date d;
d = parser.parse_date(inp,
format,
svp);
// d == 2013-05-30 00:27:04.8299-07
I originally asked this question so many years ago, I don't even remember doing it. But since then, all my database date/time code on the client side has been greatly simplified. The trick is to tell PostgreSQL the local time zone when the DB connection is first established, and let the server automatically add or remove the necessary hours/minutes when it sends back timestamps. This way, timestamps are always in local time.
You do that with a 1-time call similar to this one:
SET SESSION TIME ZONE 'Europe/Berlin';
You can also use one of the many timezone abbreviations. For example, these two lines are equivalent:
SET SESSION TIME ZONE 'Asia/Hong_Kong';
SET SESSION TIME ZONE 'HKT';
The full list of timezones can be obtained with this:
SELECT * FROM pg_timezone_names ORDER BY name;
Note: there are over 1000 timezone names to pick from!
I have more details on PostgreSQL and timezones available on this post: https://www.ccoderun.ca/programming/2017-09-14_PostgreSQL_timestamps/index.html