How does Django internally store a datetime field? Does it translate local time to UTC before storing? I am giving it a naive datetime generated from stdlib.
a = Message( fromm = fromm, to = to,
sent_time = datetime.datetime.now(), content = content)
Django ORM stores date time in your SQL database's corresponding field. The underlying SQL column is determined by the used database. E.g. for MySQL, the mappings are defined here:
https://github.com/django/django/blob/master/django/db/backends/mysql/creation.py#L16
Conversion from Django to SQL:
https://github.com/django/django/blob/master/django/db/backends/mysql/base.py#L272
Conversion from SQL to Django:
https://github.com/django/django/blob/master/django/db/models/fields/init.py#L1186
Python datetime.datetime.now() is so-called timezone-naive datetime and doesn't have any timezone information. Thus, you may lose information when storing times with it and it's usage is discouraged. It returns the local time. It seems that that if timezone information is omitted, the Django USE_TZ setting determines if the default timezone information is retrofitted internally.
Instead, you should use django.utils.timezone.now() (local time with timezone) or django.utils.timezone.utcnow() (time with UTC timezone).
More info
https://docs.djangoproject.com/en/dev/topics/i18n/timezones/#naive-and-aware-datetime-objects
Related
I have this code that behaves in a rather strange way and opens the question, how should I deal with timezones? So, first I have a datetime object I build from the info a user posts:
time_zone = request.POST.get("time_zone")
date_start = request.POST.get("date_start")
time_day = request.POST.get("time_day")
time_zone_obj = pytz.timezone("Etc/" + time_zone) # GMT + 2 in this example
date_start = datetime.strptime(date_start, "%d/%m/%Y")
date_start = date_start.replace(tzinfo=time_zone_obj)
time_day = datetime.strptime(time_day, "%I:%M %p")
date_start = date_start.replace(hour=time_day.hour, minute=time_day.minute)
...
event.date_start = date_start
event.save()
print("event.date_start.hour:%s" % event.date_start.hour)
print("event.date_start.tzinfo:%s" % event.date_start.tzinfo)
print("is_aware(event.date_start:%s)" % is_aware(event.date_start))
return redirect("event_detail", event_id=event.id)
This prints event.date_start.hour:6, event.date_start.tzinfo:Etc/GMT+2 and is_aware:True. Then, inmediatlty after saving the object and printing the hour, it redirects to the event_detail view, very simple:
def event_detail(request, event_id):
event = get_object_or_404(Event, id=event_id)
print("event.date_start.hour:%s" % event.date_start.hour)
print("event.date_start.tzinfo:%s" % event.date_start.tzinfo)
...
And it prints event.date_start.hour:8 and event.date_start.tzinfo:UTC. (it has replaced the tz info with UTC) I don't understand why. I am saving the object with a clear tz_info. Plz note that I printed the hour after I saved the object and then after I retrieved it in the other view. It has a difference of two hours that must have something to do with the timezone the user selected (GMT + 2). Why is this? Which is the best way to save this data?
The user submits "6:00 AM" + "GMT+2" in the form and then later when I want to show the time in the event detail html ({{ event.date_start|date:"h:i A" }}) it displays "8:00 AM".
I assume you're using PostgreSQL to save the timezone aware timestamp.
It's important to understand that (contrary to the name and popular belief) PostgreSQL doesn't save the timezone of the timezone aware timestamp. It's just a way to tell PostgreSQL that the value is not in some local time, but is timezone aware.
PostgreSQL then converts it to UTC and stores as such. If the original timezone is important, you need to store it separately.
More info on the topic: https://www.postgresqltutorial.com/postgresql-timestamp/
The best way to store this data is a separate column (usually called timezone). I use https://pypi.org/project/django-timezone-field/
Then either activate timezone (https://docs.djangoproject.com/en/3.1/ref/utils/#django.utils.timezone.activate) or use localtime (https://docs.djangoproject.com/en/3.1/ref/utils/#django.utils.timezone.localtime) util function.
As per the Django docs,
"When support for time zones is enabled, Django stores DateTime information in UTC in the database. It’s still good practice to store data in UTC in your database. The main reason is the Daylight Saving Time (DST). "
So saving DateTime in UTC format in the database as expected.
Now, going ahead with your requirement. In order to display the time back in the timezone which was used for saving you need to add a column in the DB to store the timezone info.
While retrieving the DateTime, convert it into the required timezone back using the tzinfo stored in DB.
This is the correct way of doing. Hope this helps you understand better.
I have deleted all timezone-naive entries out of my Sqlite-DB. So now they remaining entries have a datefield like this:
2016-09-04 13:28:16+00
When I now run my query like this:
result = Feedentry.objects.filter(date_published__gt=timezone('Europe/Berlin').localize(datetime(2016, 8, 31, 17)))
First I receive no error, but as soon as I want to access the result (or len(result)) I receive the following error:
raise ValueError('Not naive datetime (tzinfo is already set)')
Any ideas how to solve it or what I did wrong?
Django always stores datetimes in UTC. And in SQLite it doesn't store any timezone information at all:
[Databases other than PostgreSQL] store datetimes without time zone information. If you switch from USE_TZ = False to USE_TZ = True, you must convert your data from local time to UTC – which isn’t deterministic if your local time has DST.
So first you need to go through your database and convert all the values to UTC; otherwise, Django will not interpret the values correctly.
As for your comparison, it is correct if you're using pytz.timezone and a naive datetime. But the error message implies that whatever datetime you're passing to localize() is already timezone-aware. To convert a timezone-aware datetime you need to use datetime.astimezone(), as noted in the pytz documentation.
I'm trying to export some data from Django to Excel using openpyxl.
The exported datetimes are correctly interpreted as such in Excel, but their printout is in UTC, not in local timezone (CET in this case) as I'd expect.
I tried to use to_excel, but that only outputs datetimes converted to excel's internal float format. Which is in addition also interpreted as float, not datetime. When formatted as datetime, it's still in UTC
I also tried to use Django's django.utils.timezone.localtime, but again the dates are rendered in UTC.
I could just subtract the timezone offset from my dates, but I feel it's more likely I'm missing something here.
How can I export datetime data such that Excel would display it in my local timezone?
I had the similar problem and solved it in the following way.
May be it helps.
from dateutil.tz import tzutc, tzlocal
TZ_UTC = tzutc() # UTC timezone
TZ_LOCAL = tzlocal() # Local timezone
datevalue = value #value I get from API I am using, which is datetime object.
# For some reason I don't get tzinfo filled).
datevalue = datevalue.replace(tzinfo=TZ_UTC) # Adding time zone info for UTC
datevalue = datevalue.astimezone(TZ_LOCAL) # Converting to local timezone
datevalue = datevalue.replace(tzinfo=None) # Removing tzinfo to correctly record local time to Excel
cell.value = datevalue
Excel itself has no concept of timezones and will always dates and times naively. In this context the only sane thing to do is to convert to UTC which is what openpyxl does. openpyxl.utils.datetime is the module to look at if you want to change this
I ended up using a combination of javascript & server-side processing:
At the client HTML I create an input for user's local timezone:
<input type="hidden" value="" name="tz" id="xls-tz"/>
and populate its value (using jQuery):
$("#xls-tz").val(new Date().getTimezoneOffset());
At the server, I parse the timezone offset & write to openpyxl accordingly:
tz_offs = int(request.GET.get("tz", "0"))
ws.cell(row=row, column=2, value=item.time - timedelta(minutes=tz_offs))
That comes IMO pretty close to what I needed.
Thanks Charlie for the hint about Excel not being TZ aware.
I'm using ReportLab in Django. I have a model with the following field:
time_stamp = models.DateTimeField(auto_now_add=True)
And my TIME_ZONE variable in settings.py is set to:
Africa/Johannesburg
I use a formset to populate this model. The time_stamp field saves correctly with the correct time zone, but when I place the time_stamp in my ReportLab pdf, the time zone is set to UTC.
For example:
time_stamp in the saved model (as str(time_stamp)[:19] is:
2015-03-04 07:57:28
But time_stamp in pdf document (as str(time_stamp)[:19] is:
2015-03-04 05:57:28
Exactly 2 hours earlier (Africa/Johannesburg is UTC + 2hours).
How can I set the time zone for ReportLab? Should it be specified in settings.py or in views.py while generating the pdf? If there is no solution, how do I add 2 hours to the time_stamp?
Some answers suggested changing auto_now_add=True with default=datetime.datetime.now(), but this creates a warning while migrating the database (Naive expression used).
I'm not a user of Reportlab but in general I don't think setting USE_TZ=False is the right approach to solve your problem. Set it back to True and instead of truncating your time stamp yourself like:
str(time_stamp)[:19]
You should try applying Django's date template filter in your template, e.g.:
{{ time_stamp|date:"SHORT_DATETIME_FORMAT" }}
It can be confusing how Django is handling timezones. But it's best practice to save timestamps in your database in UTC. You may want to refer to Django's timezone FAQ:
https://docs.djangoproject.com/en/dev/topics/i18n/timezones/#time-zones-faq
I have :
TIME_ZONE = 'Europe/Paris'
USE_L10N = True
USE_TZ = True
in my settings.py file. I am living in Istanbul/Turkey and there is one hour difference between Paris and Istanbul.
In the admin side when selecting a date, django correctly shows 1 hour difference. And using template tag i am getting the datetime i have set in the admin.
But when i pass the datetime via python using beginning_date.strftime("%H:%M") python substracts 1 hour from the value that was set via admin which is not true.
How can i solve this?
Use the Django template defaultfilters to format your dates in Python code.
from django.template.defaultfilters import date as _date
_date(datetime_object, "%H:%M")
And, maybe related: Django cannot reliably use alternate time zones in a Windows. See documentation.
I don't think Turkey has anything to do with it.
My guess is that the one-hour difference you're seeing is between the Paris timezone—which is being used, by default, to interpret and display dates—and UTC—which is being used to store the datetime, and which is the timezone of the datetime returned from the database.
If that's correct, then you can just use django.utils.timezone.localtime to convert the datetime to the current time zone (which by default will be TIME_ZONE):
localtime(beginning_date).strftime("%H:%M")