Django. How to correctly save time & timezones - django

I have this code that behaves in a rather strange way and opens the question, how should I deal with timezones? So, first I have a datetime object I build from the info a user posts:
time_zone = request.POST.get("time_zone")
date_start = request.POST.get("date_start")
time_day = request.POST.get("time_day")
time_zone_obj = pytz.timezone("Etc/" + time_zone) # GMT + 2 in this example
date_start = datetime.strptime(date_start, "%d/%m/%Y")
date_start = date_start.replace(tzinfo=time_zone_obj)
time_day = datetime.strptime(time_day, "%I:%M %p")
date_start = date_start.replace(hour=time_day.hour, minute=time_day.minute)
...
event.date_start = date_start
event.save()
print("event.date_start.hour:%s" % event.date_start.hour)
print("event.date_start.tzinfo:%s" % event.date_start.tzinfo)
print("is_aware(event.date_start:%s)" % is_aware(event.date_start))
return redirect("event_detail", event_id=event.id)
This prints event.date_start.hour:6, event.date_start.tzinfo:Etc/GMT+2 and is_aware:True. Then, inmediatlty after saving the object and printing the hour, it redirects to the event_detail view, very simple:
def event_detail(request, event_id):
event = get_object_or_404(Event, id=event_id)
print("event.date_start.hour:%s" % event.date_start.hour)
print("event.date_start.tzinfo:%s" % event.date_start.tzinfo)
...
And it prints event.date_start.hour:8 and event.date_start.tzinfo:UTC. (it has replaced the tz info with UTC) I don't understand why. I am saving the object with a clear tz_info. Plz note that I printed the hour after I saved the object and then after I retrieved it in the other view. It has a difference of two hours that must have something to do with the timezone the user selected (GMT + 2). Why is this? Which is the best way to save this data?
The user submits "6:00 AM" + "GMT+2" in the form and then later when I want to show the time in the event detail html ({{ event.date_start|date:"h:i A" }}) it displays "8:00 AM".

I assume you're using PostgreSQL to save the timezone aware timestamp.
It's important to understand that (contrary to the name and popular belief) PostgreSQL doesn't save the timezone of the timezone aware timestamp. It's just a way to tell PostgreSQL that the value is not in some local time, but is timezone aware.
PostgreSQL then converts it to UTC and stores as such. If the original timezone is important, you need to store it separately.
More info on the topic: https://www.postgresqltutorial.com/postgresql-timestamp/
The best way to store this data is a separate column (usually called timezone). I use https://pypi.org/project/django-timezone-field/
Then either activate timezone (https://docs.djangoproject.com/en/3.1/ref/utils/#django.utils.timezone.activate) or use localtime (https://docs.djangoproject.com/en/3.1/ref/utils/#django.utils.timezone.localtime) util function.

As per the Django docs,
"When support for time zones is enabled, Django stores DateTime information in UTC in the database. It’s still good practice to store data in UTC in your database. The main reason is the Daylight Saving Time (DST). "
So saving DateTime in UTC format in the database as expected.
Now, going ahead with your requirement. In order to display the time back in the timezone which was used for saving you need to add a column in the DB to store the timezone info.
While retrieving the DateTime, convert it into the required timezone back using the tzinfo stored in DB.
This is the correct way of doing. Hope this helps you understand better.

Related

How to get queryset in django?

I am trying to send an email to those orders that is created 5 minutes before the datetime.now(). I try to filter the orders but it is not working, it is not giving me any queryset. How to do this? I am sharing my code.
def my_email():
now = datetime.now() - timedelta(minutes=5) # 11:55
now = now.replace(tzinfo=pytz.utc)
print(now)
order = Order.objects.filter(createdAt__gt = now)
print(order)
for o in order:
print(o._id)
You should not replace the timezone with UTC. now() will obtain the current datetime for a given timezone. If you replace that with UTC, then the result can be several hours ahead or behind the current time.
It might also be better to make use of a Now() expression [Django-doc], which will use the clock of the database, we thus can filter with:
from django.db.models.functions import Now
from datetime import timedelta
order = Order.objects.filter(createdAt__gt=Now()-timedelta(minutes=5))
If you plan to run this to send emails for orders the last five minutes, this will however be quite error-prone: since a scheduled task can always have a few milliseconds of delay, it thus means that certain items can, by the time the query is running, be considered too old. You probably might want to use a BooleanField that indicates if an email has been sent.

Django-Postgres: how to group by DATE a datetime field with timezone enabled

I am having this problem with prostgresql and django:
I have a lot of events that were created on a certain date at a certain time which is stored in a datetime field created .
I want to have aggregations based on the date part of the created field. The simplest examples is: how many event are in each day of this month?.
The created field is timezone aware. So the result should change depending on the timezone the user is in. For example if you created 2 events at 23:30 UTC time on 2017-10-02 if you view them from UTC-1 you should see them on 3rd of October at 00:30 and the totals should add for the 3rd.
I am struggling to find a solution to this problem that works with a lot of data. So doing for each day and SQL statement is not an option. I want something that translates into:
SELECT count(*) from table GROUP BY date
Now I found a solution for the first part of the problem:
from django.db import connection
truncate_date = connection.ops.date_trunc_sql('day', 'created')
queryset = queryset.extra({'day': truncate_date})
total_list = list(queryset.values('day').annotate(amount=Count('id')).order_by('day'))
Is there a way to add to this the timezone that should be used by the date_trunc_sql function to calculate the day? Or some other function before date_trunc_sql and then chain that one.
Thanks!
You're probably looking for this: timezone aware date_trunc function
However bear in mind this might conflict with how your django is configured. https://docs.djangoproject.com/en/1.11/topics/i18n/timezones/
Django 2.2+ supports the TruncDate database function with timezones
You can now do the following to :
import pytz
east_coast = pytz.timezone('America/New_York')
queryset.annotate(created_date=TruncDay("created", tzinfo=east_coast))
.values("created_date")
.order_by("created_date")
.annotate(count=Count("created_date"))
.order_by("-created_date")

Date exported is displayed as UTC

I'm trying to export some data from Django to Excel using openpyxl.
The exported datetimes are correctly interpreted as such in Excel, but their printout is in UTC, not in local timezone (CET in this case) as I'd expect.
I tried to use to_excel, but that only outputs datetimes converted to excel's internal float format. Which is in addition also interpreted as float, not datetime. When formatted as datetime, it's still in UTC
I also tried to use Django's django.utils.timezone.localtime, but again the dates are rendered in UTC.
I could just subtract the timezone offset from my dates, but I feel it's more likely I'm missing something here.
How can I export datetime data such that Excel would display it in my local timezone?
I had the similar problem and solved it in the following way.
May be it helps.
from dateutil.tz import tzutc, tzlocal
TZ_UTC = tzutc() # UTC timezone
TZ_LOCAL = tzlocal() # Local timezone
datevalue = value #value I get from API I am using, which is datetime object.
# For some reason I don't get tzinfo filled).
datevalue = datevalue.replace(tzinfo=TZ_UTC) # Adding time zone info for UTC
datevalue = datevalue.astimezone(TZ_LOCAL) # Converting to local timezone
datevalue = datevalue.replace(tzinfo=None) # Removing tzinfo to correctly record local time to Excel
cell.value = datevalue
Excel itself has no concept of timezones and will always dates and times naively. In this context the only sane thing to do is to convert to UTC which is what openpyxl does. openpyxl.utils.datetime is the module to look at if you want to change this
I ended up using a combination of javascript & server-side processing:
At the client HTML I create an input for user's local timezone:
<input type="hidden" value="" name="tz" id="xls-tz"/>
and populate its value (using jQuery):
$("#xls-tz").val(new Date().getTimezoneOffset());
At the server, I parse the timezone offset & write to openpyxl accordingly:
tz_offs = int(request.GET.get("tz", "0"))
ws.cell(row=row, column=2, value=item.time - timedelta(minutes=tz_offs))
That comes IMO pretty close to what I needed.
Thanks Charlie for the hint about Excel not being TZ aware.

Django ReportLab Timezone

I'm using ReportLab in Django. I have a model with the following field:
time_stamp = models.DateTimeField(auto_now_add=True)
And my TIME_ZONE variable in settings.py is set to:
Africa/Johannesburg
I use a formset to populate this model. The time_stamp field saves correctly with the correct time zone, but when I place the time_stamp in my ReportLab pdf, the time zone is set to UTC.
For example:
time_stamp in the saved model (as str(time_stamp)[:19] is:
2015-03-04 07:57:28
But time_stamp in pdf document (as str(time_stamp)[:19] is:
2015-03-04 05:57:28
Exactly 2 hours earlier (Africa/Johannesburg is UTC + 2hours).
How can I set the time zone for ReportLab? Should it be specified in settings.py or in views.py while generating the pdf? If there is no solution, how do I add 2 hours to the time_stamp?
Some answers suggested changing auto_now_add=True with default=datetime.datetime.now(), but this creates a warning while migrating the database (Naive expression used).
I'm not a user of Reportlab but in general I don't think setting USE_TZ=False is the right approach to solve your problem. Set it back to True and instead of truncating your time stamp yourself like:
str(time_stamp)[:19]
You should try applying Django's date template filter in your template, e.g.:
{{ time_stamp|date:"SHORT_DATETIME_FORMAT" }}
It can be confusing how Django is handling timezones. But it's best practice to save timestamps in your database in UTC. You may want to refer to Django's timezone FAQ:
https://docs.djangoproject.com/en/dev/topics/i18n/timezones/#time-zones-faq

Django, Python datetime.now() and storing the timezones

How does Django internally store a datetime field? Does it translate local time to UTC before storing? I am giving it a naive datetime generated from stdlib.
a = Message( fromm = fromm, to = to,
sent_time = datetime.datetime.now(), content = content)
Django ORM stores date time in your SQL database's corresponding field. The underlying SQL column is determined by the used database. E.g. for MySQL, the mappings are defined here:
https://github.com/django/django/blob/master/django/db/backends/mysql/creation.py#L16
Conversion from Django to SQL:
https://github.com/django/django/blob/master/django/db/backends/mysql/base.py#L272
Conversion from SQL to Django:
https://github.com/django/django/blob/master/django/db/models/fields/init.py#L1186
Python datetime.datetime.now() is so-called timezone-naive datetime and doesn't have any timezone information. Thus, you may lose information when storing times with it and it's usage is discouraged. It returns the local time. It seems that that if timezone information is omitted, the Django USE_TZ setting determines if the default timezone information is retrofitted internally.
Instead, you should use django.utils.timezone.now() (local time with timezone) or django.utils.timezone.utcnow() (time with UTC timezone).
More info
https://docs.djangoproject.com/en/dev/topics/i18n/timezones/#naive-and-aware-datetime-objects