I'm trying to get time left till the day of an event using this annotation
from django.db.models.functions import Now, TruncDay
Events.objects.annotate(time_left=TruncDay("start")-Now())
where "start" is the column holding date and time when the event starts.
Then if I print the "time_left" field of a resulting object I get wrong time which is bigger by my time zone shift.
How can I fix that?
Update.
I did an investigation and TruncDay seems to work right, but difference between TruncDay and Now gives incorrect answer. Interestingly, when I run the SQL query built by Django in the SQL console, it returns correct time interval.
Related
I'm using Django 2.2 and my question is: does transaction.atomic roll back increments to a pk sequence?
Below is the background bug I wrote up that led me to this issue
I'm facing a really weird issue that I can't figure out and I'm hoping someone has faced a similar issue.
An insert using the django ORM .create() function is returning django.db.utils.IntegrityError: duplicate key value violates unique constraint "my_table_pkey" DETAIL: Key (id)=(5795) already exists.
Fine. But then I look at the table and no record with id=5795 exists!
SELECT * from my_table where id=5795;
shows (0 rows)
A look at the sequence my_table_id_seq shows that it has nonetheless incremented to show last_value = 5795 as if the above record was inserted. Moreover the issue does not always occur. A successful insert with different data is inserted at id=5796. (I tried reset the pk sequence but that didn't do anything, since it doesnt seem to be the problem anyway)
I'm quite stumped by this and it has caused us a lot of issues on one specific table. Finally I realize the call is wrapped in transaction.atomic and that a particular scenario may be causing a double insert with the same pk.
So my theory is: The transaction atomic is not rolling back the increment of the
Postgres sequences do not roll back. Every time they are touched by a statement they advance whether the statement succeeds or not. For more information see Notes section here Create Sequence.
I want to work on a django project that work with time(track time of user), but I don't know to choose datetime field or timezone field or epoch time in an integer field for project model. I worked with all of them and I know we can convert them to each other. My question is for saving time in db, process on time and represent which one is recomended and why? What I need to consider for choosing between them?
The DateTimeField class would be my choice, since it supports timezones, something a simple IntegerField will not be able to do. If you don't handle timezones, there's a possibility that your computations will be wrong, and you will not be able to display dates and times that are correctly localized for your users.
Also, since a DateTimeField gets you a standard Python datetime, it works well for date computations, especially if you use a library like arrow to simplify their handling.
I have Django 1.9.5 installed with MariaDB 10.1.13 (also tested with MySQL 5.6.30) (which was updated from MySQL 5.5) and I am trying to get fractional seconds support in a date time field.
I have created a test model to try and get this working, here is the definition
class History(models.Model):
date = models.DateTimeField(null=True)
then in the shell I have run the following
History(date=datetime.now()).save()
and then when I query it
type(History.objects.get(id=1).date)
I get
<type 'NoneType'>
even though the entry appears in the database.
I can also use the field in a query
History.objects.all().order_by('date')
I know it works because inspecting the data shows that the order has changed
But I need to be able to return the date so I can compare it with another.
I was using the MySQL 5.5 without fractional support but there are records in my database that have the same datetime field and thus the order_by didnt work, I was using order_by id and that worked whilst the records were entered chronologically, but now this isnt the case I need fractional support.
Any ideas?
For this sort of situation it might be better to store the date and time as a unix timestamp in the database. There you millisecond accuracy. You can convert to python datetime objects at anytime with fromtimestamp.
Last but not least, you can easily convert unix timestamps with javascript to date times in the user's own timezone. A process that's far more complex with django or any other server side tech.
In my application I use time zones (USE_TZ=True) and all the dates I create in my code are aware UTC datetime objects (I use django.util.timezone.now for the current date and the following helper function to ensure all the dates in my instances are what I expect:)
#classmethod
def asUTCDate(cls, date):
if not timezone.is_aware(date):
return timezone.make_aware(date, timezone.utc)
return date.replace(tzinfo=timezone.utc)
I also enforced the check of naive/aware dates using this snippet (like suggested in the doc):
import warnings
warnings.filterwarnings(
'error', r"DateTimeField .* received a naive datetime",
RuntimeWarning, r'django\.db\.models\.fields')
As I understood so far, this is the right way to proceed (this is a quote from the django documentation: "The solution to this problem is to use UTC in the code and use local time only when interacting with end users."), and it seems that my app is handling dates very well… but I have just implemented a filter against a model that makes use of the Django 1.6 __hour and it force the extraction based on the user timezone, the result is something like:
django_datetime_extract('hour', "object"."date", Europe/Rome) = 15
but this breaks my query, since some results I was expecting are not included in the set, but when I use a __range to search between dates it seems to work as expected (objects with a date in the range are returned)… so it seems to me that Django takes into account timezones in queries only for the __hour filter… but I don't understand why… I was supposing that UTC is used everywhere except in templates where the displayed dates are formatted according to user tz, but maybe that's not true.
So my questions are: is the way I'm working with time zones right? Is __hour filter wrong or what?
It seems as though you're doing the right thing with dates. However, all the documentation for any date related functionality, such as filtering by hour include this note:
When USE_TZ is True, datetime fields are converted to the current time
zone before filtering.
For the range filter, this note doesn't exist because range can be used to not only filter on dates, but other types as well such as integers and characters. ie It is not necessarily datetime aware.
In essence the problem comes down to this: where do you draw the line between 'interacting with users' where times are in a local timezone, and what is internal where times are in UTC? In your case, you could imagine a user entering in a search box to search for hour==3. Does that mean for example that your form code should do the conversion between hour==3 and the UTC equivalent? This would then require a special forms.HourField. Or perhaps the value (3) should be fed directly to the query where we know that we're searching on an hour field and so a conversion is required.
We really have to follow the documentation on this one.
Values which are going to be filtered against date/time fields using any of the specialised date/time filtering functions will be treated as being in the user's local time zone.
If using the range filter for dates no time conversions occur so you are expected to convert the user's entered local time value to UTC.
I recently upgraded a Django project from 1.3 to 1.5 in order to start using the timezone handling. Because I am a moron, my Django timezone was set to "America/NewYork" instead of using UTC. Turning Django's timezone support on automatically sets Postgres to UTC. Now I'm running into issues with running direct SQL queries. I can't seem to get the timezone filtering correct. Here's what I'm doing:
Accepting a timestamp field from a user form
Swapping that to UTC in my view with stamp.astimezone(timezone('UTC'))
Passing that as a parameter (start.strftime('%Y-%m-%d %H:%M:00%z')) in a raw SQL query using a django.db.connection's cursor
The query that gets executed (logging to the console) looks correct:
SELECT to_char(created, 'YYYY-MM-DD HH12:MIam'),
COALESCE(heat_flow_1, 0.0) / 1000.0, COALESCE(heat_flow_2, 0.0) / 1000.0,
COALESCE(heat_flow_3, 0.0) / 1000.0
FROM results_flattenedresponse
WHERE installation_id = '66'
AND created BETWEEN TIMESTAMPTZ '2013-04-26 13:00:00+0000' AND TIMESTAMPTZ '2013-04-26 16:00:00+0000'
if I copy and paste that into PGAdmin, I get what I'm looking for, a set of rows starting at 9am EDT. However, the dataset that comes back from the django.db.connection's cursor has all of the dates pushed forward 4 hours (the difference between EDT and UTC). The rest of the data is actually correct, but the dates are being treated as UTC and pushed to the user's active timezone (even if I call deactivate). I feel like I have a mess of bad ideas wired together trying to fix this now, so I'm not sure what parts are good ideas and what are bad.
EDIT/ UPDATE
I've been trying a number of other things here and still getting bad results, but only when I query the database directly. The stuff in steps 2 and 3 above seems immaterial. The one quick fix I can see is actually setting the timezone in the query, e.g., SET timezone='America/New_York'; and then undoing it but that seems like a very bad idea for data integrity.
The other strange bit: I've set the Django timezone setting to UTC, but when I download a local copy and look at the data it's still marked as if it were set to America/New_York. I don't know if that's due to a setting on my local server or if there's a bug where the data isn't being localized properly by Django when it gets inserted into the second (non-default) database, though if that were the case I expect my problem would have gone away.
Since the server is now in UTC time, the created comes back as UTC by default, so the first bit needs to be
SELECT to_char(created AT TIME ZONE %s, 'YYYY-MM-DD HH12:MIam')
and pass in the timezone you want to see. This isn't the best approach in the world, but it works for me here because the queries are all run inside an object that has the relevant timezone as a property.