With inspectdb I was able to get a "interval" field from postgres into django. In Django, it was a TextField. The object that I retrieved was indeed a timedelta object!
Now I want to put this timedelta object in a new model. What's the best way to do this? Because putting a timedelta in a TextField results in the str version of the object...
Since Django 1.8 you can use DurationField.
You can trivially normalize a timedelta to a single floating-point number in days or seconds.
Here's the "Normalize to Days" version.
float(timedelta.days) + float(timedelta.seconds) / float(86400)
You can trivially turn a floating-point number into a timedelta.
>>> datetime.timedelta(2.5)
datetime.timedelta(2, 43200)
So, store your timedelta as a float.
Here's the "Normalize to Seconds" version.
timedelta.days*86400+timedelta.seconds
Here's the reverse (using seconds)
datetime.timedelta( someSeconds/86400 )
First, define your model:
class TimeModel(models.Model):
time = models.FloatField()
To store a timedelta object:
# td is a timedelta object
TimeModel.objects.create(time=td.total_seconds())
To get the timedelta object out of the database:
# Assume the previously created TimeModel object has an id of 1
td = timedelta(seconds=TimeModel.objects.get(id=1).time)
Note: I'm using Python 2.7 for this example.
https://bitbucket.org/schinckel/django-timedelta-field/src
There is a ticket which dates back to July 2006 relating to this:
https://code.djangoproject.com/ticket/2443
Several patches were written but the one that was turned in to a project:
https://github.com/johnpaulett/django-durationfield
Compared to all the other answers here this project is mature and would have been merged to core except that its inclusion is currently considered to be "bloaty".
Personally, I've just tried a bunch of solutions and this is the one that works beautifully.
from django.db import models
from durationfield.db.models.fields.duration import DurationField
class Event(models.Model):
start = models.DateTimeField()
duration = DurationField()
#property
def finish(self):
return self.start + self.duration
Result:
$ evt = Event.objects.create(start=datetime.datetime.now(), duration='1 week')
$ evt.finish
Out[]: datetime.datetime(2013, 6, 13, 5, 29, 29, 404753)
And in admin:
Change event
Duration: 7 days, 0:00:00
For PostgreSQL, use django-pgsql-interval-field here: http://code.google.com/p/django-pgsql-interval-field/
Putting this out there cause it might be another way to solve this problem.
first install this library: https://pypi.python.org/pypi/django-timedeltafield
Then:
import timedelta
class ModelWithTimeDelta(models.Model):
timedeltafield = timedelta.fields.TimedeltaField()
within the admin you will be asked to enter data into the field with the following format: 3 days, 4 hours, 2 minutes
There is a workaround explained here. If you're using Postgresql, then multiplying the result of F expression with timedelta solves the problem. For example if you have a start_time and a duration in minutes, you can calculate the end_time like this:
YourModel.objects.annotate(
end_time=ExpressionWrapper(F('start_time') + timedelta(minutes=1) * F('duration'), output_field=DateTimeField())
)
Related
I am trying to create manager method inside my app, to filter emails object, that have been created 5/10/15 minutes or what so ever, counting exactly from now.
I though I'am gonna use annotate to create new parameter, which will be bool and his state depends on simple subtraction with division and checking if the result is bigger than 0.
from django.db.models import F
from django.utils import timezone
delta = 60 * 1 * 5
current_date = timezone.now()
qs = self.annotate(passed=((current_date - F('created_at')).seconds // delta > 0)).filter(passed=True)
Atm my error says:
AttributeError: 'CombinedExpression' object has no attribute 'seconds'
It is clearly happening duo the fact, that ((current_date - F('created_at')) does not evaluate to datetime.timedelta object but to the CombinedExpression object.
I see more problems out there, i.e. how to compare the expression to 0?
Anyway, would appreciate any tips if I am somewhere close to achieve my goal or is my entire logic behind this query incorrect
Well, I managed to find the solution, even though it might not be the elegant one, it works
qs = self.annotate(foo=Sum(current_date - F('created_at'))).filter(foo__gt=Sum(timezone.timedelta(seconds=delta)))
Why not something like this:
time_cut_off = timezone.now() - timezone.timedelta(minutes=delta)
qs = self.filter(created_at__gte=time_cut_off)
This will get you the messages created in the last delta minutes. Or where you looking for messages created exactly 5 minutes ago (how do you define that if that is the question).
The documentation provides a simple and elegant solution if your timedelta is a constant :
For date and date/time fields, you can add or subtract a timedelta object. The following would return all entries that were modified more than 3 days after they were published:
>>> from datetime import timedelta
>>> Entry.objects.filter(mod_date__gt=F('pub_date') + timedelta(days=3))
In your case I don't think you even need the F() objects.
I have the following model:
class Assignment(models.Model):
extra_days = models.IntegerField(default=0)
due_date = models.DateTimeField()
Where due_date is the date the assignment is due and extra_days is the number of extra days given after the due date to finish the assignment.
I want to create a query that returns all rows where due_date + extra_days is greater than the current date. Here's what I am doing:
from django.utils import timezone
from django.db.models import F
from datetime import datetime
cur_date = timezone.make_aware(datetime.now(), timezone.get_default_timezone())
a = Assignment.objects.filter(extra_days__gt=cur_date - F('due_date'))
When I print a, I get the following error:
File "c:\Python27\lib\site-packages\MySQLdb\cursors.py", line 204, in execute
if not self._defer_warnings: self._warning_check()
File "c:\Python27\lib\site-packages\MySQLdb\cursors.py", line 117, in _warning
_check
warn(w[-1], self.Warning, 3)
Warning: Truncated incorrect DOUBLE value: '2013-09-01 02:54:31'
If I do a time difference that results in, say, 3.1 days, I'm assuming the days difference would be still be 3. I think it would more correct to do something like this:
a = Assignment.objects.filter(due_date__gt=cur_date - timedelta(days=F('extra_days')))
But that also results in an error.
How can I do this without writing a raw SQL query?
This depends on the database backend you are using, which seems to be PostgreSQL.
PostgreSQL can subtract dates directly, so the following will work:
from django.db.models import F, Func
from django.db.models.functions import Now
class DaysInterval(Func):
function = 'make_interval'
template = '%(function)s(days:=%(expressions)s)'
qs = Assignment.objects.annotate(remaining_days=F('due_date') - Now())
qs.filter(remaining_days__lt=DaysInterval(F('extra_days')))
This results in the following SQL:
SELECT "assignments_assignment"."id",
"assignments_assignment"."extra_days",
"assignments_assignment"."due_date",
("assignments_assignment"."due_date" - STATEMENT_TIMESTAMP()) AS "remaining_days"
FROM "assignments_assignment"
WHERE ("assignments_assignment"."due_date" - STATEMENT_TIMESTAMP())
< (make_interval(DAYS:="assignments_assignment"."extra_days"))
For date difference calculations in other database backends see the Datediff function created by Michael Brooks.
It seems like what I'm trying to do is not possible. I ended up writing a raw query:
cursor.execute("SELECT * FROM app_assignment WHERE DATE_ADD(due_date, INTERVAL extra_days DAYS) > utc_timestamp()")
I was so repulsed at not being able to use the ORM for doing something so seemingly simple that I considered trying out SQLAlchemy, but a raw query works fine. I always tried workarounds to make sure I could use the ORM, but I'll use raw SQL going forwards for complex queries.
As far as I know , you can not pass an F() object as a params to another function since F() base class is a tree.Node type, A class for storing a tree graph which primarily used for filter constructs in the ORM.
see F() define at django/db/models/expression.py and Node at django/utils/tree.py (django 1.3.4)
class ExpressionNode(tree.Node):
...
class F(ExpressionNode):
"""
An expression representing the value of the given field.
"""
def __init__(self, name):
super(F, self).__init__(None, None, False)
self.name = name
def __deepcopy__(self, memodict):
obj = super(F, self).__deepcopy__(memodict)
obj.name = self.name
return obj
def prepare(self, evaluator, query, allow_joins):
return evaluator.prepare_leaf(self, query, allow_joins)
def evaluate(self, evaluator, qn, connection):
return evaluator.evaluate_leaf(self, qn, connection)
you can do something like
Assignment.objects.filter(due_date__gt=F('due_date') - timedelta(days=1))
but not
Assignment.objects.filter(due_date__gt=cur_date - timedelta(days=F('extra_days')))
Correct me if i was wrong. Hope this little help.
Just in case anyone else looks for this, here's something that might be worth looking into.
I'm using Django 1.4 and am running into the exact same issue as the OP. Seems that the issue is probably due to timedelta and datetime needing to evaluate before being sent to the database, but the F object is inherently only going to resolve in the database.
I noticed that in Django 1.8, a new DurationField was introduced that looks like it would directly work like python's timedelta . This should mean that instead of needing to take the timedelta of an F object look up on an IntegerField, one could theoretically use a DurationField and then the F object wouldn't need to be in a timedelta at all. Unfortunately, due to dependencies, I'm not currently able to upgrade my project to 1.8 and test this theory.
If anyone else encounters this problem and is able to test my suggestion, I'd love to know. If I resolve my dependencies and can upgrade to 1.8, then I'll be sure to post back with my results.
I have a table containing a series of entries which relate to time periods (specifically, time worked for a client):
task_time:
id | start_time | end_time | client (fk)
1 08/12/2011 14:48 08/12/2011 14:50 2
I am trying to aggregate all the time worked for a given client, from my Django app:
time_worked_aggregate = models.TaskTime.objects.\
filter(client = some_client_id).\
extra(select = {'elapsed': 'SUM(task_time.end_time - task_time.start_time)'}).\
values('elapsed')
if len(time_worked_aggregate) > 0:
time_worked = time_worked_aggregate[0]['elapsed'].total_seconds()
else:
time_worked = 0
This seems inelegant, but it does work. Or at least so I thought: it turns out that it works fine on a PostgreSQL database, but when I move over to SQLite, everything dies.
A bit of digging suggests that the reason for this is that DateTimes aren't first-class data in SQLite. The following raw SQLite query will do my job:
SELECT SUM(strftime('%s', end_time) - strftime('%s', start_time)) FROM task_time WHERE ...;
My question is as follows:
The Python sample above seems roundabout. Can we do this more elegantly?
More importantly at this stage, can we do it in a way that will work on both Postgres and SQLite? Ideally, I'd like not to be writing raw SQL queries and switching on the database backend that happens to be in place; in general, Django is extremely good at protecting us from this. Does Django have a reasonable abstraction for this operation? If not, what's a sensible way for me to do a conditional switch on the backend?
I should mention for context that the dataset is many thousands of entries; the following is not really practical:
sum([task_time.end_date - task_time.start_date for task_time in models.TaskTime.objects.filter(...)])
Almost the same solution as #andri proposed. In the final result you will get the same data.
ExpressionWrapper - New in Django 1.8.
from datetime import timedelta
from django.db.models import ExpressionWrapper, F, fields
from app.models import MyModel
duration = ExpressionWrapper(F('closed_at') - F('opened_at'), output_field=fields.DurationField())
objects = MyModel.objects.closed().annotate(duration=duration).filter(duration__gt=timedelta(seconds=2))
for obj in objects:
print obj.id, obj.duration, obj.duration.seconds
# sample output
# 807 0:00:57.114017 57
# 800 0:01:23.879478 83
# 804 3:40:06.797188 13206
# 801 0:02:06.786300 126
I think since Django 1.8 we can do better:
I would like just to draw the part with annotation, the further part with aggregation should be straightforward:
from django.db.models import F, Func
SomeModel.objects.annotate(
duration = Func(F('end_date'), F('start_date'), function='age')
)
[more about postgres age function here: http://www.postgresql.org/docs/8.4/static/functions-datetime.html ]
each instance of SomeModel will be anotated with duration field containg time difference, which in python will be a datetime.timedelta() object [more about datetime timedelta here: https://docs.python.org/2/library/datetime.html#timedelta-objects ]
I will do it step by step:
first step:annotate the timedelta
group by and sum timedelta
the code like this:
from django.db.models import Count, Sum, F
times_obj_list = models.TaskTime.objects.annotate(times=F("end_time")-F("start_time"))
groupby_obj_list = times_obj_list.values("client").annotate(cnt=Count("id"),seconds=Sum(times)).order_by()
Django currently only supports aggregates for Min, Max, Avg and Count, so using raw SQL is the only way to achieve what you want. When you use raw SQL, database-independence is out the window, so unfortunately, you're out of luck. You'll have to just detect the database and alter the SQL appropriately.
I'm trying to find the cumulated duration of some events, 'start' and 'end' field are both django.db.models.DateTimeField fields.
What I would like to do should have been written like this:
from django.db.models import F, Sum
from my.models import Event
Event.objects.aggregate(anything=Sum(F('start') - F('end')))
# this first example return:
# AttributeError: 'ExpressionNode' object has no attribute 'split'
# Ok I'll try more SQLish:
Event.objects.extra(select={
'extra_field': 'start - end'
}).aggregate(Sum('extra_field'))
# this time:
# FieldError: Cannot resolve keyword 'extra_field' into field.
I can't agreggate (Sum) start and end separately then substract in python because DB can't Sum DateTime objects.
A good way to do without raw sql?
Can't help Christophe without a Delorean, but I was hitting this error and was able to solve it in Django 1.8 like:
total_sum = Event.objects\
.annotate(anything=Sum(F('start') - F('end')))\
.aggregate(total_sum=Sum('anything'))['total_sum']
When I couldn't upgrade all my dependencies to 1.8, I found this to work with Django 1.7.9 on top of MySQL:
totals = self.object_list.extra(Event.objects.extra(select={
'extra_field': 'sum(start - end)'
})[0]
If you are on Postgres, then you can use the django-pg-utils package and compute in the database. Cast the duration field into seconds and then take the sum
from pg_utils import Seconds
from django.db.models import Sum
Event.objects.aggregate(anything=Sum(Seconds(F('start') - F('end'))))
This answer don't realy satisfy me yet, my current work around works but it's not DB computed...
reduce(lambda h, e: h + (e.end - e.start).total_seconds(), events, 0)
It returns the duration of all events in the queryset in seconds
Better SQL less solutions?
I have the following model:
class Process(models.Model):
title = models.Charfield(max_length=255)
date_up = models.DateTimeField(auto_now_add=True)
days_activation = models.PositiveSmallIntegerField(default=0)
Now I need to query for all Process objects that have expired, according to their value of days_activation.
I tried
from datetime import datetime, timedelta
Process.objects.filter(date_up__lte=datetime.now()-timedelta(days=F('days_activation')))
and received the following error message:
TypeError: unsupported type for timedelta days component: F
I can of course do it in Python:
filter (lambda x: x.date_up<=datetime.now() - timedelta(days=x.days_activation),
Process.objects.all ()),
but I really need to produce a django.db.models.query.QuerySet.
7 days == 1 day * 7
F is deep-black Django magic and the objects that encounter it
must belong to the appropriate magical circles to handle it.
In your case, django.db.models.query.filter knows about F, but datetime.timedelta does not.
Therefore, you need to keep the F out of the timedelta argument list.
Fortunately, multiplication of timedelta * int is supported by F,
so the following can work:
Process.objects.filter(date_up__lte=datetime.now()-timedelta(days=1)*F('days_activation'))
As it turns out, this will work with PostgreSQL, but will not work with SQlite (for which Django 1.11 only supports + and - for timedelta,
perhaps because of a corresponding SQlite limitation).
You are mixing two layers: run-time layer and the database layer. F function is just a helper which allows you to build slightly more complex queries with django ORM. You are using timedelta and Ftogether and expecting that django ORM will be smart enough to convert these things to raw SQL, but it can't, as I see. Maybe I am wrong and do not know something about django ORM.
Anyway, you can rewrite you ORM call with extra extra and build the WHERE clause manually using native SQL functions which equals to datetime.now() and timedelta.
You have to extend Aggregate. Do like below:
from django.db import models as DM
class BaseSQL(object):
function = 'DATE_SUB'
template = '%(function)s(NOW(), interval %(expressions)s day)'
class DurationAgr(BaseSQL, DM.Aggregate):
def __init__(self, expression, **extra):
super(DurationAgr, self).__init__(
expression,
output_field=DM.DateTimeField(),
**extra
)
Process.objects.filter(date_up__lte=DurationAgr('days_activation'))
Hopefully, It will work for you. :)
I tried to use solution by Lutz Prechelt above, but got MySQL syntax error.
It's because we can't perform arithmetic operations with INTERVAL in MySQL.
So, for MySQL my solution is create a custom DB function:
class MysqlSubDate(Func):
function = 'SUBDATE'
output_field = DateField()
Example of usage:
.annotate(remainded_days=MysqlSubDate('end_datetime', F('days_activation')))
Also you can use timedelta, it will be converted into INTERVAL
.annotate(remainded_days=MysqlSubDate('end_datetime', datetime.timedelta(days=10)))