I am writing queryset that will return this type
date
total_shipping_fee
2021-04-16
5,000
2021-04-17
100,000
where
class Payments(models.Model):
created = models.DateTimeField()
.....
SELECT DATE(created) from payment_payment .... group by 'created' -> outputs a correct query
My question is how to query/ or cast
Payment.objects.filter(created_range=("2021-05-14", "2021-05-14")).values('created').annotate(total_shipping_fee=Sum('total_shipping_fee'))
so that I can have queryset in above raw sql. I think that is my problem to CAST DATE(created) in django queryset. Thanks
You can work with:
from django.db.models import F, Sum
Payment.objects.filter(
created__date_range=('2021-05-14', '2021-05-14')
).values(create_date=F('created__date')).annotate(
total_shipping_fee=Sum('total_shipping_fee')
).order_by('create_date')
here we thus first takes as values the truncated date of the DateTimeField, and then we use .order_by(…) [Django-doc] to enforce grouping by that date.
The output is a QuerySet of dictionaries, each with two items: { 'create_date': …, 'total_shipping_fee': … }.
Related
I have a model like this
class MyModel(models.Model):
date = models.DatetimeField(auto_now_add=True)
and i have a list of dates (dates of last seven days, it can be dates of last 30 days as well). What i am trying to do is get the number of MyModel objects created in a particular day in all of the dates. Like below.
[{'date1': 2}, {'date2': 3}, {'date3': 7}....{'daten':26}]
the keys of the dictoary are dates and values are the number of objects created in that particular date. by using a forloop and hitting db multiple times i am getting the output, but is there a way to get it from a single query ?
You can make a query that counts the number of objects for each date:
from datetime import timedelta
from django.db.models import Count, F
from django.utils.timezone import now
qs = MyModel.objects.filter(
date__date__gte=now().date()-timedelta(days=30)
).values(
datedate=F('date__date')
).annotate(
number=Count('pk')
).order_by('datedate')
Next we can post-process this to a list of dictionaries, or perhaps better a single dictionary that maps the data on the count:
# list of dictionaries
[{r['datedate']: r['number']} for r in qs]
# single dictionary
{r['datedate']: r['number'] for r in qs}
If the date is not present, it will not be in the dictionary. So dates that are not in the dictionary have count 0. This is due to the closed world assumption (CWA).
django 2.0.2 python 3.4
models.py
Post(models.Model):
Id = pk
content = text
Reply(models.Model):
Id = pk
PostId = Fk(Post)
content = text
view.py
Post.objects.all().annotate(lastreply=F("Reply__content__last"))
can use last query in F() ?
As far as I know, latest cannot be used with F().
One possible solution is including a timestamp in the reply class
Post(models.Model):
Id = pk
content = text
Reply(models.Model):
Id = pk
PostId = Fk(Post)
content = text
timestamp = DateTime(auto)
Then you can use a query of this format to get the latest reply for each post.
Reply.objects.annotate(max_time=Max('Post__Reply__timestamp')).filter(timestamp=F('max_time'))
Please note that this is really time consuming for large number of records.
If you are using a Postgres DB you can use distinct()
Reply.objects.order_by('Post__Id','-timestamp').distinct('Post__Id')
F expression has no way to do that.
but Django has another way to handle it.
https://docs.djangoproject.com/en/2.0/ref/models/expressions/#subquery-expressions
for this problem, the code below can solve this:
from django.db.models import OuterRef, Subquery
sub_qs = Reply.objects.filter(
PostId=OuterRef('pk')
).order_by('timestamp')
qs = Post.objects.annotate(
last_reply_content=Subquery(
sub_qs.values('content')[:1]))
how does it work?
sub_qs is the related model queryset, where you want to take only the last reply for each post, to do that, we use the OuterRef, it will take care to get replies related to this post, and finally the order_by that will order by the timestamp, the first is the most recent, and the last is the eldest.
sub_qs = Reply.objects.filter(
PostId=OuterRef('pk')
).order_by('timestamp')
the second part is the Post queryset with a annotate, we wanna apply the sub_qs in an extra field, and using subquery will allow us to insert another queryset inside of annotate
we use .values('content') to get only the content field, and slice the sub_qs with [:1] to get only the first occurrence.
qs = Post.objects.annotate(
last_reply_content=Subquery(
sub_qs.values('content')[:1]))
I want to get a list of max ids for a filter I have in Django
class Foo(models.Model):
name = models.CharField()
poo = models.CharField()
Foo.objects.filter(name__in=['foo','koo','too']).latest_by_id()
End result a queryset having only the latest objects by id for each name. How can I do that in Django?
Edit: I want multiple objects in the end result. Not just one object.
Edit1: Added __in. Once again I need only latest( as a result distinct) objects for each name.
Something like this.
my_id_list = [Foo.objects.filter(name=name).latest('id').id for name in ['foo','koo','too']]
Foo.objects.filter(id__in=my_id_list)
The above works. But I want a more concise way of doing it. Is it possible to do this in a single query/filter annotate combination?
you can try:
qs = Foo.objects.filter(name__in=['foo','koo','too'])
# Get list of max == last pk for your filter objects
max_pks = qs.annotate(mpk=Max('pk')).order_by().values_list('mpk', flat=True)
# after it filter your queryset by last pk
result = qs.filter(pk__in=max_pks)
If you are using PostgreSQL you can do the following
Foo.objects.order_by('name', '-id').distinct('name')
MySQL is more complicated since is lacks a DISTINCT ON clause. Here is the raw query that is very hard to force Django to generate from ORM function calls:
Foo.objects.raw("""
SELECT
*
FROM
`foo`
GROUP BY `foo`.`name`
ORDER BY `foo`.`name` ASC , `foo`.`id` DESC
""")
I have a django model that has a date field and a separate time field. I am trying to use a filter to find a value on the latest record by date/time that is less than the current record's date time.
How do I use annotate/aggregate to combine the date and time fields into one and then do a filter on it?
models.py
class Note(models.model):
note_date = models.DateField(null=True)
note_time = models.TimeField(null=True)
note_value = models.PositiveIntegerField(null=True)
def get_last(n):
"""
n: Note
return: Return the note_value of the most recent Note prior to given Note.
"""
latest = Note.objects.filter(
note_date__lte=n.note_date
).order_by(
'-note_date', '-note_time'
).first()
return latest.note_value if latest else return 0
This will return any notes from a previous date, but if I have a two notes on the same date, one at 3pm and one at 1pm, and I send the 3pm note to the function, I want to get the value of the 1pm note. Is there a way to annotate the two fields into one for comparison, or do I have to perform a raw SQL query? Is there a way to convert the date and time component into one, similar to how you could use Concat for strings?
Note.objects.annotate(
my_dt=Concat('note_date', 'note_time')
).filter(
my_dt__lt=Concat(models.F('note_date'), models.F('note_time')
).first()
I am too late but here is what I did
from django.db.models import DateTimeField, ExpressionWrapper, F
notes = Note.objects.annotate(my_dt=ExpressionWrapper(F('note_date') + F('note_time'), output_field=DateTimeField()))
Now we have added a new field my_dt of datetime type and can add a filter further to do operations
Found an answer using models.Q here: filter combined date and time in django
Note.objects.filter(
models.Q(note_date__lt=n.note_date) | models.Q(
note_date=n.note_date,
note_time__lt=n.note_time
)
).first()
I guess I just wasn't searching by the right criteria.
Here is another Approach which is more authentic
from django.db.models import Value, DateTimeField
from django.db.models.functions import Cast, Concat
notes = Note.objects.annotate(my_dt=Cast(
Concat('note_date', Value(" "), 'note_time', output_field=DateTimeField()),
output_field=DateTimeField()
).filter(my_dt__lte=datetime.now())
Here is another solution following others.
def get_queryset(self):
from django.db import models
datetime_wrapper = models.ExpressionWrapper(models.F('note_date') + models.F('note_time'), output_field=models.DateTimeField())
return Note.objects.annotate(
note_datetime=datetime_wrapper
).filter(note_datetime__gt=timezone.now()).order_by('note_datetime')
Is it possible to annotate with complex value?
Like if I have table
class Test(models.model):
value = models.PositiveIntegerField(_('value'))
next = 5
import math
Test.objects.annotate(new_field=math.sqrt(next-value)/math.atan(value))
No, you can't pass math functions into annotate().
If you want to do this calculation in Test model then create a method:
class Test(models.model):
value = models.PositiveIntegerField(_('value'))
def calc_value(self, next):
return math.sqrt(next-self.value)/math.atan(self.value))
for t in Test.objects.all():
print t.value. t.calc_value(5)
But if you want to use this calculation to order the queryset then you have to do the math at SQL level:
next = 5
Test.objects.extra(select={'new_field': 'SQRT(%d-value)/ATAN(value)' % next}) \
.order_by('new_field'))
To filter the queryset by new field use where argument of the same extra() method:
Test.objects.extra(select={'new_field': 'SQRT(%d-value)/ATAN(value)' % next},
where=['new_field > 10'])
SQLite doesn't support math functions by default but with Postgres and MySQL this code should work just fine.
No, annotations can only be done on django aggregations.
Annotates each object in the QuerySet with the provided list of aggregate values (averages, sums, etc) that have been computed over the objects that are related to the objects in the QuerySet.