I have a table, lets call it as DummyTable.
It has fields - price_effective, store_invoice_updated_date, bag_status, gstin_code.
Now I want to get the output which does a group by of - month, year from the field store_invoice_updated_date and gstin_code.
Along with that group by I wanna do thse calculations -
Sum of price_effective as 'forward_price_effective' if the bag_status is other than 'return_accepted' or 'rto_bag_accepted'. Dont know how to do an exclude here i.e. using a filter in annotate
Sum of price effective as 'return_price_effective' if the bag_status is 'return_accepted' or 'rto_bag_accepted'.
A field 'total_price' that subtracts the 'return_price_effective' from 'forward_price_effective'.
I have formulated this query, which doesn't work
from django.db.models.functions import TruncMonth
from django.db.models import Count, Sum, When, Case, IntegerField
DummyTable.objects.annotate(month=TruncMonth('store_invoice_updated_date'), year=TruncYear('store_invoice_updated_date')).annotate(forward_price_effective=Sum(Case(When(bag_status__in=['delivery_done']), then=Sum(forward_price_effective)), output_field=IntegerField()), return_price_effective=Sum(Case(When(bag_status__in=['return_accepted', 'rto_bag_accepted']), then=Sum('return_price_effective')), output_field=IntegerField())).values('month','year','forward_price_effective', 'return_price_effective', 'gstin_code')
Solved it by multiple querysets.
Just couldnt find out a way to appropriately use 'Case' with 'When' with 'filter' and 'exclude'.
basic_query = BagDetails.objects.filter(store_invoice_updated_date__year__in=[2018]).annotate(month=TruncMonth('store_invoice_updated_date'), year=TruncYear('store_invoice_updated_date') ).values('year', 'month', 'gstin_code', 'price_effective', 'company_id', 'bag_status')
forward_bags = basic_query.exclude(bag_status__in=['return_accepted', 'rto_bag_accepted']).annotate(
Sum('price_effective')).values('year', 'month', 'gstin_code', 'price_effective', 'company_id')
return_bags = basic_query.filter(bag_status__in=['return_accepted', 'rto_bag_accepted']).annotate(
Sum('price_effective')).values('month', 'gstin_code', 'price_effective', 'company_id')
Related
In use: django 3.2.10, postgresql 13.4
I have next query set with aggregation function Count
queryset = Model.objects.all().aggregate(
trues=Count('id', filter=Q(criteria=True)),
falses=Count('id', filter=Q(criteria=False)),
)
What I want:
queryset = Model.objects.all().aggregate(
trues=Count('id', filter=Q(criteria=True)),
falses=Count('id', filter=Q(criteria=False)),
total=trues+falses, <--------------THIS
)
How to do this?
There is little thing you can do after aggregation, as it returns a python dict object.
I do understand your example here is not your real situation, as you can simply do
Model.objects.aggregate(
total = (Count('id', filter=Q(criteria=True))
+ Count('id', filter=Q(criteria=False)))
)
What I want to say is Django provides .values().annotate() to achieve GROUP BY clause as in sql language.
Take your example here
queryset = Model.objects.values('criteria').annotate(count=Count('id'))
queryset here is still a 'QuerySet' object, and you can further modify the queryset like
queryset = queryset.aggregate(
total=Sum('count')
)
Hopefully it helps.
it seems you want the total number of false and true criteria so you can simply do as follow
queryset = Model.objects.all().filter(
Q(criteria=True) | Q(criteria=False)).count()
or you can use (not recommended except you want to show something in the middle)
from django.db.models import Avg, Case, Count, F, Max, Min, Prefetch, Q, Sum, When
query = Model.objects.annotate(trues=Count('id',filter=Q(criteria=True)),
falses=Count('id',filter=Q(criteria=False))).annotate(trues_false=F('trues')+F('falses')).aggregate(total=Sum('trues_false'))
I'm trying to perform a query in django that is equivalent to this:
SELECT SUM(quantity * price) from Sales GROUP BY date.
My django query looks like this:
Sales.objects.values('date').annotate(total_sum=Sum('price * quantity'))
The above one throws error:
Cannot resolve keyword 'price * quantity' into field
Then I came up with another query after checking this https://stackoverflow.com/a/18220269/12113049
Sales.objects.values('date').annotate(total_sum=Sum('price', field='price*quantity'))
Unfortunately, this is not helping me much. It gives me SUM(price) GROUP BY date instead of SUM(quantity*price) GROUP BY date.
How do I query this in django?
You should be using F expressions to perform operations on fields:
from django.db.models import F
Sales.objects.values('date').annotate(total_sum=Sum(F('price') * F('quantity')))
Edit: assuming that price is a DecimalField and quantity is a IntegerField (of different types) you would need to specify output_field in Sum:
from django.db.models import DecimalField, F
Sales.objects.values('date').annotate(total_sum=Sum(F('price') * F('quantity'), output_field=DecimalField()))
Question is regarding custom transform or lookup( can’t figure it out yet).
For example I have following model.
from django.contrib.postgres.fields import DateRangeField
class Example(models.Model).
date_range = DateRangeField()
Each date_range is a object with start and stop dates, for example [2020-02-24, 2020-04-16)
Question is – is it possible to create a transform or(and) lookup in order to filter instances of model by their range between start date and stop date?
Example
I want to find instances where difference between start and stop date would be more then 1 year.
This would be something like
True - [2020-02-24, 2021-04-16) - delta more then one year
False - [2020-02-24, 2020-04-16) - delta less then one year
Example.objects.filter(date_range__transform_name_here__gt=365)
I can do it via raw SQL but I don’t want to use it as it is quite common task.
Thank you.
I know it's probably a bit late to help you, but you should be able to do this to achieve your goal:
from django.db.models import DurationField, ExpressionWrapper, F
from django.db.models.functions import Lower, Upper
from django.utils import timezone
greater_than_a_year = Example.objects.all().annotate(
delta=ExpressionWrapper(
F('date_range__endswith') - F('date_range__startswith'), output_field=DurationField()
).filter(
delta__gt=timezone.timedelta(days=365)
)
Here we annotate by subtracting the lower portion of the range from the upper, and output as a DurationField() with the name delta. Then we simply filter on delta like any other DurationField.
clicks = SellerClick.objects.extra({'date' : "date(timestamp)"}).values('date').annotate(count=Count('timestamp'))
The model has a datetime field called timestamp that was are using. I first, convert the datetime field to just a date field. Then the rest is guessing. I need to group by, and then count how many objects are of each date.
So the desired result would be a date, then a count, based on how many objects have that date in the timestamp field.
I prefer to use annotate over extra
from django.db.models.expressions import RawSQL
SellerClick.objects.annotate(
date=RawSQL('date(date_joined)',[]),
).values('date').annotate(count=Count('date')))
You've got everything but an initial queryset there. The extra sql you're passing doesn't include a select so you need to give it something to act on.
clicks = SellerClick.objects.all()
.extra({'date' : "date(timestamp)"})
.values('date')
.annotate(count=Count('timestamp'))
Ref: StackOverflow: Count number of records by date in Django
I've got a simple Model like this:
class Order(models.Model):
created = model.DateTimeField(auto_now_add=True)
total = models.IntegerField() # monetary value
And I want to output a month-by-month breakdown of:
How many sales there were in a month (COUNT)
The combined value (SUM)
I'm not sure what the best way to attack this is. I've seen some fairly scary-looking extra-select queries but my simple mind is telling me I might be better off just iterating numbers, starting from an arbitrary start year/month and counting up until I reach the current month, throwing out simple queries filtering for that month. More database work - less developer stress!
What makes most sense to you? Is there a nice way I can pull back a quick table of data? Or is my dirty method probably the best idea?
I'm using Django 1.3. Not sure if they've added a nicer way to GROUP_BY recently.
Django 1.10 and above
Django documentation lists extra as deprecated soon. (Thanks for pointing that out #seddonym, #Lucas03). I opened a ticket and this is the solution that jarshwah provided.
from django.db.models.functions import TruncMonth
from django.db.models import Count
Sales.objects
.annotate(month=TruncMonth('created')) # Truncate to month and add to select list
.values('month') # Group By month
.annotate(c=Count('id')) # Select the count of the grouping
.values('month', 'c') # (might be redundant, haven't tested) select month and count
Older versions
from django.db import connection
from django.db.models import Sum, Count
truncate_date = connection.ops.date_trunc_sql('month', 'created')
qs = Order.objects.extra({'month':truncate_date})
report = qs.values('month').annotate(Sum('total'), Count('pk')).order_by('month')
Edits
Added count
Added information for django >= 1.10
Just a small addition to #tback answer:
It didn't work for me with Django 1.10.6 and postgres. I added order_by() at the end to fix it.
from django.db.models.functions import TruncMonth
Sales.objects
.annotate(month=TruncMonth('timestamp')) # Truncate to month and add to select list
.values('month') # Group By month
.annotate(c=Count('id')) # Select the count of the grouping
.order_by()
Another approach is to use ExtractMonth. I ran into trouble using TruncMonth due to only one datetime year value being returned. For example, only the months in 2009 were being returned. ExtractMonth fixed this problem perfectly and can be used like below:
from django.db.models.functions import ExtractMonth
Sales.objects
.annotate(month=ExtractMonth('timestamp'))
.values('month')
.annotate(count=Count('id'))
.values('month', 'count')
metrics = {
'sales_sum': Sum('total'),
}
queryset = Order.objects.values('created__month')
.annotate(**metrics)
.order_by('created__month')
The queryset is a list of Order, one line per month, combining the sum of sales: sales_sum
#Django 2.1.7
Here's my dirty method. It is dirty.
import datetime, decimal
from django.db.models import Count, Sum
from account.models import Order
d = []
# arbitrary starting dates
year = 2011
month = 12
cyear = datetime.date.today().year
cmonth = datetime.date.today().month
while year <= cyear:
while (year < cyear and month <= 12) or (year == cyear and month <= cmonth):
sales = Order.objects.filter(created__year=year, created__month=month).aggregate(Count('total'), Sum('total'))
d.append({
'year': year,
'month': month,
'sales': sales['total__count'] or 0,
'value': decimal.Decimal(sales['total__sum'] or 0),
})
month += 1
month = 1
year += 1
There may well be a better way of looping years/months but that's not really what I care about :)
Here is how you can group data by arbitrary periods of time:
from django.db.models import F, Sum
from django.db.models.functions import Extract, Cast
period_length = 60*15 # 15 minutes
# Annotate each order with a "period"
qs = Order.objects.annotate(
timestamp=Cast(Extract('date', 'epoch'), models.IntegerField()),
period=(F('timestamp') / period_length) * period_length,
)
# Group orders by period & calculate sum of totals for each period
qs.values('period').annotate(total=Sum(field))
By month:
Order.objects.filter().extra({'month':"Extract(month from created)"}).values_list('month').annotate(Count('id'))
By Year:
Order.objects.filter().extra({'year':"Extract(year from created)"}).values_list('year').annotate(Count('id'))
By day:
Order.objects.filter().extra({'day':"Extract(day from created)"}).values_list('day').annotate(Count('id'))
Don't forget to import Count
from django.db.models import Count
For django < 1.10
i have orders table in my database . i am going to count orders per month in the last 3 months
from itertools import groupby
from dateutil.relativedelta import relativedelta
date_range = datetime.now()-relativedelta(months=3)
aggs =Orders.objects.filter(created_at=date_range)\
.extra({'date_created':"date(created_at)"}).values('date_created')
for key , group in groupby(aggs):
print(key,len(list(group)))
created_at is datetime field. by extra function what done is taking date from datetime values. when using datetime we may not get the count correct because objects are created at different time in a day.
The for loop will print date and number of count