I have this model for employee overtime hours
class Overtime(IncomeBase):
day = models.DateField(verbose_name="Date")
value = models.FloatField(default=1)
I need to extract the total value for each month. Now I am using a daily QuerySet in the manager.
class OvertimeManager(models.Manager):
def daily_report(self):
return self.values('day').annotate(hours=models.Sum('value')).order_by('-day')
However now I need a monthly report that will get the Sum of value for each month.
I tried to extract the month first but then I lose the values.
Note: the month should not have the total sum for all years, so specifically I need to group by month,year
If you are using Postgresql you can do this. Ofc there is similar fuctions.
Overtime.objects.extra({'month': "to_char(day, 'Mon')", "year": "extract(year from day)"}).values('month', 'year').annotate(Sum('value'))
More info:
http://www.postgresql.org/docs/7.4/static/functions-formatting.html
http://www.postgresql.org/docs/9.1/static/functions-datetime.html
Or django way:
from django.db import connection
truncate_month = connection.ops.date_trunc_sql('month','day')
Overtime.objects.extra({'month': truncate_month}).values('month').annotate(Sum('value'))
I think this will help you.
Related
I have a model Sales with:
Saler
Product sold
Date
I would like to get the first sale for each date and for each saler, how can I do that?
Thanks
Considering the model Posted in the question, the Django ORM query will be:
first_sale = Sales.objects.order_by("Saler", "Date").distinct("Saler")
Not tested, but I would try (I assumed the field for the date of the sale is named sale_date, and is of type Datetime):
first_sale = Sales.objects.filter(saler=the_saler, sale_date__date=datetime.date(2021, 05, 19)).order_by('sale_date').first()
filter will restrict the search to a given saler (the_saler), and to a given day (see the __date expression: https://docs.djangoproject.com/fr/3.1/ref/models/querysets/#date)
order_by and first will give you the first of the day.
My proj has a model that goes like:
class Data(Model):
data = FloatField(verbose_name='Data', null=True, blank=True)
created_at = DateTimeField(verbose_name='Created at')
And my app creates a few hundred logs of this model per day.
I'm trying to sum only the greatest values of each day, without having to iterate over them (using only Django queries).
Is it possible without writing SQL queries?
PS: I'm able to get the greatest 'data' of each day, so the current logic iterates over days and sums the greatest values of each day. But that solution is becoming too slow and I'd like to solve it directly into db level.
Annotations and aggregates to the rescue:
from django.db.models import Sum, Max
from django.db.models.functions import Trunc
report = (Data.objects
.annotate(day=Trunc('created_at', 'day'))
.values('day')
.annotate(greatest=Max('data'))
.values('greatest')
.aggregate(total=Sum('greatest'))
)
print(report['total'])
The resulting SQL is almost simpler than the code:
SELECT SUM("greatest")
FROM
(SELECT MAX("app_data"."data_id") AS "greatest"
FROM "app_data"
GROUP BY DATE_TRUNC('day', "app_data"."created_at")) subquery
If you are using a database backed that supports distinct on fields (like postgres does) you can do.
Data.objects.order_by('created_at__date', '-data').distinct('created_at__date')
Hi I am writing a Django view which ouputs data for graphing on the client side (High Charts). The data is climate data with a given parameter recorded once per day.
My query is this:
format = '%Y-%m-%d'
sd = datetime.datetime.strptime(startdate, format)
ed = datetime.datetime.strptime(enddate, format)
data = Climate.objects.filter(recorded_on__range = (sd, ed)).order_by('recorded_on')
Now, as the range is increased the dataset obviously gets larger and this does not present well on the graph (aside from slowing things down considerably).
Is there an way to group my data as averages in time periods - specifically average for each month or average for each year?
I realize this could be done in SQL as mentioned here: django aggregation to lower resolution using grouping by a date range
But I would like to know if there is a handy way in Django itself.
Or is it perhaps better to modify the db directly and use a script to populate month and year fields from the timestamp?
Any help much appreciated.
Have you tried using django-qsstats-magic (https://github.com/kmike/django-qsstats-magic)?
It makes things very easy for charting, here is a timeseries example from their docs:
from django.contrib.auth.models import User
import datetime, qsstats
qs = User.objects.all()
qss = qsstats.QuerySetStats(qs, 'date_joined')
today = datetime.date.today()
seven_days_ago = today - datetime.timedelta(days=7)
time_series = qss.time_series(seven_days_ago, today)
print 'New users in the last 7 days: %s' % [t[1] for t in time_series]
I've got a simple Model like this:
class Order(models.Model):
created = model.DateTimeField(auto_now_add=True)
total = models.IntegerField() # monetary value
And I want to output a month-by-month breakdown of:
How many sales there were in a month (COUNT)
The combined value (SUM)
I'm not sure what the best way to attack this is. I've seen some fairly scary-looking extra-select queries but my simple mind is telling me I might be better off just iterating numbers, starting from an arbitrary start year/month and counting up until I reach the current month, throwing out simple queries filtering for that month. More database work - less developer stress!
What makes most sense to you? Is there a nice way I can pull back a quick table of data? Or is my dirty method probably the best idea?
I'm using Django 1.3. Not sure if they've added a nicer way to GROUP_BY recently.
Django 1.10 and above
Django documentation lists extra as deprecated soon. (Thanks for pointing that out #seddonym, #Lucas03). I opened a ticket and this is the solution that jarshwah provided.
from django.db.models.functions import TruncMonth
from django.db.models import Count
Sales.objects
.annotate(month=TruncMonth('created')) # Truncate to month and add to select list
.values('month') # Group By month
.annotate(c=Count('id')) # Select the count of the grouping
.values('month', 'c') # (might be redundant, haven't tested) select month and count
Older versions
from django.db import connection
from django.db.models import Sum, Count
truncate_date = connection.ops.date_trunc_sql('month', 'created')
qs = Order.objects.extra({'month':truncate_date})
report = qs.values('month').annotate(Sum('total'), Count('pk')).order_by('month')
Edits
Added count
Added information for django >= 1.10
Just a small addition to #tback answer:
It didn't work for me with Django 1.10.6 and postgres. I added order_by() at the end to fix it.
from django.db.models.functions import TruncMonth
Sales.objects
.annotate(month=TruncMonth('timestamp')) # Truncate to month and add to select list
.values('month') # Group By month
.annotate(c=Count('id')) # Select the count of the grouping
.order_by()
Another approach is to use ExtractMonth. I ran into trouble using TruncMonth due to only one datetime year value being returned. For example, only the months in 2009 were being returned. ExtractMonth fixed this problem perfectly and can be used like below:
from django.db.models.functions import ExtractMonth
Sales.objects
.annotate(month=ExtractMonth('timestamp'))
.values('month')
.annotate(count=Count('id'))
.values('month', 'count')
metrics = {
'sales_sum': Sum('total'),
}
queryset = Order.objects.values('created__month')
.annotate(**metrics)
.order_by('created__month')
The queryset is a list of Order, one line per month, combining the sum of sales: sales_sum
#Django 2.1.7
Here's my dirty method. It is dirty.
import datetime, decimal
from django.db.models import Count, Sum
from account.models import Order
d = []
# arbitrary starting dates
year = 2011
month = 12
cyear = datetime.date.today().year
cmonth = datetime.date.today().month
while year <= cyear:
while (year < cyear and month <= 12) or (year == cyear and month <= cmonth):
sales = Order.objects.filter(created__year=year, created__month=month).aggregate(Count('total'), Sum('total'))
d.append({
'year': year,
'month': month,
'sales': sales['total__count'] or 0,
'value': decimal.Decimal(sales['total__sum'] or 0),
})
month += 1
month = 1
year += 1
There may well be a better way of looping years/months but that's not really what I care about :)
Here is how you can group data by arbitrary periods of time:
from django.db.models import F, Sum
from django.db.models.functions import Extract, Cast
period_length = 60*15 # 15 minutes
# Annotate each order with a "period"
qs = Order.objects.annotate(
timestamp=Cast(Extract('date', 'epoch'), models.IntegerField()),
period=(F('timestamp') / period_length) * period_length,
)
# Group orders by period & calculate sum of totals for each period
qs.values('period').annotate(total=Sum(field))
By month:
Order.objects.filter().extra({'month':"Extract(month from created)"}).values_list('month').annotate(Count('id'))
By Year:
Order.objects.filter().extra({'year':"Extract(year from created)"}).values_list('year').annotate(Count('id'))
By day:
Order.objects.filter().extra({'day':"Extract(day from created)"}).values_list('day').annotate(Count('id'))
Don't forget to import Count
from django.db.models import Count
For django < 1.10
i have orders table in my database . i am going to count orders per month in the last 3 months
from itertools import groupby
from dateutil.relativedelta import relativedelta
date_range = datetime.now()-relativedelta(months=3)
aggs =Orders.objects.filter(created_at=date_range)\
.extra({'date_created':"date(created_at)"}).values('date_created')
for key , group in groupby(aggs):
print(key,len(list(group)))
created_at is datetime field. by extra function what done is taking date from datetime values. when using datetime we may not get the count correct because objects are created at different time in a day.
The for loop will print date and number of count
I have this model:
class Person(models.Model):
city = models.CharField(max_length=20, blank=True)
added_date = models.DateField(default=datetime.date.today)
I want to create a template/view that has a table of months and the number of people added that month (ie, 5 in january, 10 in february, 8 in march, etc.). I have a similar table for all the people from a each city using:
cities = Patient.objects.values('city').annotate(city_count=Count('city')).order_by('-city_count')
I don't know how to do that for my months table. I could filter for a particular month, then count all. But then I'd need to run that through a loop over every month, which would be multiple database hits. I don't want to do that.
Is there a way to do this without writing sql and just using django's api?
Its a very old thread, but i guess I'll answer in case someone else ended up here looking for a solution.
The solution is for Django 1.10+ using the ExtractMonth function, for more detail visit official documentation
First you have to import ExtractMonth, like
from django.db.models.functions import ExtractMonth
Then using your Persons model, the code will be like this
personsMonthlyData = Person.objects.annotate(month=ExtractMonth('added_date')).values('month').annotate(count=Count('id')).order_by('month')
personsMonthlyData will output something like this
[{month: 1, count: 3}, {month: 2: count: 1}]
where month represent the month number e.g. 1 for January and 2 for February and the count against each month is assigned to the count item.
I hope this helps.
The fact that most DBs have such a perfect way of doing this with a GROUP BY query that has no analog in Django AFAIK would lead me to drop into SQL to do this. I did a google search for "django sql" and turned up this post by Doug Hellman on this exact problem: http://blog.doughellmann.com/2007/12/using-raw-sql-in-django.html. I would use that as a starting point for getting your month counts into Django.