Question
Is it possible using Django's aggregation capabilities to calculate a sumproduct?
Background
I am modeling an invoice, which can contain multiple items. The many-to-many relationship between the Invoice and Item models is handled through the InvoiceItem intermediary table.
The total amount of the invoice—amount_invoiced—is calculated by summing the product of unit_price and quantity for each item on a given invoice. Below is the code that I'm currently using to accomplish this, but I was wondering if there is a better way to handle this using Django's aggregation capabilities.
Current Code
class Item(models.Model):
item_num = models.SlugField(unique=True)
description = models.CharField(blank=True, max_length=100)
class InvoiceItem(models.Model):
item = models.ForeignKey(Item)
invoice = models.ForeignKey('Invoice')
unit_price = models.DecimalField(max_digits=10, decimal_places=2)
quantity = models.DecimalField(max_digits=10, decimal_places=4)
class Invoice(models.Model):
invoice_num = models.SlugField(max_length=25)
invoice_items = models.ManyToManyField(Item,through='InvoiceItem')
def _get_amount_invoiced(self):
invoice_items = self.invoiceitem_set.all()
amount_invoiced = 0
for invoice_item in invoice_items:
amount_invoiced += (invoice_item.unit_price *
invoice_item.quantity)
return amount_invoiced
amount_invoiced = property(_get_amount_invoiced)
Yes, it is possible since Django 1.1 where aggregate functions were introduced. Here's a solution for your models:
def _get_amount_invoiced(self):
self.invoiceitem_set.extra(select=("item_total": "quantity * unit_price")
).aggregate(total=Sum("item_total")["total"]
It is, however, highly recommended to store item_total in a database, because it may be subject to discounts, taxes and other changes that make calculating it evety time impractical or even impossible.
Related
i am working on a shop for clothes, shoes etc. And i am trying to sort the products by price. The problems comes when the product's discount_price field is populated which means that this product is discounted. So when i want to order the products in my view i am expecting to see products with lower discount_price before products with higher price, but it is not working like this.
models.py
class Item(models.Model):
price = models.IntegerField(null=True, blank=True)
discount_price = models.IntegerField(null=True, blank=True)
The query i am performing
items = Item.objects.all().order_by('price', 'discount_price')
You can make use of Coalesce(..) [Django-doc] to first sort by 'discount_price' if it is not NULL, and use price otherwise:
from django.db.models.functions import Coalesce
items = Item.objects.order_by(Coalesce('discount_price', 'price').asc())
I have two models: the gas station and the price of a product. The price can up to have 4 choices, one for each product type, not every station has all four products. I want to query the latest entry of each of those products, preferably in a single query:
class GasStation(models.Model):
place_id = models.IntegerField(primary_key=True)
name = models.CharField(max_length=255, null=True, blank=True)
class Price(models.Model):
class Producto(models.TextChoices):
GASOLINA_REGULAR = 'GR', _('Gasolina regular')
GASOINA_PREMIUM = 'GP', _('Gasolina premium')
DIESEL_REGULAR = 'DR', _('Diesel regular')
DIESEL_PREMIUM = 'DP', _('Diesel premium')
product = models.CharField(max_length=2, choices=Producto.choices)
created = models.DateTimeField(auto_now_add=True)
updated = models.DateTimeField(auto_now=True)
price = models.FloatField(null=True, blank=True)
estacion = models.ForeignKey(GasStation,
on_delete=models.SET_NULL,
null=True,
related_name='prices')
I've tried with:
station.price.filter(product__in=['GR', 'GP', 'DR', 'DP']).latest()
But it only returns the latest of the whole queryset, not the latest price of each product type. I want to avoid querying for each individual product because some stations don't sell all types .Any advice?
You're looking for annotations and Subquery. Below is what I think might work. Your models aren't fully defined. If you need the whole Price instance, then this won't work for you. Subquery can only annotate a single field.
from django.db.models import OuterRef, Subquery
stations = GasStation.objects.annotate(
latest_regular=Subquery(
Price.objects.filter(station_id=OuterRef("pk"), product="GR").order_by('-updated').values("price")[:1]
),
latest_premium=Subquery(
Price.objects.filter(station_id=OuterRef("pk"), product="GP").order_by('-updated').values("price")[:1]
),
...
)
station = stations.get(something_here)
station.latest_premium, station.latest_regular
You can make this more concise by using a dict comprehension iterating over your Product short codes and then doing .annotate(**annotations)
I've been working on a Purchase Order app but I'm getting a little confused how I'm going to put it all together.
I have 3 models -
class PurchaseOrder(models.Model):
po_number = models.IntegerField(default=get_po_number, unique=True)
po_date = models.DateField()
invoice_number = models.ForeignKey(Invoice, on_delete=models.CASCADE)
....
class PurchaseOrderItem(models.Model):
po_number_fk = models.ForeignKey(PurchaseOrder, on_delete=models.CASCADE)
qty = models.IntegerField()
unit = models.CharField(max_length=100, blank=True, null=True)
description = models.CharField(max_length=255)
unit_price = models.DecimalField(max_digits=6, decimal_places=2)
amount = models.DecimalField(max_digits=6, decimal_places=2)
class PurchaseOrderTotal(models.Model):
po_number_fk = models.ForeignKey(PurchaseOrder, on_delete=models.CASCADE)
subtotal = models.DecimalField(max_digits=6, decimal_places=2)
tax = models.DecimalField(max_digits=6, decimal_places=2, default="7.82")
shipping = models.DecimalField(max_digits=6, decimal_places=2)
other = models.DecimalField(max_digits=6, decimal_places=2)
total = models.DecimalField(max_digits=6, decimal_places=2)
the first (PurchaseOrder) holds information about the purchase order itself. ie. what the invoice number is, the vendor, etc.
the second (PurchaseOrderItem) lists items in the purchase order to purchase
the third (PurchaseOrderTotal) totals up the amounts from the items and adds tax etc. (I may not need this model.. I can probably put this info in the first model?)
Does it look like I'm going about this in the right way or should I take away the third model and put those fields from the third model into the first model? How do I total up all prices for all items? I'm sure I'll need to do some sort of loop to total up all prices but where do I do that? In the form_valid fucntion? or do I override the save function and do it there? Thanks!
I would combine model 1 and 3. You can total up the numbers when you submit the form and then save it to the same model. Its better to try and keep all related model information on the same table. If you find out after testing that for some reason one model doesn't work well you can always change it.
There are multiple ways to total up the totals:
Do it in your view when you submit the form and save it to the DB.
This would be my recommended choice for your situation since this data while not likely
change very often.
Do not save the totals in the DB and
sum them up in your view before you display that information.
Do not save the totals in the DB and sum them up in your
template.
I would highly discourage the third option since your sounds like your data is fairly static and there is no need to do that processing in your template. I added it to illustrate all the possibilities. This also applies to the second option, though in this situation option two is better than three because it is done in the view instead the template but is still not the best option since it would not make since to do that calculation over and over again for data that never changed.
Example:
def your_view(request):
if request.method == 'POST':
# create a form instance and populate it with data from the request:
form = InvoiceForm(request.POST)
# check whether it's valid:
if form.is_valid():
tax = form.cleaned_data.get('tax')
sub = form.cleaned_data.get('sub_total')
fee = form.cleaned_data.get('fee')
form.total = tax + sub + fee
form.save()
return HttpResponseRedirect('/thanks/')
# if a GET (or any other method) we'll create a blank form
else:
form = InvoiceForm()
return render(request, 'name.html', {'form': form})
I've the below inefficient 'destroy' method for deleting Ratings that are held in Stimulus which itself is held within Experiment (I have simplified my models, for reasons of clarity).
Could you advise on a more efficient way of achieving this?
class Rating(models.Model):
rater = TextField(null=True)
rating = FloatField(null=True)
created = models.DateTimeField(null=True)
class Stimulus(TimeStampedModel):
genes = TextField()
weights = ListField()
ratings = ManyToManyField(Rating, null=True)
evaluation = FloatField(null=True)
complete = BooleanField(default=False)
Class Experiment(models.Model):
all_individuals = ManyToManyField(Stimulus, null=True)
def destroy(self):
all_ratings = Rating.objects.all()
for ind in self.all_individuals.all():
ratings = ind.ratings.all()
for rating in ratings:
if rating in all_ratings:
Rating.objects.filter(id = rating.id).delete()
Background: I am using Django to run an experiment (Experiment) which shows Users many Stimuli (Stimulus). Each Stimulus gets rated many times. Thus, I need to save multiple ratings per stimulus (and multiple stimuli per experiment).
Some simple improvements
Remove the if rating in all_ratings, every rating will be in the list of all ratings
Do the delete on the database side
ind.ratings.all().delete()
Use prefetch_related to get the foreign key objects
self.all_individuals.prefetch_related('ratings'):
Combined would be:
def destroy(self):
for ind in self.all_individuals.prefetch_related('ratings'):
ratings = ind.ratings.all().delete()
I think that in this case using ManyToManyField isn't the best choice.
You'll have less problems using common ForeignKey's changing a little the structure of this models.
Eg.
class Rating(models.Model):
rater = TextField(null=True)
rating = FloatField(null=True)
created = models.DateTimeField(null=True)
stimulus = models.ForeignKey('Stimulus', related_name='ratings')
class Stimulus(TimeStampedModel):
genes = TextField()
weights = ListField()
#ratings = ManyToManyField(Rating, null=True)
evaluation = FloatField(null=True)
complete = BooleanField(default=False)
experiment = models.ForeignKey('Experiment', related_name='stimulus')
class Experiment(models.Model):
#all_individuals = ManyToManyField(Stimulus, null=True)
name = models.CharField(max_length=100)
This is a more clear structure and when you delete Experiment by, experiment_instance.delete() a delete cascade will delete all other related models.
Hope it helps.
I'd like to create a filter-sort mixin for following values and models:
class Course(models.Model):
title = models.CharField(max_length=70)
description = models.TextField()
max_students = models.IntegerField()
min_students = models.IntegerField()
is_live = models.BooleanField(default=False)
is_deleted = models.BooleanField(default=False)
teacher = models.ForeignKey(User)
class Session(models.Model):
course = models.ForeignKey(Course)
title = models.CharField(max_length=50)
description = models.TextField(max_length=1000, default='')
date_from = models.DateField()
date_to = models.DateField()
time_from = models.TimeField()
time_to = models.TimeField()
class CourseSignup(models.Model):
course = models.ForeignKey(Course)
student = models.ForeignKey(User)
enrollment_date = models.DateTimeField(auto_now=True)
class TeacherRating(models.Model):
course = models.ForeignKey(Course)
teacher = models.ForeignKey(User)
rated_by = models.ForeignKey(User)
rating = models.IntegerField(default=0)
comment = models.CharField(max_length=300, default='')
A Course could be 'Discrete mathematics 1'
Session are individual classes related to a Course (e.g. 1. Introduction, 2. Chapter I, 3 Final Exam etc.) combined with a date/time
CourseSignup is the "enrollment" of a student
TeacherRating keeps track of a student's rating for a teacher (after course completion)
I'd like to implement following functions
Sort (asc, desc) by Date (earliest Session.date_from), Course.Name
Filter by: Date (earliest Session.date_from and last Session.date_to), Average TeacherRating (e.g. minimum value = 3), CourseSignups (e.g. minimum 5 users signed up)
(these options are passed via a GET parameters, e.g. sort=date_ascending&f_min_date=10.10.12&...)
How would you create a function for that?
I've tried using
denormalization (just added a field to Course for the required filter/sort criterias and updated it whenever changes happened), but I'm not very satisfied with it (e.g. needs lots of update after each TeacherRating).
ForeignKey Queries (Course.objects.filter(session__date_from=xxx)), but I might run into performance issues later on..
Thanks for any tipp!
In addition to using the Q object for advanced AND/OR queries, get familiar with reverse lookups.
When Django creates reverse lookups for foreign key relationships. In your case you can get all Sessions belonging to a Course, one of two ways, each of which can be filtered.
c = Course.objects.get(id=1)
sessions = Session.objects.filter(course__id=c.id) # First way, forward lookup.
sessions = c.session_set.all() # Second way using the reverse lookup session_set added to Course object.
You'll also want to familiarize with annotate() and aggregate(), these allow you you to calculate fields and order/filter on the results. For example, Count, Sum, Avg, Min, Max, etc.
courses_with_at_least_five_students = Course.objects.annotate(
num_students=Count('coursesignup_set__all')
).order_by(
'-num_students'
).filter(
num_students__gte=5
)
course_earliest_session_within_last_240_days_with_avg_teacher_rating_below_4 = Course.objects.annotate(
min_session_date_from = Min('session_set__all')
).annotate(
avg_teacher_rating = Avg('teacherrating_set__all')
).order_by(
'min_session_date_from',
'-avg_teacher_rating'
).filter(
min_session_date_from__gte=datetime.now() - datetime.timedelta(days=240)
avg_teacher_rating__lte=4
)
The Q is used to allow you to make logical AND and logical OR in the queries.
I recommend you take a look at complex lookups: https://docs.djangoproject.com/en/1.5/topics/db/queries/#complex-lookups-with-q-objects
The following query might not work in your case (what does the teacher model look like?), but I hope it serves as an indication of how to use the complex lookup.
from django.db.models import Q
Course.objects.filter(Q(session__date__range=(start,end)) &
Q(teacher__rating__gt=3))
Unless absolutely necessary I'd indeed steer away from denormalization.
Your sort question wasn't entirely clear to me. Would you like to display Courses, filtered by date_from, and sort it by Date, Name?