I am trying to use an aggregated column in a case statement in Django and I am having no luck getting Django to accept it.
The code is to return a list of people who have played a game, the number of times they have played the game and their total score. The list is sorted by total score descending. However, the game has a minimum number of plays in order to qualify. Players without sufficient plays are listed at the bottom. For example:
Player Total Plays
Jill 109 10
Sam 92 11
Jack 45 9
Sue 50 3
Sue is fourth in the list because her number of plays (3) is less than the minimum (5).
The relevant models and function are:
class Player(models.Model):
name = models.CharField()
class Game(models.Model):
name = models.CharField()
min_plays = models.IntegerField(default=1)
class Play(models.Model):
game = models.ForeignKey(Game)
class Score(models.Model):
play = models.ForeignKey(Play)
player = models.ForeignKey(Player)
score = models.IntegerField()
def game_standings(game):
query = Player.objects.filter(score__play__game_id=game.id)
query = query.annotate(plays=Count('score', filter=Q(score__play__game_id=self.id)))
query = query.annotate(total_score=Sum('score', filter=Q(score__play__game_id=self.id)))
query = query.annotate(sufficient=Case(When(plays__ge=game.minimum_plays, then=1), default=0)
query = query.order_by('-sufficient', '-total_score', 'plays')
When the last annotate method is hit, a "Unsupported lookup 'ge' for IntegerField or join on the field not permitted" error is reported. I tried to change the case statement to embed the count instead of using the annotated field:
query = query.annotate(
sufficient=Case(When(
Q(Count('score', filter=Q(score__play__game_id=game.id)))> 3, then=1), default=0
)
)
but Django reports a TypeError with '>' and Q and int.
The SQL I am trying to get to is:
SELECT "player"."id",
"player"."name",
COUNT("score"."id") FILTER (WHERE "play"."game_id" = 8) AS "plays",
SUM("score"."score") FILTER (WHERE "play"."game_id" = 8) AS "total_score",
case when COUNT("score"."id") FILTER (WHERE "play"."game_id" = 8) >= 5 then 1
else 0
end as sufficient
FROM "player"
LEFT OUTER JOIN "score" ON ("player"."id" = "score"."player_id")
LEFT OUTER JOIN "play" ON ("score"."play_id" = "play"."id")
WHERE "play"."game_id" = 8
GROUP BY "player"."id"
ORDER BY sufficent desc, total_score desc
I can't seem to figure out how to have the case statement use to play count.
Thanks
Related
In my model, I have many Things that can have many Labels, and this relationship is made by user-submitted Descriptions via form. I cannot figure out how to count how much of each Label each Thing has.
In models.py, I have:
class Label(models.Model):
name = models.CharField(max_length=100)
class Thing(models.Model):
name = models.CharField(max_length=100)
class Description(models.Model):
thingname = models.ForeignKey(Thing, on_delete=models.CASCADE)
labels = models.ManyToManyField(Label,blank=True)
If we say our current Thing is a cat, and ten people have submitted a Description for the cat, how can we make our template output an aggregate count of each related Label for the Thing?
For example:
Cat
10 fluffy
6 fuzzy
4 cute
2 dangerous
1 loud
I've tried a few things with filters and annotations like
counts = Label.objects.filter(description_form = pk).annotate(num_notes=Count('name'))
but I think there's something obvious I'm missing either in my views.py or in my template.
You can use this to retrive this information:
Description.objects.prefetch_related("labels").values("labels__name", "thing_name__name").annotate(num_notes=models.Count("labels__name"))
this will be equal to:
SELECT "core_label"."name",
"core_thing"."name",
Count("core_label"."name") AS "num_notes"
FROM "core_description"
LEFT OUTER JOIN "core_description_labels"
ON ( "core_description"."id" =
"core_description_labels"."description_id" )
LEFT OUTER JOIN "core_label"
ON ( "core_description_labels"."label_id" =
"core_label"."id" )
INNER JOIN "core_thing"
ON ( "core_description"."thing_name_id" = "core_thing"."id" )
GROUP BY "core_label"."name",
"core_thing"."name"
I am working on a django web app that manages payroll based on reports completed, and then payroll generated. 3 models as follows. (ive tried to limit to data needed for question).
class PayRecord(models.Model):
rate = models.FloatField()
user = models.ForeignKey(User)
class Payroll(models.Model):
company = models.ForeignKey(Company)
name = models.CharField()
class PayrollItem(models.Model):
payroll = models.ForeignKey(Payroll)
record = models.OneToOneField(PayRecord, unique=True)
What is the most efficient way to get all the PayRecords that aren't also in PayrollItem. So i can select them to create a payroll item.
There are 100k records, and my initial attempt takes minutes. Attempt tried below (this is far from feasible).
records_completed_in_payrolls = [
p.report.id for p in PayrollItem.objects.select_related(
'record',
'payroll'
)
]
Because you have the related field record in PayrollItem you can reach into that model while you filter PayRecord. Using the __isnull should give you what you want.
PayRecord.objects.filter(payrollitem__isnull=True)
Translates to a sql statement like:
SELECT payroll_payrecord.id,
payroll_payrecord.rate,
payroll_payrecord.user_id
FROM payroll_payrecord
LEFT OUTER JOIN payroll_payrollitem
ON payroll_payrecord.id = payroll_payrollitem.record_id
WHERE payroll_payrollitem.id IS NULL
Depending on your intentions, you may want to chain on a .select_related (https://docs.djangoproject.com/en/3.1/ref/models/querysets/#select-related)
PayRecord.objects.filter(payrollitem__isnull=True).select_related('user')
which translates to something like:
SELECT payroll_payrecord.id,
payroll_payrecord.rate,
payroll_payrecord.user_id,
payroll_user.id,
payroll_user.name
FROM payroll_payrecord
LEFT OUTER JOIN payroll_payrollitem
ON (payroll_payrecord.id = payroll_payrollitem.record_id)
INNER JOIN payroll_user
ON (payroll_payrecord.user_id = payroll_user.id)
WHERE payroll_payrollitem.id IS NULL
I'd like to annotate a queryset based on both all related objects and filtered subset. Let's say we have some books and they are sold at some stores at some prices. Now, for one book, I'd like to get all the stores which sell that book, the price of the book in those stores and the average price of books in each of those stores.
My models.py:
from django.db import models
class Book(models.Model):
title = models.CharField(max_length=100)
class Store(models.Model):
name = models.CharField(max_length=100)
books = models.ManyToManyField(Book, through='BookInStore')
class BookInStore(models.Model):
book = models.ForeignKey(Book)
store = models.ForeignKey(Store)
price = models.IntegerField()
class Meta:
unique_together = ('book', 'store')
Create some objects:
book1 = Book.objects.create(title='book1')
book2 = Book.objects.create(title='book2')
book3 = Book.objects.create(title='book3')
store = Store.objects.create(name='store')
BookInStore.objects.create(book=book1, store=store, price=10)
BookInStore.objects.create(book=book2, store=store, price=100)
BookInStore.objects.create(book=book3, store=store, price=1000)
Now, for book1, I'm trying filter the stores that sell book1, get the prices in those stores and also the average price of all books in each of the stores:
book_availability = (
Store.objects
.annotate(avg_price=Avg('bookinstore__price'))
.filter(bookinstore__book=book1)
.annotate(
duplicates=Count('bookinstore__price'),
book_price=Sum('bookinstore__price')
)
)
However, it doesn't work correctly:
for b in book_availability:
print("Avg price:", b.avg_price)
print("Number of copies (should be 1):", b.duplicates)
print("Price of book1 (should be 10):", b.book_price)
I get the following output:
Avg price: 370.0
Number of copies (should be 1): 3
Price of book1 (should be 10): 30
The average price is correct. But for some reason, the price of the book has been multiplied by the number of total books in the store. What am I doing wrong? How should I get the kind of queryset I'm after?
One workaround: Because book&store combinations are unique, it shouldn't matter whether one uses Sum, Avg, Min, Max or something similar for getting the price of the particular book, because there should be only one item. However, using Sum sums the weird duplicates, so it is better to use Min or Max to get the correct integer because they give the correct answer in the presence of duplicates:
book_availability = (
Store.objects
.annotate(avg_price=Avg('bookinstore__price'))
.filter(bookinstore__book=book1)
.annotate(
duplicates=Count('bookinstore__price'),
book_price=Min('bookinstore__price')
)
)
And printing now:
for b in book_availability:
print("Avg price:", b.avg_price)
print("Number of copies (should be 1):", b.duplicates)
print("Price of book1 (should be 10):", b.book_price)
We see the correct book price now:
Avg price: 370.0
Number of copies (should be 1): 3
Price of book1 (should be 10): 10
This workaround happens to give the correct answer but it doesn't remove the duplicates, so I'm still wondering if there is something wrong in the approach.
One possibility is to use Case and When to do the filtering for the annotation:
book_availability = (
Store.objects
.annotate(
avg_price=Avg('bookinstore__price'),
book_price=Sum(Case(When(
bookinstore__book=book1,
then='bookinstore__price'
)))
)
.filter(bookinstore__book=book1)
)
for b in book_availability:
print("Avg price:", b.avg_price)
print("Price of book1 (should be 10):", b.book_price)
Which gives:
Avg price: 370.0
Price of book1 (should be 10): 10
I've been having issues filtering annotations. The models are:
class VenueBookmark(models.Model):
venue = models.ForeignKey(Venue)
cost_per_guest = models.DecimalField(max_digits=19, decimal_places=2, blank=True, null=True)
class Venue(DateAwareModel):
name = models.CharField(max_length=255)
And in my view I basically have:
venues = venues.annotate(min_cost=Min('venuebookmark__cost_per_guest'))
venues = venues.filter(min_cost__lte=user_input)
However, I still get results which are > user_input. Any insights on this?
Edit:
I've tried converting user_input to Decimal type, but still get the same result:
venues = venues.filter(min_cost__lte=Decimal(user_input))
This is also most of the SQL I got from the Django Debug Toolbar:
SELECT "venue_search_venue"."name", MIN("bookmarks_venuebookmark"."cost_per_guest") AS "min_cost"
FROM "venue_search_venue"
LEFT OUTER JOIN "bookmarks_venuebookmark" ON ( "venue_search_venue"."id" = "bookmarks_venuebookmark"."venue_id" )
GROUP BY "venue_search_venue"."id", "venue_search_venue"."name"
HAVING MIN("bookmarks_venuebookmark"."cost_per_guest") <= %s
DESC LIMIT 9' - PARAMS = ("Decimal('100')",)
After querying the database directly and comparing the results, I suspect the issue is Django renders the user_input as a string instead of a number, e.g.
HAVING MIN("bookmarks_venuebookmark"."cost_per_guest") <= '100'
instead of:
HAVING MIN("bookmarks_venuebookmark"."cost_per_guest") <= 100
This is a funny question. The answer is simple, you should change min by max:
venues = venues.annotate(max_cost=Max('venuebookmark__cost_per_guest'))
venues = venues.filter(max_cost__lte=max_cost)
Illustrating it with an example:
venues VenueBookmark Cost
1 5
1 7
With your actual condition, if we take 6 as max_cost, venues number 1 will appears in result-set because min of VenueBookmark Cost is 5 and 5 < 6.
Changing min by max, venues number 1 will not be included in results, because 7 is not < than 6
Edit
Also, include order_by() to clear default ordering:
venues = (venues
.order_by()
.annotate(min_cost=Min('venuebookmark__cost_per_guest'))
)
Default ordering injected sort fields on group by clause.
I have a django model Level for a game.
class Level(models.Model):
key = models.CharField(max_length=100)
description = models.CharField(max_length=500)
requiredPoints = models.IntegerField()
badgeurl = models.CharField(max_length=100)
challenge = models.ForeignKey(Challenge)
I now want to query for the highest level with a pointsRequired value smaller than a given value.
If i have:
Level 1: requiredPoints: 200
Level 2: requiredPoints: 800
Level 3: requiredPoints: 2000
When I enter e.g. 900 or 1999 as a query parameter, I want level 2 to be returned, when entering 10000 it should be level 3.
in sql it would look like
select pointsRequired,
abs(pointsRequired - parameter) as closest
from the_table
order by closest
limit 1
Any tips? Do I have to use an extra Query-Set? How would it look like
I don't think your SQL is correct. It should return 'level 3' if the parameter is 1999. Based on your description the SQL could be:
SELECT pointsRequired FROM the_table WHERE pointsRequired < parameter ORDER BY pointsRequired DESC LIMIT 1;
Or in Django:
try:
Level.objects.filter(requiredPoints__lt=parameter).order_by('-requiredPoints')[0]
except IndexError:
# Do something