I have 2 models linked with each other with Foreign key. A Book and a Library models.
I have a list of books IDs.
And I want to get the list of Libraries where I can find all those books
I’m trying not loop through all the libraries as there the number of both libraries and books list isn’t small
Assuming the code looks like this, based on your description above that uses just a ForeignKey:
class Library(models.Model):
...
class Book(models.Model):
title = ...
author = ...
library = models.ForeignKey(Library, related_name='books', on_delete=models.CASCADE)
....
Each Book has a ForeignKey pointing to its Library, but using a related_name defined for that ForeignKey you can have a reverse relation between the models.
In this example you can access all the books of a library with that related name by writing a simple query like library.books.all().
To come to a conclusion, you can get all the libraries from a list of book ids using the related_name property in a query like this:
Library.objects.filter(books__id__in=book_ids).distinct()
Query above will filter all Library objects through books related name and return only those Library objects that have a Book whose id is found in the list of IDs called book_ids. Using .distinct() is just a nice practice to get a unique queryset.
Additional suggestion: In a setup like this where you have a ForeignKey only - one book cannot be found in more than one library. This is fine if each book is unique by some parameter. Though if you have a general book that can be found in multiple libraries I suggest you use a ManyToMany field.
EDIT:
We want to find the library that contains all the book ids from the list. How about trying something like this:
book_ids = [...]
Library.objects.annotate(
books_count=Count('books', filter=Q(books__id__in=book_ids))
).filter(books_count__gte=books.count())
We count number of books for each Library but only if they satisfy a filter books__id__in=book_ids meaning that the book ID is in the list of ids. If the total count of filtered library books is the same as the list of ids than we've found our library.
Related
This is similar to other questions regarding complicated prefetches but seems to be slightly different. Here are some example models:
class Author(Model):
pass
class Book(Model):
authors = ManyToManyField(Author, through='BookAuthor')
class BookAuthor(Model):
book = ForeignKey(Book, ...)
author = ForeignKey(Author, ...)
class Event(Model):
book = ForeignKey(Book, ...)
author = ForeignKey(Author, ...)
In summary: a BookAuthor links a book to one of its authors, and an Event also concerns a book and one of its authors (the latter constraint is unimportant). One could design the relationships differently but it's too late for that now, and in my case, this in fact is part of migrating away from the current setup.
Now suppose I have a BookAuthor instance and want to find any events that relate to that combination of book and author. I can follow either the book or author relations on BookAuthor and their event_set reverse relationship, but neither gives me what I want and, if I have several BookAuthors it becomes a pain.
It seems that the way to get an entire queryset "annotated" onto a model instance is via prefetch_related but as far as I can tell there is no way, in the Prefetch object's queryset property, to refer to the root object's fields. I would like to do something like this:
BookAuthor.objects.filter(...).prefetch_related(
Prefetch(
'book__event_set',
queryset=Event.objects.filter(
author=OuterRef('author')
)
}
)
However, OuterRef only works in subqueries and this is not one. The answer to this question suggests I could use a subquery here but I don't understand how it could ever work: you have to put the subquery inside a query to work in the Prefetch, and the OuterRef refers to that object/DB row; there is no way to get back to the original, root object. If I translate the code there into my situation it is clear that the OuterRef is referring to the outer Event query, not the outer BookAuthor.
To make the question precise: what I want is O(1) queries which, for a queryset of BookAuthors annotate each instance - or one of its foreign keys - with a collection of the corresponding Events. Obviously one can get all the Events and glue everything together in python. I want to avoid this but if anyone has a particularly elegant way of doing that, it would also be useful to know.
I'd like to filter an annotation using the Django ORM. A lot of the articles I've found here at SO are fairly dated, targeting Django back in the 1.2 to 1.4 days:
Filtering only on Annotations in Django - This question from 2010 suggests using an extra clause, which isn't recommended by the official Django docs
Django annotation with nested filter - Similar suggestions are provided in this question from 2011.
Django 1.8 adds conditional aggregation, which seems like what I might want, but I can't quite figure out the syntax that I'll eventually need. Here are my models and the scenario I'm trying to reach (I've simplified the models for brevity's sake):
class Project(models.Model):
name = models.CharField()
... snip ...
class Milestone_meta(models.Model):
name = models.CharField()
is_cycle = models.BooleanField()
class Milestone(models.Model):
project = models.ForeignKey('Project')
meta = models.ForeignKey('Milestone_meta')
entry_date = models.DateField()
I want to get each Project (with all its fields), along with the Max(entry_date) and Min(entry_date) for each associated Milestone, but only for those Milestone records whose associated Milestone_meta has the is_cycle flag set to True. In other words:
For every Project record, give me the maximum and minimum Milestone entry_dates, but only when the associated Milestone_meta has a given flag set to True.
At the moment, I'm getting a list of projects, then getting the Max and Min Milestones in a loop, resulting in N+1 database hits (which gets slow, as you'd expect):
pqs = Projects.objects.all()
for p in pqs:
(theMin, theMax) = getMilestoneBounds(p)
# Use values from p and theMin and theMax
...
def getMilestoneBounds(pid):
mqs = Milestone.objects.filter(meta__is_cycle=True)
theData = mqs.aggregate(min_entry=Min('entry_date'),max_entry=Max('entry_date'))
return (theData['min_entry'], theData['max_entry'])
How can I reduce this to one or two queries?
As far as I know, you can not get all required project objects in one query.
However, if you don't need the objects and can work with just their id, one way would be-
Milestone.objects.filter(meta__is_cycle=True).values('project').annotate(min_entry=Min('entry_date')).annotate(max_entry=Max('entry_date'))
It will give a list of dicts having data of distinct projects, you can then use their 'id' to lookup the objects when needed.
I'm new to django and trying to understand how I might create my models. I'm building an app that can be used to create documents from a template. These documents can be of a number of different types, and contain generic sections as well as sections specific to each document type. I'm intending for the specific sections that the user can choose from (to include in their document), to change according to the document type they have chosen to create. Yet the generic sections will be used no matter what the document type.
I'm having a hard time getting my head round how I might build my models to achieve this. I've considered giving the document model and sections model a document type that can be set and referenced in html, matching the sections to each document:
class Document(models.Model):
document_type = models.CharField(max_length=50)
class Sections(models.Model):
document_type = models.CharField(max_length=50)
or adding in a document type model:
class Document(models.Model):
document_type = models.ForeignKey(DocumentType)
class Sections(models.Model):
document_type = models.ForeignKey(DocumentType)
class DocumentType(models.Model):
name = models.CharField(max_length=50)
But I'm worried that this will cause problems due to many documents sharing some generic sections. And so I wondered if I separate the generic and specific sections:
class GenericSection():
document_type = models.ManyToManyField(DocumentType)
class SpecificSection():
document_type = models.ForeignKey(DocumentType)
or even separate each document type into it's own app. I think I've got myself into a twist with this and would appreciate any feedback on whether there's a more appropriate way to approach this.
If a document can be of only one type, I would personally have that in it's own table. That way they are not duplicated everywhere.
Then your document table's document type should be a foreign key to the document type table (assuming a document can only have one type or a many to many relation to the document type table if a document can be more than one type)
foreign key's are a great way to make sure your table doesn't turn into a nightmare of pointing to the wrong values or dead values etc...
I have these models :
class Package(models.Model):
title = CharField(...)
class Item(models.Model)
package = ForeignKey(Package)
price = FloatField(...)
class UserItem(models.Model)
user = ForeignKey(User)
item = ForeignKey(Item)
purchased = BooleanField()
I am trying to achieve 2 functionality with the best performance possible :
In my templete I would like to calculate each package price sum of all its items. (Aggregate I assume ?)
More complicated : I wish that for each user I can sum up the price of all item purchased. so the purchased = True.
Assume I have 10 items in one package which each of them cost 10$ the package sum should be 100$. assume the user purchase 5 items the second sum should be 50$.
I can easily do simple queries with templetetags but I believe it can be done better ? (Hopefully)
To total the price for a specific package a_package you can use this code
Item.objects.filter(package=a_package).aggregate(Sum('price'))
There is a a guide on how to do these kind of queries, and the aggregate documentation with all the different functions described.
This kind of query can also solve your second problem.
UserItem.objects.filter(user=a_user).filter(purchased=True).aggregate(sum('price'))
You can also use annotate() to attach the count to each object, see the first link above.
The most elegant way in my opinion would be to define a method total on the Model class and decorate it as a property. This will return the total (using Django ORM's Sum aggregate) for either Package or User.
Example for class Package:
from django.db.models import Sum
...
class Package(models.Model):
...
#property
def total(self):
return self.item_set.aggregate(Sum('price'))
In your template code you would use total as any other model attribute. E.g.:
{{ package_instance.total }}
#Vic Smith got the solution.
But I would add a price attribute on the package model if you wish
the best performance possible
You would add a on_save signal to Item, and if created, you update the related package object.
This way you can get the package price very quickly, and even make quick sorting, comparing, etc.
Plus, I don't really get the purpose of the purchased attribute. But you probably want to make a ManyToMany relationship between Item and User, and define UserItem as the connection with the trhough parameter.
Anyway, my experience is that you usually want to make a relationship between Item and a Purchasse objet, which is linked to User, and not a direct link (unless you start to get performances issues...). Having Purchasse as a record of the event "the user bough this and that" make things easier to handle.
The Django documentation gives examples for using annotate() to produce aggregate results based on related fields of a QuerySet (i.e., using joins in annotate()).
A simplified example from the docs is for instance Store.objects.annotate(min_price=Min('books__price')), where books is a ManyToMany field of Store to Book and price is a field of Book.
To continue this example, how would I generate an annotated QuerySet of Store objects with the lowest prices not for all books in the store, but just for books with "author='William Shakespeare'"? In other words, how do I filter the related fields used to calculate the aggregate?
The documentation explains how to do this:
Store.objects.filter(books__author='William Shakespeare').annotate(
min_price=Min('books__price'))
As that link notes, the order of filter and annotate matters here - because you want to only count books that match the filter in the annotation, the filter must come first.