If my Models look like:
class Publisher(models.Model):
pass
class Book(models.Model):
publisher = models.ForeignKey(Publisher)
class Page(models.Model):
book = models.ForeignKey(Book)
and I would like to get the queryset for Publisher I do Publisher.object.all().
If then want to make sure to prefetch I can do:
Publisher.objects.all().prefetch_related('book_set')`
My questions are:
Is there a way to do this prefetching using select_related or
must I use prefetch_related?
Is there a way to prefetch the
page_set? This does not work:
Publisher.objects.all().prefetch_related('book_set', 'book_set_page_set')
Since Django 1.7, instances of django.db.models.Prefetch class can be used as an argument of .prefetch_related. Prefetch object constructor has a queryset argument that allows to specify nested multiple levels prefetches like that:
Project.objects.filter(
is_main_section=True
).select_related(
'project_group'
).prefetch_related(
Prefetch(
'project_group__project_set',
queryset=Project.objects.prefetch_related(
Prefetch(
'projectmember_set',
to_attr='projectmember_list'
)
),
to_attr='project_list'
)
)
It is stored into attributes with _list suffix because I use ListQuerySet to process prefetch results (filter / order).
No, you cannot use select_related for a reverse relation. select_related does a SQL join, so a single record in the main queryset needs to reference exactly one in the related table (ForeignKey or OneToOne fields). prefetch_related actually does a totally separate second query, caches the results, then "joins" it into the queryset in python. So it is needed for ManyToMany or reverse ForeignKey fields.
Have you tried two underscores to do the multi level prefetches? Like this: Publisher.objects.all().prefetch_related('book_set', 'book_set__page_set')
Related
I have a serializer like that :
class UserSerializerDetail(UserSerializer):
active_projects = serializers.SerializerMethodField()
def get_active_projects(self, obj):
return obj.projects.filter(project__active=True).count()
the problem I am having here is that the SerializerMethodField is calling an extra filter and I wish to use select related to create a join and overcome another hit in the database.
The second issue is how can I select_related a count?
I assume the you meant user have many projects. 1 to many relationship.
The best is the combination of prefetch_related with aggregation (select_related mostly used for 1 to 1 relationship).
example:
from django.db.models import Count, Q
projects = User.objects.prefetch_related('projects').annotate(active_projects=Count('projects', filter=Q(projects__active=True)))
Then your users now have active_projects field.
I have 2 models, company and Product.
class Product(Meta):
company = models.ForeignKey(Company, related_name='products', on_delete=models.CASCADE)
form the database I'm trying to get the Company data and the corresponding products.
From the products I want to get only the name and to be ordered descending by updated_at, created_at.
I'm working with Prefetch object and prefetch_related and definitively I have multiple misunderstandings how they work.
def get_queryset(self):
qs = Company.objects.prefetch_related(
Prefetch('products', queryset=Product.objects.only('name').order_by('-updated_at', '-created_at'))).get()
return qs
The error that I receive is:
get() returned more than one Company
Because I closed the prefetch_related method/function with ))) :
I thought get() will act over the Company object and get it using the pk/slug from the url(as get do by default in DetailView). Seems that is not the case.
I'm already using 'products' the related name in the Prefetch object, why in queryset is necessary to tell again the model queryset=Product.objects.... ?
I was looking at the following example in django documentation:
Question.objects.prefetch_related(Prefetch('choice_set')).get().choice_set.all()
If there is 'choice_set' in Prefetch object why is called at the end choice_set.all() ?
Isn't Django attached to the quesryset in prefetch_related the products to the queryset (question.choice_set) ?
I think my problem is that I don't understand the order of execution, and I'm confused how methods are chained, even if are closed by ')'
queryset.get() will only work if the queryset has a single object. If it contains zero or more than one objects, you'll get an error.
You should return a queryset from the get_queryset object. In the class based view, the code that filters on the pk/slug is in get_object.
The prefetch_related method is useful if you want to fetch the products for multiple countries. The way that the Django docs use get() is confusing in my opinion - if the queryset has a single item then prefetch_related is over-complicated.
If you have a single company, then there is no advantage, and the code will be simpler if you fetch the countries separately, e.g. in get_context_data.
def get_context_data(self, **kwargs):
context = super(MyView, self).get_context_data(**kwargs)
context['products'] = Product.objects.filter(company=self.object).Product.objects.only('name').order_by('-updated_at', '-created_at')))
return context
I've removed the only('name') call. It's an optimisation that you probably don't need.
If you really want to use prefetch_related, then remove the get().
qs = Company.objects.prefetch_related(
Prefetch('products', queryset=Product.objects.order_by('-updated_at', '-created_at')))
By specifying the queryset above, you are able to change the order (you could filter it if you like). If you don't want to customize the queryset you can simply do:
Company.objects.prefetch_related('products')
When you use Question.objects.prefetch_related(...), the queryset is still a list of questions. You need to call choice_set.all() on the individual instances to access their choices. This won't cause any additional queries, because Django has already prefetched the choices.
queryset = Question.objects.prefetch_related(Prefetch('choice_set'))
for question in queryset:
print(question) # the question
print(question.choice_set.all()) # the related choices
I am trying to make queries using prefetch_selected to prefetch many to many objects.
Using prefetch_selected when doing an all() query works okay but I do not know how i’m supposed to use it with get() or filter(). Here is my model:
class Project(models.Model):
…
funders = models.ManyToManyField(Organization, related_name="funders")
class Organization(models.Model):
…
name = models.CharField(max_length=200, unique=True)
Trying the lines below doesn’t seem to work:
Project.objects.get(id=project_id).select_related('funders')
and
Project.objects.filter(id__in=[‘list-of-ids]).select_related('funders')
How i’m I supposed to go about it?
Thanks in advance.
You chained them in the wrong order, do it this way:
Project.objects.select_related('funders').get(id=project_id)
and
Project.objects.select_related('funders').filter(id__in=[‘list-of-ids])
You have to call select_related and prefetch_related on the Manager object (Project.objects).
select_related() is a queryset method. The documentation on querysets has two sections of methods: methods that return a new queryset, and methods that do not return a queryset. get() is in the second section, so you can't chain any other queryset methods after it.
One other thing: prefetch_related() runs an extra query for each model. Since you're only fetching a single project, project.funders.all() will run exactly 1 query to fetch all related organizations regardless of your use of prefetch_related(). prefetch_related becomes useful when you need the related organizations for multiple projects.
Since funder is m2m you cannot use select_related, you have to us prefetch instead. select_related only work on foreign key and one to one relation
Project.objects.prefetch_related('funders').filter(id__in=[id1, id2])
I need to get list of all companies and join the company user with minimal companyuser id.
There are two models:
class Company(models.Model):
name = models.CharField(max_length=255)
kind = models.CharField(max_length=255)
class CompanyUser(models.Model):
company = models.ForeignKey('Company')
email = models.EmailField(max_length=40, unique=True)
#other fields
I've tried something like this:
companies = Company.objects.all().select_related(Min('companyuser__email'))
but It doesn't work. How can I do this with Django ORM? Is there any way to do it without raw SQL?
from django.db.models import Min
Company.objects.annotate(lowest_companyuser_id=Min("companyuser__id"))
Explanation
select_related() can be used for telling Django which related tables should be joined to the resulting queryset for reducing the number of queries, namely solving the dreaded "N+1 problem" when looping over a queryset and accessing related objects in iteration. (see docs)
With using Min() you were on the right track, but it ought to be used in conjunction with the annotate() queryset method. Using annotate() with aggregate expressions like Min(), Max(), Count(), etc. translates in an SQL query using one of the aforementioned aggregate expressions with GROUP BY. (see docs about annotate() in Django, about GROUP BY in Postgres docs)
As Burhan said - do not rely on the pk, but if u must...
companies = Company.objects.all().order_by('pk')[0]
I'm trying to optimise my queries but prefetch_related insists on joining the tables and selecting all the fields even though I only need the list of ids from the relations table.
You can ignore the 4th query. It's not related to the question.
Related Code:
class Contact(models.Model):
...
Groups = models.ManyToManyField(ContactGroup, related_name='contacts')
...
queryset = Contact.objects.all().prefetch_related('Groups')
Django 1.7 added Prefetch objects which let you customise the queryset used when prefetching.
In particular, see only().
In this case, you'd want something like:
queryset = Contact.objects.all().prefetch_related(
Prefetch('Groups', queryset=Group.objects.all().only('id')))