I'm wondering. If I use select_related on the view, will it save a database hit if I use the object on the template?
Lets say:
views.py
one_thing = things.objects.filter(...).select_related("another_thing")
template.html
<p>{{ one_thing.another_thing }}</p>
The docs about select_related write:
Returns a QuerySet that will “follow” foreign-key relationships,
selecting additional related-object data when it executes its query.
This is a performance booster which results in a single more complex
query but means later use of foreign-key relationships won’t require
database queries.
The template code for django is executed in the backend and render the data in the template when the page finishes loading, so there is no difference whether you use it in your views.py or in your template.
Related
I have 2 models Product an Categories. between them there is a many-to-many-relation.
I have the following code:
In view:
context['products']Product.objects.all().prefetch_related(Prefetch('categories', queryset=Category.objects.only('name')))
In template:
{% for product in products %}
{{ product.categories.all.0.name}}
In this case prefetch_related is ignore, and it exeutes anew query in categories for each product.
If I remove only, it executes just 2 queries
Why ?
You can't use a queryset with only the name field, as Django will at least need the product_id to match the Category object with the Product - since you have excluded it, Django will automatically make a separate query each time to fetch it. This happens at query time, even before the object it sent to the template.
Category.objects.only('name', 'product_id') would work.
My models are similar to the following:
class Reporter(models.Model):
def gold_star(self):
return self.article_set.get().total_views >= 100000
class Article(models.Model):
reporter = models.ForeignKey(Reporter, on_delete=models.CASCADE)
total_views = models.IntegerField(default=0, blank=True)
Then in one of the templates I have this line:
{% if r.gold_star %}<img src="{% static 'gold-star.png' %}">{% endif %}
Obviously django sends as many queries as there are reporters on the page... Ideally this could be just one query, which would select reporters by criteria and join appropriate articles. Is there a way?
EDIT
Neither select_related nor prefetch_related doesn't seem to work as I'm selecting on the Reporter table and then use RelatedManager to access related data on the Article.
In other words django doesn't know what to prefetch until there's non empty queryset.
Because an article can only have one reporter it's for sure possible to join these tables together and then apply filter to subquery, I just can't find how it's done in django query language.
There's alternative - select on the Article table and filter by Reporter fields, but there's a problem with such approach. If I deleted all the articles of some reporter then I wouldn't be able to include that reporter in the list as from the Article point of view such reporter doesn't exist and yet reporter is in the Reporter table.
EDIT2
I tried what people suggested in the comments. The following generates desired query:
reporters = Reporter.objects.filter(**query).select_related().annotate(
gold_star=Case(
When(article__total_views__gte=0, then=Value(1)),
default=Value(0),
output_field=IntegerField()
)
)
Query generated by django:
SELECT
`portal_reporter`.`id`,
...,
CASE WHEN `portal_article`.`total_views` >= 0 THEN 1 ELSE 0 END AS `gold_star`
FROM
`portal_reporter`
LEFT OUTER JOIN `portal_article`
ON (`portal_reporter`.`id` = `portal_article`.`reporter_id`)
WHERE
...
Now I just need to work out a way how to produce similar query but without Case/When statements.
EDIT3
If I chose slightly different strategy, then django selects wrong join type:
query['article__id__gte'] = 0
reporters = Reporter.objects.filter(**query).select_related()
This code produce similar query but with the INNER JOIN instead of desired LEFT OUTER JOIN:
SELECT
`portal_reporter`.`id`,
...,
FROM
`portal_reporter`
INNER JOIN `portal_article`
ON (`portal_reporter`.`id` = `portal_article`.`reporter_id`)
WHERE
...
You can use select_related (https://docs.djangoproject.com/en/1.11/ref/models/querysets/#select-related) to do a join on the related table.
There's also prefetch_related (https://docs.djangoproject.com/en/1.11/ref/models/querysets/#prefetch-related) which uses an IN clause to fetch the related objects with an extra query. The difference is explained in the docs, but is reproduced below:
select_related works by creating an SQL join and including the fields of the related object in the SELECT statement. For this reason, select_related gets the related objects in the same database query. However, to avoid the much larger result set that would result from joining across a ‘many’ relationship, select_related is limited to single-valued relationships - foreign key and one-to-one.
prefetch_related, on the other hand, does a separate lookup for each relationship, and does the ‘joining’ in Python. This allows it to prefetch many-to-many and many-to-one objects, which cannot be done using select_related, in addition to the foreign key and one-to-one relationships that are supported by select_related. It also supports prefetching of GenericRelation and GenericForeignKey, however, it must be restricted to a homogeneous set of results. For example, prefetching objects referenced by a GenericForeignKey is only supported if the query is restricted to one ContentType.
Try annotating the new field gold_star and set it to 1 if reporter has an article that has more than 100000 total_views like this:
from django.db.models import Case, When, Value, IntegerField
reporters = Reporter.objects.annotate(
gold_star=Case(
When(article__total_views__gte=100000, then=Value(1)),
default=Value(0),
output_field=IntegerField()
)
)
You can leave the template code as it is.
This QuerySet (Let's say Model model has 12 field):
objects = Model.objects.filter(...)
And this template:
{% for object in object %}
<a href='{{ object.get_absolut_url }}'>Foo: {{ object.bar }}</a>
{% endfor %}
perform SQL query which gets unnecessary fields (every 12 fields + relations). I want Django to get only 'bar' field. How can I do this?
By the way I know about values() method, but as it returns dict, I can't call Model methods such as get_absolute_url().
You want to use only():
objects = Model.objects.select_related().only("bar").filter(...)
Keep in mind however if you limit too much of the data down and then use the objects in other ways you can actually cause the ORM to execute extra queries, so make sure to be using something like django-debug-toolbar to ensure you aren't removing these unnecessary fields to only incur the hit of lots of unnecessary queries which is a worse situation.
FYI you can also use defer() to list the fields you don't want loaded if you want to think about it in the other direction.
Say I have a model:
class Entry(models.Model):
...
tags = TaggableManager()
When I iterate over Entry.objects.all() in a template, entry.tags.all produces one more query to the database. Is it possible to reduce queries number? Using something like select_related() (I know it won't work, since django-taggit uses manytomany relation, but I am sure there should be a way to select all entries with related tags in 1 hit)?
From Django 1.4 onward, you can use prefetch_related to retrieve one-to-many relations on a queryset in a single query. Unfortunately this doesn't work brilliantly with django-taggit, because the 'tags' property is a manager rather than a true relation, and so prefetch_related can't make sense of it. Instead, you need to follow the tagged_items relation:
entries = Entry.objects.prefetch_related('tagged_items__tag')
You then need to go through some similar contortions in the template code to access the prefetched tags, because entry.tags.all will run another query rather than making use of the prefetch:
{% for tagged_item in entry.tagged_items %}
<li>{{ tagged_item.tag.name }}</li>
{% endfor %}
Try using Select Reverse it's designed to grab the entirity of a many2many relationship with a single query.
I'm building a "preview" function into the CMS for my website which uses the existing front-end template to render a model. This model has an association:
class FeatureWork(models.Model):
name = models.CharField(max_length=100)
...
class FeatureWorkLink(models.Model):
feature_work = models.ForeignKey(FeatureWork)
In the view for the preview, I'm trying to build the model such that when the template calls feature.featureworklink_set.all it returns the associated links. Since neither model has been saved yet, all the standard Django form techniques seem to go out the window.
This is what I have so far, but it blows up when I call the add method on the manager, since the parent hasn't been saved yet:
form = FeatureWorkAdminForm(initial=request.POST)
featured = form.save(commit=False)
for link in request.POST['links'].split(","):
featured.featureworklink_set.add(FeatureWorkLink(image=link))
Why don't you just add:
preview = BooleanField()
to your models, save everything to database and don't look for hacks. This way you can have drafts for free.
You can actually save it in transaction and rollback when template is ready. It's not very efficient, but at least it will work.
featured.featureworklink_set.add(FeatureWorkLink(image=link)) will immediately attempt to create a relationship between a FeatureWork and FeatureWorkLink, which isn't going to happen because that instance of FeatureWork is not present in the database and you cannot satisfy the predicates for building the foreign key relationship.
But the great thing is that Django's Model and ModelForm instances will not validate foreign key relationships until your actually trying to commit the data. So manually constructing your FeaturedWorkLink with an uncommited, non-existing FeatureWork should satisfy any presentation of the data you need to do, much to what you'd expect:
links = []
form = FeatureWorkAdminForm(initial=request.POST)
featured = form.save(commit=False)
for link in request.POST['links'].split(","):
links.add(FeatureWorkLink(image=link, feature_work=featured))
# then somewhere in your templates, from the context
{% for link in links %}
<img src="{{ link.image }}"
title="Image for the featured work: '{{ link.feature_work.name }}'" />
{% endfor %}
So basically, during the course of collecting the data to create a FeatureWork, you'll have to maintain the FeatureWorkLink instances through subsequent requests. This is where you'd use a model form set, but provide an uncommitted FeatureWork for the feature_work property for every model form instance of the set, up until the point where all the data has been collected, where you then provide a commited FeatureWork instance, so that the model form set can satisfy referential integrity and finally be commited to the database.