I have 2 models Product an Categories. between them there is a many-to-many-relation.
I have the following code:
In view:
context['products']Product.objects.all().prefetch_related(Prefetch('categories', queryset=Category.objects.only('name')))
In template:
{% for product in products %}
{{ product.categories.all.0.name}}
In this case prefetch_related is ignore, and it exeutes anew query in categories for each product.
If I remove only, it executes just 2 queries
Why ?
You can't use a queryset with only the name field, as Django will at least need the product_id to match the Category object with the Product - since you have excluded it, Django will automatically make a separate query each time to fetch it. This happens at query time, even before the object it sent to the template.
Category.objects.only('name', 'product_id') would work.
Related
Rendering a page gets really slow when related objects are requested in a template.
class Item(models.Model):
name = models.CharField(max_length=160)
...
class Box(models.Model):
...
items = models.ForeignKey(Item, on_delete=models.CASCADE, null=True)
#template
{% for item in items %}
{{ item.box_set.first }}
{{ item.box_set.latest }}
{% endfor %}
Debug toolbar shows there many duplicate queries.
Why is this happening? Is there a way to speed this up?
The Django ORM has to make a request to the database when accessing a related field unless it's already cached. The main ways of caching the related objects are via select_related and prefetch_related.
What you're trying to do is a bit more difficult; you're trying to get two specific items from a collection. You can use .annotate() and Subquery to pull singular fields from a related model. This would be useful if you just wanted to display single field from box, but if you need the whole box instance this won't work.
Trying to figure out Django... I'd like to build a table in a template that is largely all the data from a Django model. My preference would be to send the model queryset PersonDynamic.objects.all() to the template and have it figure out the column names and then loop through each row, adding the fields for that row. This kind of approach would give me a lot of flexibility because if the model changes the template will automatically change with it.
One approach could be to just move the column headers and each record into lists which I then send to the template but this seems less efficient than just sending the queryset.
What is the best way to do this?
Am I right about efficiency?
Does sending lists rather than model instances come with a larger risk of problems if there are a bazillion records or does that risk exist, regardless of approach?
Thanks!
Try looking at the code for django.contrib.admin, and how the built-in admin site makes tables. The admin site adds a lot of extra features to change how fields are displayed, and what fields to display, but it should give you some idea of how things work.
The absolute simplest way I can think to do what you want is to send the template a list of headers you want to use, and the queryset called with the values_list method. values_list takes a series of field names, and returns another queryset that will yield a tuple of field values instead of a model object. For example, say your PersonDynamic looks like this:
class PersonDynamic(Model):
id = AutoField()
name = CharField()
likes_python = BooleanField()
You could then send something like this to your template:
fields = ['id', 'name', 'likes_python']
qs = PersonDynamic.objects.values_list(*fields)
context = Context({'headers': fields, 'rows': qs})
template.render(context)
And your template could do:
<table>
<tr>{% for item in header %}<th>{{ item }}</th>{% endfor %}</tr>
{% for row in rows %}
<tr>{% for cell in row %}<td>{{ cell }}</td>{% endfor %}</tr>
{% endfor %}
</table>
For your other questions, this will run into problems for larger tables; it takes time and memory to loop over large lists like this, so generally a strategy like pagination is used, where only X items are displayed per page.
I'm wondering. If I use select_related on the view, will it save a database hit if I use the object on the template?
Lets say:
views.py
one_thing = things.objects.filter(...).select_related("another_thing")
template.html
<p>{{ one_thing.another_thing }}</p>
The docs about select_related write:
Returns a QuerySet that will “follow” foreign-key relationships,
selecting additional related-object data when it executes its query.
This is a performance booster which results in a single more complex
query but means later use of foreign-key relationships won’t require
database queries.
The template code for django is executed in the backend and render the data in the template when the page finishes loading, so there is no difference whether you use it in your views.py or in your template.
So I have a a very simple (simplified) model
class MyObject(models.Model):
owning_user = models.ForeignKey(User, null=True)
Now in one of my templates I'm trying to iterate over a list of these objects to determine whether something should be displayed similar to this
{% for my_object in foo.my_object_set %}
{% if my_object.owning_user.id == user.id %}
Show Me!
{% endif %}
This works fine, but what I am finding is that the query
my_object.owning_user.id
returns every field from the owning user before getting the id which is verified both in django debug tool bar, and checking the connection queries
# django-debug-toolbar states this is repeated multiple times
SELECT ••• FROM "auth_user" WHERE "auth_user"."id" = 1
# The following test code also confirms this
from django.db import connection
conn = connection
bearing_type.owning_user.id
print conn.queries[-1]
Now since this query repeats over 1000 times and takes 2ms per query it is taking 2 seconds just to perform this - when all I care about is the id...
Is there anyway at all that I can just perform a query to get just the id from the owning_user instead of having to query all the fields?
Note, I'm trying really hard here to avoid making a raw query
If you use my_object.owning_user_id instead of my_object.owning_user, then Django will use the id instead of looking up the user.
In this case, you only need the id, but if you needed other user attributes, you could use select_related. In the view, you would do:
foo.my_object_set.select_related('user')
In the template, you don't have as much control, since you can't pass arguments.
{{ foo.my_object_set.select_related }}
Say I have a model:
class Entry(models.Model):
...
tags = TaggableManager()
When I iterate over Entry.objects.all() in a template, entry.tags.all produces one more query to the database. Is it possible to reduce queries number? Using something like select_related() (I know it won't work, since django-taggit uses manytomany relation, but I am sure there should be a way to select all entries with related tags in 1 hit)?
From Django 1.4 onward, you can use prefetch_related to retrieve one-to-many relations on a queryset in a single query. Unfortunately this doesn't work brilliantly with django-taggit, because the 'tags' property is a manager rather than a true relation, and so prefetch_related can't make sense of it. Instead, you need to follow the tagged_items relation:
entries = Entry.objects.prefetch_related('tagged_items__tag')
You then need to go through some similar contortions in the template code to access the prefetched tags, because entry.tags.all will run another query rather than making use of the prefetch:
{% for tagged_item in entry.tagged_items %}
<li>{{ tagged_item.tag.name }}</li>
{% endfor %}
Try using Select Reverse it's designed to grab the entirity of a many2many relationship with a single query.