Using prefetch_related for reducing queries overhead - django

I have two models.
class Category(models.Model):
title = models.CharField(max_length=60, unique=True)
def __unicode__(self):
return self.title
class Post(models.Model):
title = models.CharField(max_length=60, unique=True)
category = models.ForeignKey(Category)
#code
views.py
class PostList(ListView):
model = Post
def get_queryset(self):
queryset = Post.objects.all().\
select_related('category')
But for my main page i need further all categories for the navbar. What the best approach to add categories to get_queryset()?
I tried to use prefetch_related.
post = Post.objects.all().prefetch_related('category')
but i don't understand how to fetch all categories.
Is it correct solution?
class PostList(ListView):
def get_queryset(self):
p = Post.objects.all().\
select_related('category') #like tag for each post
p.categories = Category.objects.all() #all categories for navbar
return p
Or Django has own methods for this task?

I don't understand what this has to do with either select_related or prefetch_related. Neither of those will help in getting categories that are not related to your posts: as their names imply, they are to do with getting related items, not unrelated ones.
If you need all the categories, you should simply do Category.objects.all().

Related

How can I prefetch nested tables?

I'm working on an app with a DRF API. Development has been going on for some months, but only now are we running into performance issues when populating the database with actual data. I have done some profiling and found out that many endpoints query the database hundreds of times to fetch the necessary data. This is something that can be solved with select_related and prefetch_related, that much I know, but I'm having a hard time picturing it.
Models
class LegalFile(models.Model):
code = models.CharField(max_length=75)
description = models.CharField(max_length=255)
contactrole = models.ManyToManyField(LegalFileContactRole, blank=True, related_name='contactroles')
class LegalFileContactRole(models.Model):
contact = models.ForeignKey(Contact, on_delete=models.DO_NOTHING, blank=True, related_name='legal_file_contact')
subtype = models.ForeignKey(ContactSubType, on_delete=models.DO_NOTHING, related_name='legal_file_contact_role', null=True)
class Contact(models.Model):
name = models.CharField(max_length=150)
surname_1 = models.CharField(max_length=50,blank=True, null=True)
surname_2 = models.CharField(max_length=50,blank=True, null=True)
class ContactSubType(models.Model):
pass
Say I want to list all the LegalFiles in the database, and fetch the names and both surnames of the contacts associated to each LegalFile through LegalFileContactRole. Is there a DRYer way than using lowercase notation to prefetch like LegalFile.prefetch_related('contactrole__contact__name', 'contactrole__contact__surname_1', 'contactrole__contact__surname_2')? This kind of nested relationship is a recurring thing in the app.
Editing in response to Marc Compte's comment:
I have not explained myself properly, I think. In fact, this is missing a good chunk of context - I have not mentioned serializers at all, for instance.
I have taken the approach described in this post and created a method to set eager loading up:
class LegalFileReadSerializer(serializers.ModelSerializer):
contactrole = LegalFileContactRoleSerializer(many=True)
#classmethod
def setup_eager_loading(cls, queryset):
queryset = queryset.prefetch_related('contactrole__contact__name', 'contactrole__contact__surname_1', 'contactrole__contact__surname_2')
return queryset
class Meta:
model = LegalFile
fields = '__all__'
read_only_fields = ('__all__',)
class LegalFileViewSet(CustomViewSet):
model = models.LegalFile
read_serializer_class = serializers.LegalFileReadSerializer
write_serializer_class = serializers.LegalFileWriteSerializer
def get_queryset(self):
queryset = super().get_queryset()
queryset = self.get_serializer_class().setup_eager_loading(queryset)
return queryset
(Note this is still work in progress and will have to be decoupled further for reuse, in a mixin or something)
Could I, so to speak, chain serializers? Like:
class LegalFileReadSerializer(serializers.ModelSerializer):
contactrole = LegalFileContactRoleSerializer(many=True)
#classmethod
def setup_eager_loading(cls, queryset):
queryset = queryset.prefetch_related('contactrole',)
return queryset
class Meta:
model = LegalFile
fields = '__all__'
read_only_fields = ('__all__',)
class LegalFileContactRoleSerializer(serializers.ModelSerializer):
contact = ContactReadSerializer()
subtype = ContactSubTypeReadSerializer()
#classmethod
def setup_eager_loading(cls, queryset):
queryset = queryset.prefetch_related('contact',)
return queryset
class Meta:
model = models.LegalFileContactRole
fields = '__all__'
class ContactReadSerializer(serializers.ModelSerializer):
# And so on
pass
This, in my head at least, makes sense, but I don't know if this behavior is possible or even advisable. It seems that if I chain serializers like this (implying again that it's possible to do so) I'm gonna run into problems picking up too much unneeded information.

querysets in models with manytomany fields (Django)

My question is associated with making querysets on models that are interconnected across many to many fields.
context - an app that a student enters and has to rate his or her teachers. the questions the app shows the student have the following logic: each student must rate some teachers. Each teacher has different categories of questions associated with them ("intelligence", "respect", "empathy",etc.) and each of these categories has some questions associated with it.
The models are:
class Items(models.Model):
item = models.TextField()
def __str__(self):
return self.item
class Categories(models.Model):
category = models.CharField(max_length=50,null=True)
items_associated = models.ManyToManyField(Items)
def __str__(self):
return self.category
class Professors(models.Model):
professor = models.CharField(max_length=50,null=True)
categories_assigned = models.ManyToManyField(Categories)
def __str__(self):
return self.professor
class Students(models.Model):
student_logged = models.CharField(max_length=50,null=True)
professors_to_evaluate = models.ManyToManyField(Professors)
def __str__(self):
return self.student_logged
when a student enters the web has some associated teachers (model Students) these teachers in turn have some categories assigned (model Professors), these categories in turn have some questions associated (model Categories). I want to store in a dictionary these questions that are in the model Items. How can I do it?
I've tried to filter and __in but I can't get it.
Many thanks and thank you for the wisdom
I highly suggest you use the related_name attribute. I've added _x to the related names to make the query more obvious.
class Items(models.Model):
item = models.TextField()
def __str__(self):
return self.item
class Categories(models.Model):
category = models.CharField(max_length=50,null=True)
items_associated = models.ManyToManyField(Items, related_name='category_x')
def __str__(self):
return self.category
class Professors(models.Model):
professor = models.CharField(max_length=50,null=True)
categories_assigned = models.ManyToManyField(Categories, related_name='professor_x')
def __str__(self):
return self.professor
class Students(models.Model):
student_logged = models.CharField(max_length=50,null=True)
professors_to_evaluate = models.ManyToManyField(Professors, related_name='student_x')
def __str__(self):
return self.student_logged
items_for_student = Items.objects.filter(category_x__professor_x__student_x=student)
But also the naming conventions you are using for the fields are a bit quirky. I've used best practices below so you can see what that would look like.
Don't have a field with the same name as the model
Models should be singular (with rare exceptions)
ManyToMany or ForeignKey relations should share the name of the model to make querying self document.
With those rules here is what the best practice looks like.
class Item(models.Model):
name = models.TextField()
def __str__(self):
return self.name
class Category(models.Model):
name = models.CharField(max_length=50,null=True)
items = models.ManyToManyField(Item, related_name='categories')
def __str__(self):
return self.name
class Professor(models.Model):
name = models.CharField(max_length=50,null=True)
categories = models.ManyToManyField(Category, related_name='professors')
def __str__(self):
return self.name
class Student(models.Model):
name = models.CharField(max_length=50,null=True)
professors = models.ManyToManyField(Professor, related_names='students')
def __str__(self):
return self.name
And with that structure the query would look like:
items = Item.objects.filter(categories__professors__students=student)
Also note that the above query will be very expensive to run on a database as it would evaluate to 3 joins.

Update several objects with formset

I'm 1st year on CS so forgive my noobness if i am totally unclear.
I have several objects from my "Products" Model. Now I would like to update the same field on all of the objects, the 'quantity' field with different values. But instead of clicking in and out of each product as with updateview, i would like to list all products and set the value for each and change them at the simultaneously. As far as i can see "FormSet" should do the trick?
My category model looks like this (to assign for a product)
class Category(models.Model):
category = models.CharField(max_length=30)
def __str__(self):
return self.category
My Product model looks like this:
class Product(models.Model):
title = models.CharField(max_length=200)
description = models.CharField(max_length=200)
category = models.ForeignKey(Category)
price = models.DecimalField(max_digits=5, decimal_places=2)
stock = models.PositiveIntegerField()
def __str__(self):
return self.title
My update view for updating single product looks like this:
class UpdateProductView(UpdateView):
model = Product
form_class = ProductForm
template_name = "product_form.html"
success_url = '/products'
class CreateCategoryView(FormView):
template_name = "category_form.html"
form_class = CategoryForm
I read the documentation on formset, but i gotta admit i didn't feel much smarter about how to actually use it afterwards.. can anyone give a hand?
Didn't find quantity field on your Product model, but as I can see, you want to use ModelFormSet.
# Generate your formset class with `modelformset_factory`
ProductFormSetClass = modelformset_factory(Product, fields=('quantity',))
# Now you can create your formset, bind data, etc
formset = ProductFormSetClass(request.POST)
formset.save()
More details by link:
https://docs.djangoproject.com/en/1.9/topics/forms/modelforms/#model-formsets
Also, don't forget to inspect all modelformset_factory arguments.

Paginate category by posts in django

I've got a blog system, with categories and posts, as shown simplified below.
I want to have a detail page for category, where I paginate the posts belonging to that category.
class Category(models.Model):
name = models.CharField(max_length=150)
slug = models.SlugField()
class Post(models.Model):
title = models.CharField(max_length=90)
category = models.ForeignKey(Category, related_name="posts")
I use django 1.4 and class based views. What is the correct way of doing this? I guess I need to use paginate_queryset(queryset, page_size), but I don't know how. This is my basis for the view:
class CategoryDetailView(DetailView):
model = Category
queryset = object.posts.all()
def paginate_queryset(self, queryset, page_size):
pass
You might have a look at the docs. It should help to get you started.

How do I reference Django Model from another model

Im looking to create a view in the admin panel for a test program which logs Books, publishers and authors (as on djangoproject.com)
I have the following two models defined.
class Author(models.Model):
first_name = models.CharField(max_length=30)
last_name = models.CharField(max_length=30)
email = models.EmailField()
def __unicode__(self):
return u'%s %s' % (self.first_name, self.last_name)
class Book(models.Model):
title = models.CharField(max_length=100)
authors = models.ManyToManyField(Author)
publisher = models.ForeignKey(Publisher)
publication_date = models.DateField()
def __unicode__(self):
return self.title
What I want to do, is change the Book model to reference the first_name of any authors and show this using admin.AdminModels.
#Here is the admin model I've created.
class BookAdmin(admin.ModelAdmin):
list_display = ('title', 'publisher', 'publication_date') # Author would in here
list_filter = ('publication_date',)
date_hierarchy = 'publication_date'
ordering = ('-publication_date',)
fields = ('title', 'authors', 'publisher', 'publication_date')
filter_horizontal = ('authors',)
raw_id_fields = ('publisher',)
As I understand it, you cannot have two ForeignKeys in the same model. Can anyone give me an example of how to do this?
I've tried loads of different things and its been driving me mad all day. Im pretty new to Python/Django.
Just to be clear - I'd simply like the Author(s) First/Last name to appear alongside the book title and publisher name.
Thanks
You can have more than one Foreign Key on a model.
If you would put a Foreign-Key field's name in list_display you will always just see the __unicode__ representation of the associated model.
But you can add a function like this to your BookAdmin:
def first_names(self, obj):
return ','.join(a.first_name for a in obj.authors.all())
get_sites.short_description = 'First Names'
Then add 'first_names' to list_display.