get_queryset vs manager in Django - django

As far as I know, we bring the database by model managers right?
e.g. queryset = Model.objects.all()
But sometimes, I see some code that seems almost same thing but is a bit different,
post = self.get_queryset()
which also fetches database but not by manager.
What's the difference between fetching database by manager and get_queryset() and their usage?

The below example helps you to understand what ModelManager and get_queryset are:
class PersonQuerySet(models.QuerySet):
def authors(self):
return self.filter(role='A')
def editors(self):
return self.filter(role='E')
class PersonManager(models.Manager):
def get_queryset(self):
return PersonQuerySet(self.model, using=self._db)
def authors(self):
return self.get_queryset().authors()
def editors(self):
return self.get_queryset().editors()
class Person(models.Model):
first_name = models.CharField(max_length=50)
last_name = models.CharField(max_length=50)
role = models.CharField(max_length=1, choices=(('A', _('Author')), ('E', _('Editor'))))
people = PersonManager()
Every model have at least one model manager and get_queryset is the model manager base QuerySet.
When you use Person.objects.all() it will return all results from Person model, not filters or anything else.
In the above example, we use a custom model manager named PersonManager, where we override get_queryset.
Firstly, we apply authors filter with role='A'.
Secondly, we apply editors filter with role='E'.
So, now if we use Person.people.all() it will return only authors. Look here we use all() but it returns only authors. Because we override the Default model manager queryset.

Related

Why does a queryset applied in a ModelForm not inherit a queryset from a ModelManager?

I have a custom queryset on a model manager:
class TenantManager(models.Manager):
def get_queryset(self):
return super().get_queryset().filter(myfield=myvalue)
class TenantModel(TenantModelMixin, models.Model):
objects = TenantManager()
class Meta:
abstract = True
I use the abstract TenantModel as a mixin with another model to apply the TenantManager. E.g.
class MyModel(TenantModel):
This works as expected, applying the TenantManager filter every time MyModel.objects.all() is called when inside a view.
However, when I create a ModelForm with the model, the filter is not applied and all results (without the filter are returned. For example:
class AddPersonForm(forms.ModelForm):
class Meta:
model = MyModel
fields = ('person', )
Why is this and how to I ensure the ModelManager is applied to the queryset in ModelForm?
Edit
#Willem suggests the reason is forms use ._base_manager and not .objects (although I can not find this in the Django source code), however the docs say not to filter this kind of manager, so how does one filter form queries?
Don’t filter away any results in this type of manager subclass
This
manager is used to access objects that are related to from some other
model. In those situations, Django has to be able to see all the
objects for the model it is fetching, so that anything which is
referred to can be retrieved.
If you override the get_queryset() method and filter out any rows,
Django will return incorrect results. Don’t do that. A manager that
filters results in get_queryset() is not appropriate for use as a base
manager.
You can do it in two ways:
First: When creating the form instance, add the queryset for the desired field.
person_form = AddPersonForm()
person_form.fields["myfield"].queryset = TenantModel.objects.filter(myfield="myvalue")
Second: Override the field's queryset in the AddPersonForm itself.
class AddPersonForm(forms.ModelForm):
class Meta:
model = MyModel
fields = ('person', )
def __init__(self, *args, **kwargs):
super(AddPersonForm, self).__init__(*args, **kwargs)
self.fields['myfield'].queryset = TenantModel.objects.filter(myfield="myvalue")
I'm not sure why your code doesn't properly works. Probably you haven't reload django app. You could load queryset in __init__ of your form class
class AddPersonForm(forms.ModelForm):
person = forms.ModelMultipleChoiceField(queryset=None)
class Meta:
model = MyOtherModel
fields = ('person', )
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.fields['person'].queryset = MyModel.objects.all()

Accessing parent model instance within model admin to provide custom queryset

I want to provide a custom queryset within a model admin class that inherits from TabluarInline, but I want to provide this queryset by calling a method of current instance of the model object.
I have two models. One for tracks belonging to an album, and one for the Album itself. Some tracks can be hidden and I have a method in Album to return only the visible tracks.
class Track(models.Model):
name = models.CharField()
length = models.IntegerField()
album = ForeignKey(Album)
hidden = BooleanField()
class Album(models.Model):
name = models.CharField()
def get_visible_tracks_queryset(self):
return self.track_set.filter(hidden=False)
And I have a tracks inline admin which is included on the django admin page for an album. I want to re-use the get_visible_tracks_queryset to define the queryset for this inline admin, I don't want to repeat the logic again. I can't figure out how to do it. I could do something like the following, however I'm using a simplified example here, I actually have more complex logic and I don't want to be repeating the logic in multiple places.
class TracksInlineAdmin(admin.TabularInline):
fields = ("name", "length")
model = Track
def get_queryset(self, request):
qs = super(TracksInlineAdmin, self).get_queryset(request)
return qs.filter(hidden=False)
Ideally I could do something like:
class TracksInlineAdmin(admin.TabularInline):
fields = ("name", "length")
model = Track
def get_queryset(self, request, parent_model_instance):
return parent_model_instance.get_visible_tracks_queryset()
Any thoughts on how to achieve this?
The cleanest way is to define a custom QuerySet class for your model in which you can define any complex filters for re-use in various places:
class Track(models.Model):
# fields defined here
objects = TrackManager()
class TrackManager(models.Manager):
def get_queryset(self):
return TrackQuerySet(self.model, using=self._db)
class TrackQuerySet(models.QuerySet):
def visible(self):
return self.filter(hidden=False)
Now, anywhere in code, when you have a queryset of tracks (e.g. Track.objects.filter(name="my movie")) you can add .visible() to filter further. Also on a related set:
album.track_set.all().visible()

Django Beginner. How do I update all objects and set a certain field to a value that is a function of another field?

EDIT:
I needed a student_count field in course because I'm supposed to use this model for REST API. If you can tell me how to add fields in serializer without adding to model, I'd take that too.
This is my model:
class Course(models.Model):
student_count = models.PositiveIntegerField(default=0)
students = models.ManyToManyField()
What I try to do is the following but it doesn't work.
Course.objects.update(student_count=F('students__count'))
The following works but it's not ideal
courses = Course.objects.all()
for course in courses:
course.student_count = course.students.count()
course.save()
return Course.objects.all()
Try to add/update your serializer as below,
from django.db.models import Count
class CourseSerializer(serializers.ModelSerializer):
count = serializers.SerializerMethodField(read_only=True)
def get_count(self, model):
return model.students.aggregate(count=Count('id'))['count']
class Meta:
model = Course
fields = ('count',)
You can use aggregation Count() method instead of queryset.count() its more faster. Read this SO post, aggregate(Count()) vs queryset.count()

Django REST Framework: Setting up prefetching for nested serializers

My Django-powered app with a DRF API is working fine, but I've started to run into performance issues as the database gets populated with actual data. I've done some profiling with Django Debug Toolbar and found that many of my endpoints issue tens to hundreds of queries in the course of returning their data.
I expected this, since I hadn't previously optimized anything with regard to database queries. Now that I'm setting up prefetching, however, I'm having trouble making use of properly prefetched serializer data when that serializer is nested in a different serializer. I've been using this awesome post as a guide for how to think about the different ways to prefetch.
Currently, my ReadingGroup serializer does prefetch properly when I hit the /api/readinggroups/ endpoint. My issue is the /api/userbookstats/ endpoint, which returns all UserBookStats objects. The related serializer, UserBookStatsSerializer, has a nested ReadingGroupSerializer.
The models, serializers, and viewsets are as follows:
models.py
class ReadingGroup(models.model):
owner = models.ForeignKeyField(settings.AUTH_USER_MODEL)
users = models.ManyToManyField(settings.AUTH_USER_MODEL)
book_type = models.ForeignKeyField(BookType)
....
<other group related fields>
def __str__(self):
return '%s group: %s' % (self.name, self.book_type)
class UserBookStats(models.Model):
reading_group = models.ForeignKey(ReadingGroup)
user = models.ForeignKey(settings.AUTH_USER_MODEL)
alias = models.CharField()
total_books_read = models.IntegerField(default=0)
num_books_owned = models.IntegerField(default=0)
fastest_read_time = models.IntegerField(default=0)
average_read_time = models.IntegerField(default=0)
serializers.py
class ReadingGroupSerializer(serializers.ModelSerializer):
users = UserSerializer(many = True,read_only=True)
owner = UserSerializer(read_only=True)
class Meta:
model = ReadingGroup
fields = ('url', 'id','owner', 'users')
#staticmethod
def setup_eager_loading(queryset):
#select_related for 'to-one' relationships
queryset = queryset.select_related('owner')
#prefetch_related for 'to-many' relationships
queryset = queryset.prefetch_related('users')
return queryset
class UserBookStatsSerializer(serializers.HyperlinkedModelSerializer):
reading_group = ReadingGroupSerializer()
user = UserSerializer()
awards = AwardSerializer(source='award_set', many=True)
class Meta:
model = UserBookStats
fields = ('url', 'id', 'alias', 'total_books_read', 'num_books_owned',
'average_read_time', 'fastest_read_time', 'awards')
#staticmethod
def setup_eager_loading(queryset):
#select_related for 'to-one' relationships
queryset = queryset.select_related('user')
#prefetch_related for 'to-many' relationships
queryset = queryset.prefetch_related('awards_set')
#setup prefetching for nested serializers
groups = Prefetch('reading_group', queryset ReadingGroup.objects.prefetch_related('userbookstats_set'))
queryset = queryset.prefetch_related(groups)
return queryset
views.py
class ReadingGroupViewset(views.ModelViewset):
def get_queryset(self):
qs = ReadingGroup.objects.all()
qs = self.get_serializer_class().setup_eager_loading(qs)
return qs
class UserBookStatsViewset(views.ModelViewset):
def get_queryset(self):
qs = UserBookStats.objects.all()
qs = self.get_serializer_class().setup_eager_loading(qs)
return qs
I've optimized the prefetching for the ReadingGroup endpoint (I actually posted about eliminating duplicate queries for that endpoint here), and now I'm working on the UserBookStats endpoint.
The issue I'm having is that, with my current setup_eager_loading in the UserBookStatsSerializer, it doesn't appear to use the prefetching set up by the eager loading method in the ReadingGroupSerializer. I'm still a little hazy on the syntax for the Prefetch object - I was inspired by this excellent answer to try that approach.
Obviously the get_queryset method of UserBookStatsViewset doesn't call setup_eager_loading for the ReadingGroup objects, but I'm sure there's a way to accomplish the same prefetching.
prefetch_related() supports prefetching inner relations by using double underscore syntax:
queryset = queryset.prefetch_related('reading_group', 'reading_group__users', 'reading_group__owner')
I don't think Django REST provides any elegant solutions out of the box for fetching all necessary fields automatically.
An alternative to prefetching all nested relationships manually, there is also a package called django-auto-prefetching which will automatically traverse related fields on your model and serializer to find all the models which need to be mentioned in prefetch_related and select_related calls. All you need to do is add in the AutoPrefetchViewSetMixin to your ViewSets:
from django_auto_prefetching import AutoPrefetchViewSetMixin
class ReadingGroupViewset(AutoPrefetchViewSetMixin, views.ModelViewset):
def get_queryset(self):
qs = ReadingGroup.objects.all()
return qs
class UserBookStatsViewset(AutoPrefetchViewSetMixin, views.ModelViewset):
def get_queryset(self):
qs = UserBookStats.objects.all()
return qs
Any extra prefetches with more complex Prefetch objects can be added in the get_queryset method on the ViewSet.

How can I apply a filter to a nested resource in Django REST framework?

In my app I have the following models:
class Zone(models.Model):
name = models.SlugField()
class ZonePermission(models.Model):
zone = models.ForeignKey('Zone')
user = models.ForeignKey(User)
is_administrator = models.BooleanField()
is_active = models.BooleanField()
I am using Django REST framework to create a resource that returns zone details plus a nested resource showing the authenticated user's permissions for that zone. The output should be something like this:
{
"name": "test",
"current_user_zone_permission": {
"is_administrator": true,
"is_active": true
}
}
I've created serializers like so:
class ZonePermissionSerializer(serializers.ModelSerializer):
class Meta:
model = ZonePermission
fields = ('is_administrator', 'is_active')
class ZoneSerializer(serializers.HyperlinkedModelSerializer):
current_user_zone_permission = ZonePermissionSerializer(source='zonepermission_set')
class Meta:
model = Zone
fields = ('name', 'current_user_zone_permission')
The problem with this is that when I request a particular zone, the nested resource returns the ZonePermission records for all the users with permissions for that zone. Is there any way of applying a filter on request.user to the nested resource?
BTW I don't want to use a HyperlinkedIdentityField for this (to minimise http requests).
Solution
This is the solution I implemented based on the answer below. I added the following code to my serializer class:
current_user_zone_permission = serializers.SerializerMethodField('get_user_zone_permission')
def get_user_zone_permission(self, obj):
user = self.context['request'].user
zone_permission = ZonePermission.objects.get(zone=obj, user=user)
serializer = ZonePermissionSerializer(zone_permission)
return serializer.data
Thanks very much for the solution!
I'm faced with the same scenario. The best solution that I've found is to use a SerializerMethodField and have that method query and return the desired values. You can have access to request.user in that method through self.context['request'].user.
Still, this seems like a bit of a hack. I'm fairly new to DRF, so maybe someone with more experience can chime in.
You have to use filter instead of get, otherwise if multiple record return you will get Exception.
current_user_zone_permission = serializers.SerializerMethodField('get_user_zone_permission')
def get_user_zone_permission(self, obj):
user = self.context['request'].user
zone_permission = ZonePermission.objects.filter(zone=obj, user=user)
serializer = ZonePermissionSerializer(zone_permission,many=True)
return serializer.data
Now you can subclass the ListSerializer, using the method I described here: https://stackoverflow.com/a/28354281/3246023
You can subclass the ListSerializer and overwrite the to_representation method.
By default the to_representation method calls data.all() on the nested queryset. So you effectively need to make data = data.filter(**your_filters) before the method is called. Then you need to add your subclassed ListSerializer as the list_serializer_class on the meta of the nested serializer.
subclass ListSerializer, overwriting to_representation and then calling super
add subclassed ListSerializer as the meta list_serializer_class on the nested Serializer
If you're using the QuerySet / filter in multiple places, you could use a getter function on your model, and then even drop the 'source' kwarg for the Serializer / Field. DRF automatically calls functions/callables if it finds them when using it's get_attribute function.
class Zone(models.Model):
name = models.SlugField()
def current_user_zone_permission(self):
return ZonePermission.objects.get(zone=self, user=user)
I like this method because it keeps your API consistent under the hood with the api over HTTP.
class ZoneSerializer(serializers.HyperlinkedModelSerializer):
current_user_zone_permission = ZonePermissionSerializer()
class Meta:
model = Zone
fields = ('name', 'current_user_zone_permission')
Hopefully this helps some people!
Note: The names don't need to match, you can still use the source kwarg if you need/want to.
Edit: I just realised that the function on the model doesn't have access to the user or the request. So perhaps a custom model field / ListSerializer would be more suited to this task.
I would do it in one of two ways.
1) Either do it through prefetch in your view:
serializer = ZoneSerializer(Zone.objects.prefetch_related(
Prefetch('zone_permission_set',
queryset=ZonePermission.objects.filter(user=request.user),
to_attr='current_user_zone_permission'))
.get(id=pk))
2) Or do it though the .to_representation:
class ZoneSerializer(serializers.HyperlinkedModelSerializer):
class Meta:
model = Zone
fields = ('name',)
def to_representation(self, obj):
data = super(ZoneSerializer, self).to_representation(obj)
data['current_user_zone_permission'] = ZonePermissionSerializer(ZonePermission.objects.filter(zone=obj, user=self.context['request'].user)).data
return data