How to always prefetch_related for a specific django model - django

One of my models has number of related objects in it's __str__. This makes the admin site run very slow.
Is it possible to set up the model in a way that would always do prefetch_related, even if not explicitly requested?

You can implement a manager [Django-doc] that will automatically add a .prefetch_related(..) to the queryset.
For example:
class MyModelManager(models.Manager):
def get_queryset(self):
return super().get_queryset().prefetch_related('related_model')
class MyModel(models.Model):
# …
_base_manager = MyModelManager()
objects = MyModelManager()

Adding as an answer since I cannot add a comment (this answer):
The _base_manager attribute needs to be a class and not an object.
class MyModel(models.Model):
# …
_base_manager = MyModelManager
objects = MyModelManager()

Related

Prevent need for the same select_related clause on multiple views in DRF

Given the following models...
class Player(models.Model):
user = models.ForeignKey(User)
class Activity(models.Model):
player = models.ForeignKey(Player)
and these serializers...
class PlayerSerializer(serializers.ModelSerializer):
class Meta:
model = Player
fields = ['user']
class ActivitySerializer(serializers.ModelSerializer):
player = PlayerSerializer()
class Meta:
model = Activity
fields = ['player']
if I want to use django-rest-framework to list all the activities, I need to do something like this...
class ActivityViewSet(viewsets.ReadOnlyModelViewSet):
queryset = Activity.objects.select_related("player__user") <--- this is needed because the activity serializer is going to serialize the player which has a user
serializer_class = ActivitySerializer
That's all fine. But then, every time I write a new view which, at some level, uses PlayerSerializer, I'm going to have to remember to do the proper select_related clause in order to keep the number of DB lookups low.
I'm going to end up writing a lot of views which have the same select_related clauses (mine are actually a LOT more complicated than this example) and, if the PlayerSerializer ever changes, I'm going to need to remember to change all the view lookups.
This doesn't seem very DRY to me and I feel like there must be a better way to do it. Have I missed something obvious?
Maybe having a base parent class like:
class BaseViewset(viewsets.ReadOnlyModelViewSet):
SELECT_RELATED_FIELD = None # for providing select related field in your queryset or maybe have a default value for this one (based on your use case)
def get_queryset(self):
if not self.SELECT_RELATED_FIELD or not isinstance(self.SELECT_RELATED_FIELD, str):
raise NotImplementedError("Ensure 'SELECT_RELATED_FIELD' is not empty/ Is instance of string")
return super().get_queryset().select_related(self.SELECT_RELATED_FIELD)
And then use this parent class in all of your views instead of ReadOnlyModelViewSet:
class ActivityViewSet(BaseViewset):
SELECT_RELATED_FIELD = "player__user"
queryset = Activity.objects.all()
serializer_class = ActivitySerializer
would be a good start (in all views that you have select_related part).

"list_editable" doesn't work in admin changelist view, using custom get_queryset() method

I use custom abstract model with manager throughout my project.
class BaseQueryset(models.QuerySet):
pass
class BaseManager(models.Manager):
queryset_class = BaseQueryset
def get_queryset(self, exclude_no_published=True):
""" exclude all objects with is_published=False by default """
q = self.queryset_class(self.model)
if exclude_no_published:
q = q.exclude(is_published=False)
return q
def all_objects(self):
""" allows geting all objects in admin """
return self.get_queryset(exclude_no_published=False)
class BaseAbstractModel(models.Model):
is_published = models.BooleanField(default=True)
objects = BaseManager()
class Meta:
abstract = True
All models inherit from this abstract model and I need a way to represent all objects in admin. So I wrote my own mixin for admin classes with get_queryset method
class AdminFullQuerysetMixin(object):
def get_queryset(self, request):
"""
Allows showing all objects despite on is_public=False
"""
qs = self.model.objects.all_objects()
ordering = self.get_ordering(request)
if ordering:
qs = qs.order_by(*ordering)
return qs
There is my typical admin class:
#admin.register(SomeModel)
class SomeModelAdmin(AdminFullQuerysetMixin, admin.ModelAdmin):
list_display = ('name', 'slug', 'is_published')
list_filter = ('is_published',)
list_editable = ('is_published',)
All works fine, I can see all objects in admin whether with is_published False or True. But such attributes like list_filter or list_editable don't work, when I use it in admin objects list page. There is no exception provided, just text at the top of the list: "Please correct the error below".
What methods except get_queryset should I override for solving my problem?
You may want to read this :
if you use custom Manager objects, take note that the first Manager
Django encounters (in the order in which they’re defined in the model)
has a special status. Django interprets the first Manager defined in a
class as the “default” Manager, and several parts of Django (...) will
use that Manager exclusively for that model. As a result, it’s a
good idea to be careful in your choice of default manager in order to
avoid a situation where overriding get_queryset() results in an
inability to retrieve objects you’d like to work with.
I strongly suspect you fell upon one of those cases...
The solution would then be to change your model to:
class BaseAbstractModel(models.Model):
is_published = models.BooleanField(default=True)
# this one will be the default manager
all_objects = models.Manager()
# and this one will be known as 'objects'
objects = BaseManager()
class Meta:
abstract = True
Then you can remove your AdminFullQuerysetMixin (or rewrite it's get_queryset() method to use self.model._default_manager instead)
NB : I may of course be wrong and the problem be totally unrelated ;)

Using ModelSerializer with joined records

I am trying to make a tool for drawing diagrams on the web. I have a model like so:
class PlaneableItem(Model):
name = models.CharField(max_length=NAME_LENGTH, blank=True)
class View(PlaneableItem):
# Some useful details
class Anchor(Model):
view = models.ForeignKey(View)
planeable = models.ForeignKey(PlaneableItem)
class BlockRepresentation(Anchor):
# Useful details
class LineRepresentation(Anchor):
# Useful details
I try to make a rest API that returns lists of all blocks and lines for a specific view, including the name of the planeable that they refer to.
I can get a queryset for this using:
qs = BlockRepresentation.objects.filter(view=theview).all()
qs.select_related('planeable')
qs.extra(select={'name': 'rest_api_planeableitem.name'})
However, now I can't use a ModelSerializer on it, because the field 'name' is not part of the BlockRepresentation.
I really like ModelSerializers, is there a better way of doing this?
Is there a particular reason you need that extra() call? If the sole purpose of that call is to rename a field, you can omit that from the queryset and rename the field using a SerializerMethodField from your serializer. I will assume planeable is the ForeignKey field in BlockRepresentation model to the PlaneableItem model. Sample code:
from rest_framework import serializers
class BlockRepresentationSerializer(serializers.ModelSerializer):
# Some fields
name = serializers.SerializerMethodField()
class Meta:
model = BlockRepresentation
def get_name(self, obj):
if obj.planeable:
return obj.planeable.name
return ''

djangorestframework: Filtering in a related field

Basically, I want to filter out inactive users from a related field of a ModelSerializer. I tried Dynamically limiting queryset of related field as well as the following:
class MySerializer(serializers.ModelSerializer):
users = serializers.PrimaryKeyRelatedField(queryset=User.objects.filter(active=True), many=True)
class Meta:
model = MyModel
fields = ('users',)
Neither of these approaches worked for just filtering the queryset. I want to do this for a nested related Serializer class as a field (but couldn't even get it to work with a RelatedField).
How do I filter queryset for nested relation?
I'll be curious to see a better solution as well. I've used a custom method in my serializer to do that. It's a bit more verbose but at least it's explicit.
Some pseudo code where a GarageSerializer would filter the nested relation of cars:
class MyGarageSerializer(...):
users = serializers.SerializerMethodField('get_cars')
def get_cars(self, garage):
cars_queryset = Car.objects.all().filter(Q(garage=garage) | ...).select_related()
serializer = CarSerializer(instance=cars_queryset, many=True, context=self.context)
return serializer.data
Obviously replace the queryset with whatever you want. You don't always need the to give the context (I used it to retrieve some query parameters in the nested serializer) and you probably don't need the .select_related (that was an optimisation).
One way to do this is to create a method on the Model itself and reference it in the serializer:
#Models.py
class MyModel(models.Model):
#...
def my_filtered_field (self):
return self.othermodel_set.filter(field_a = 'value_a').order_by('field_b')[:10]
#Serialziers.py
class MyModelSerialzer(serializers.ModelSerializer):
my_filtered_field = OtherModelSerializer (many=True, read_only=True)
class Meta:
model = MyModel
fields = [
'my_filtered_field' ,
#Other fields ...
]
Another way to avoid the SerializerMethodField solution and therefore still allow writing to the serializer as well would be to subclass the RelatedField and do the filtering there.
To only allow active users as values for the field, the example would look like:
class ActiveUsersPrimaryKeyField(serializers.PrimaryKeyRelatedField):
def get_queryset(self):
return super().get_queryset().filter(active=True)
class MySerializer(serializers.ModelSerializer):
users = ActiveUsersPrimaryKeyField(many=True)
class Meta:
model = MyModel
fields = ('users',)
Also see this response.
Note that this only restricts the set of input values to active users, though, i.e. only when creating or updating model instances, inactive users will be disallowed.
If you also use your serializer for reading and MyModel already has a relation to a user that has become inactive in the meantime, it will still be serialized. To prevent this, one way is to filter the relation using django's Prefetch objects. Basically, you'll filter out inactive users before they even get into the serializer:
from django.db.models import Prefetch
# Fetch a model instance, eagerly prefetching only those users that are active
model_with_active_users = MyModel.objects.prefetch_related(
Prefetch("users", queryset=User.objects.filter(active=True))
).first()
# serialize the data with the serializer defined above and see that only active users are returned
data = MyModelSerializer(model_with_active_users).data

Override default queryset in Django admin

One of my models has a deleted flag, which is used to hide objects globally:
class NondeletedManager(models.Manager):
"""Returns only objects which haven't been deleted"""
def get_query_set(self):
return super(NondeletedManager, self).get_query_set().exclude(deleted=True)
class Conversation(BaseModel):
...
deleted = models.BooleanField(default=False)
objects = NondeletedManager()
all_conversations = models.Manager() # includes deleted conversations
How can I override the default queryset used by Django admin module to include deleted conversations?
You can override get_queryset method in your model admin class.
class MyModelAdmin(admin.ModelAdmin):
def get_queryset(self, request):
qs = super().get_queryset(request)
if request.user.is_superuser:
return qs
return qs.filter(author=request.user)
Note in Django<=1.5 the method was named just queryset.
Konrad is correct, but this is more difficult than the example given in the documentation.
Deleted conversations can't be included in a queryset that already excludes them. So I don't see an option other than re-implementing admin.ModelAdmin.queryset entirely.
class ConversationAdmin (admin.ModelAdmin):
def queryset (self, request):
qs = Conversation.all_conversations
ordering = self.get_ordering(request)
if ordering:
qs = qs.order_by(*ordering)
return qs
You can do this with a Django proxy model.
# models.py
class UnfilteredConversation(Conversation):
class Meta:
proxy = True
# this will be the 'default manager' used in the Admin, and elsewhere
objects = models.Manager()
# admin.py
#admin.register(UnfilteredConversation)
class UnfilteredConversationAdmin(Conversation):
# regular ModelAdmin stuff here
...
Or, if you have an existing ModelAdmin class you want to re-use:
admin.site.register(UnfilteredConversation, ConversationAdmin)
This approach avoids issues that can arise with overriding the default manager on the original Conversation model - because the default manager is also used in ManyToMany relationships and reverse ForeignKey relationships.
What would be so wrong with the following:
class Conversation(BaseModel):
...
deleted = models.BooleanField(default=False)
objects = models.Manager() # includes deleted conversations
nondeleted_conversations = NondeletedManager()
So in your own apps/projects, you use Conversation.nondeleted_conversations() and let the built-in admin app do it's thing.
Natan Yellin is correct, but you can change the managers order and the first will be the default, then it is the used by the admin:
class Conversation(BaseModel):
...
deleted = models.BooleanField(default=False)
all_conversations = models.Manager() # includes deleted conversations
objects = NondeletedManager()
The admin implementation of get_queryset() use ._default_manager instead .objects, as show next
qs = self.model._default_manager.get_queryset()
ref Django github BaseModelAdmin implementation
This only ensures that every time you use YourModel.objects, you will not include deleted objects, but the generic views and others uses ._default_manager too. Then if you don't override get_queryset is not a solution. I've just check on a ListView and admin.
The accepted solution works great for me but I needed a little bit more flexibility, so I ended up extending the changelist view to add in a custom queryset parameter. I can now configure my default queryset/filter as such and it can still be modified by using a different filter (get parameters):
def changelist_view(self, request, extra_context=None):
if len(request.GET) == 0 :
q = request.GET.copy()
q['status__gt'] = 4
request.GET = q
request.META['QUERY_STRING'] = request.GET.urlencode()
return super(WorksheetAdmin,self).changelist_view(request, extra_context=extra_context)
To extend on some of these answers with what I found most concise and useful.
I've made the assumption you have a field like "name" to show the entries.
# admin.py
from django.contrib import admin
#admin.register(Conversation)
class ConversationAdmin(admin.ModelAdmin):
list_display = ('name', '_is_deleted')
# Nice to have but indicates that an object is deleted
#admin.display(
boolean=True,
ordering='deleted'
)
def _is_deleted(self, obj):
return obj.deleted
def get_queryset(self, request):
return Conversation.all_conversations
Which will give you an interface like:
The problem I found with subclassing a model was that it caused issues with meta inheritance and reverse-path lookups.