I have two models, one (Device) which has a foreign key to the other (Storage). In the admin overview of Storage I wanna be able to add a count of how many Device has pointed to each Storage. Using the code below I'm able to correctly count the number of Devices pointing to each Storage. However, when trying to sort by the number in storage it crashes with the following error:
Cannot resolve keyword '_get_device_number' into field. Choices are:
... #List of fields
How do I add my calculated field into the list allowed searches?
class Device(models.Model):
...
storage_id = models.ForeignKey(Storage, on_delete=models.PROTECT, blank=True,null=True) #Allowed to be null since a Device may not be in storage.
class Storage(models.Model):
...
def _get_device_number(self,):
return Device.objects.filter(storage_id=self.storage_id).count()
class StorageAdmin(admin.ModelAdmin):
...
list_display = ['__str__', 'get_device_number',]
def get_device_number(self, obj):
return obj._get_device_number()
get_device_number.admin_order_field = '_get_device_number'
The admin list view cannot access a model method. However, annotating the admin queryset does work.
class StorageAdmin(admin.ModelAdmin):
def get_queryset(self, request):
queryset = super().get_queryset(request)
queryset = queryset.annotate(
_get_device_number=Count('device')
)
return queryset
def get_device_number(self, obj):
return obj._get_device_number
get_device_number.admin_order_field = '_get_device_number'
Related
I have 2 models is Product and Watch. A Product may have multiple Wacht.
When I call the API to get the list of products. I want to provide a feature for ordering the products by a number of watches that each product has like below.
domainname/products/?ordering=numberOfWatch
Here is my model
class Product(models.Model):
# product model fields
and Wacht model
class Watch(models.Model):
product = models.ForeignKey(
Product,
related_name='watches',
on_delete=models.CASCADE,
)
# other fields
and the ProductList View
class ProductList(generics.ListCreateAPIView):
queryset = Product.objects.all()
permission_classes = (IsAuthenticated, IsAdmin,)
name = 'product-list'
filter_fields = ('category', 'brand', 'seller')
search_fields = ('name',)
ordering_fields = ('-updated_at', 'price', 'discount_rate', 'discount')
ordering = ('-updated_at',)
I'm thinking of add watch_count field in Product model and ordering for that field. But is that a good way to get what I need?
General way:
from django.db.models import Count
Product.objects.all().annotate(watch_count=Count("watches")).order_by("-watches")
the direction of the ordering can be changed by removing - from order_by
In order to implement this in DRF (with Django-Filter), you have to use custom ordering filter. Simple example:
class CustomOrderingFilter(filters.OrderingFilter):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.extra['choices'] += [
('watch_count', 'Watch count low to high'),
('-watch_count', 'Watch count high to low'),
]
def filter(self, qs, value):
if value and any(v in ['watch_count', '-watch_count'] for v in value):
qs = Product.objects.all().annotate(watch_count=Count("watches"))
if 'watch_count' in value:
qs = qs.order_by("watches")
elif '-watch_count' in value:
qs = qs.order_by("-watches")
return qs
return super().filter(qs, value)
ordering = CustomOrderingFilter(
fields=(
('updated_at', 'updated_at'), # field value, url param value
),
)
You can read more about this here:
https://django-filter.readthedocs.io/en/stable/ref/filters.html?highlight=OrderingFilter#orderingfilter
I'm thinking of add watch_count field in Product model and ordering
for that field. But is that a good way to get what I need?
It depends. Solution above will work well for most cases, but for very large datasets it is possible that you will encounter some performance issues. For huge amounts of data, solution provided by yourself will be better. It is also very possible that the incoming django release (3.2) will help with fixing performance issues by enabling creation of functional indexes: https://docs.djangoproject.com/en/3.2/releases/3.2/#functional-indexes
The generic structure of the models is that there are teachers and devices, each device has a ForeignKey relationship with the teachers ID/PK.
I'm trying to create my API in such a way that when going to the detail view for a teacher, all of the associated devices are displayed. I've overridden get_serializer_class() to specify which serializer to use at the appropriate time, but can't figure out how to correctly change the Queryset based on detail view or not. Error posted below.
Got AttributeError when attempting to get a value for field `brand` on serializer `DeviceSerializer`.
The serializer field might be named incorrectly and not match any attribute or key on the `Teacher` instance.
Original exception text was: 'Teacher' object has no attribute 'brand'.
class TeacherViewSet(viewsets.ModelViewSet):
queryset = Teacher.objects.order_by('campus','name')
serializer_class = TeacherSerializer
detail_serializer_class = DeviceSerializer
def get_serializer_class(self):
if self.action == 'retrieve':
if hasattr(self, 'detail_serializer_class'):
return self.detail_serializer_class
return super(TeacherViewSet, self).get_serializer_class()
def get_queryset(self, pk=None):
if pk is not None:
return Device.objects.filter(device__owner=self.kwargs.get('pk')
return Teacher.objects.all()
I was able to get the desired output by adding a nested DeviceSerializer in my TeacherSerializer that parses the device object list.
class TeacherSerializer(serializers.ModelSerializer):
devices = DeviceSerializer(many=True)
class Meta:
model = Teacher
fields = ('id', 'name', 'campus', 'email', 'devices')
I assume you are using DRF. If that is the case, just tweak TeacherSerializer to something like:
def TeachSearializer(serializer.ModelSerializer):
devices = serializers.SerializerMethodField()
class Meta:
model = Teacher
fields = '__all__'
def get_devices(self, obj):
return Devices.objects.filter(teacher=obj)
And that is it, everytime you use the serializer on a teacher object, their devices will be added on a field devices
I have a very simple ( with a first look) problem. Case - A product can be sold in a several places(shops), and every product can be represented in a single shop with a different categories and sub categories ( That is why category linked via ForeignKey with Assortment twice).
So here is My Assortment model, with several FKs.
class Assortment(models.Model):
category = models.ForeignKey('category.Category', null=True, blank=True, default=None,related_name='assortment_child')
parent_category = models.ForeignKey('category.Category', null=True, blank=True, default=None,related_name='assortment_parent')
product = models.ForeignKey(Product)
shop = models.ForeignKey(Shop)
View, based on rest_framework.generics.ListAPIView
class InstitutionTreeCategories(generics.ListAPIView):
"""Resource to get shop's tree of categories."""
serializer_class = serializers.InstitutionCategoriesSerializer
def get_queryset(self):
shop = self.get_shop()
return Category.objects.filter(assortment_parent__shop=shop).distinct()
And finally, serializers
class CategoryListSerializer(serializers.ModelSerializer):
class Meta:
"""Meta class."""
model = Category
fields = ('id', 'name', 'image')
class CategoriesTreeSerializer(CategoryListSerializer):
# childs = CategoryListSerializer(many=True, source='assortment_child__parent_category')
childs = serializers.SerializerMethodField()
class Meta(CategoryListSerializer.Meta):
"""Meta class."""
fields = ('id', 'name', 'image', 'childs')
def get_childs(self, obj):
qs = Category.objects.filter(assortment_child__parent_category=obj.id).distinct()
return CategoryListSerializer(qs, many=True, context=self.context).data
And i need to show Category Tree for a one single shop with my API.
But the problem is - If I use serializer.SerializerMethodField - it works, but too many queries (for every parent category). I tried to avoid it using 'source' option with my 'CategoryListSerializer' by I can't make it. Every time, I get - 'Category' object has no attribute assortment_child__parent_category. In a shell model i've tried
In [8]: cat.assortment_parent.values('category').distinct()
Out[8]: (0.003) SELECT DISTINCT "marketplace_assortment"."category_id" FROM "marketplace_assortment" WHERE "marketplace_assortment"."parent_category_id" = 4 LIMIT 21; args=(4,)
<AssortmentQuerySet [{'category': 3}]>
So - category object has this attributes, of course it does, i used it a get_childs method. So question is - how i can use it with serializer.ModelSerializer and it's source option? ( Of course using select_related method with queryset, to avoid excess queries).
by source option you should use . in instead of __:
childs = CategoryListSerializer(many=True, source='assortment_child.parent_category')
but still you will has many queries, to fix it you should use prefetch-related
def get_queryset(self):
shop = self.get_shop()
qs = Category.objects.filter(assortment_parent__shop=shop).all()
return qs.prefetch_related('assortment_child').distinct()
more detail you can read in the how-can-i-optimize-queries-django-rest-framework
I had the similar problem and the best solution I have found is to do some manual processing in order to receive desired tree representation.
So firstly we fetch all Assortment for shop and then build the tree manually.
Let's look at the example.
def get_categories_tree(assortments, context):
assortments = assortments.select_related('category', 'parent_category')
parent_categories_dict = OrderedDict()
for assortment in assortments:
parent = assortment.parent_category
# Each parent category will appear in parent_categories_dict only once
# and it will accumulate list of child categories
if parent not in parent_categories_dict:
parent_data = CategoryListSerializer(instance=parent, context=context).data
parent_categories_dict[parent] = parent_data
parent_categories_dict[parent]['childs'] = []
child = assortment.category
child_data = CategoryListSerializer(instance=child, context=context).data
parent_categories_dict[parent]['childs'].append(child_data)
# convert to list as we don't need the keys already - they were used only for matching
parent_categories_list = list(parent_categories_dict.values())
return parent_categories_list
class InstitutionTreeCategories(generics.ListAPIView):
def list(self, request, *args, **kwargs):
shop = self.get_shop()
assortments = Assortment.objects.filter(shop=shop)
context = self.get_serializer_context()
categories_tree = get_categories_tree(assortments, context)
return Response(categories_tree)
All in single DB query.
The problem here is that there is no explicit relation between category and parent_category. If you define ManyToManyField in Category using Assortment as through intermediate model, you will get an access which Django can understand, so you would just use attribute childs on Category for example. However this will still return all children (the same would happen if your source example works) categories, ignoring shop, so some clever Prefetch would have to be done to achieve correct results. But I believe manual "join" is simpler.
you need to use prefetch_related along with serializer method field
serializer:
class CategoriesTreeSerializer(CategoryListSerializer):
children = serializers.SerializerMethodField()
class Meta(CategoryListSerializer.Meta):
fields = (
'id',
'name',
'image',
'children'
)
def get_children(self, obj):
children = set()
for assortment in obj.assortment_parent.all():
children.add(assortment.category)
serializer = CategoryListSerializer(list(children), many=True)
return serializer.data
your get queryset method:
def get_queryset(self):
shop = self.get_shop()
return (Category.objects.filter(assortment_parent__shop=shop)
.prefetch_related(Prefetch('assortment_parent', queryset=Assortment.objects.all().select_related('category')
.distinct())
My Django-powered app with a DRF API is working fine, but I've started to run into performance issues as the database gets populated with actual data. I've done some profiling with Django Debug Toolbar and found that many of my endpoints issue tens to hundreds of queries in the course of returning their data.
I expected this, since I hadn't previously optimized anything with regard to database queries. Now that I'm setting up prefetching, however, I'm having trouble making use of properly prefetched serializer data when that serializer is nested in a different serializer. I've been using this awesome post as a guide for how to think about the different ways to prefetch.
Currently, my ReadingGroup serializer does prefetch properly when I hit the /api/readinggroups/ endpoint. My issue is the /api/userbookstats/ endpoint, which returns all UserBookStats objects. The related serializer, UserBookStatsSerializer, has a nested ReadingGroupSerializer.
The models, serializers, and viewsets are as follows:
models.py
class ReadingGroup(models.model):
owner = models.ForeignKeyField(settings.AUTH_USER_MODEL)
users = models.ManyToManyField(settings.AUTH_USER_MODEL)
book_type = models.ForeignKeyField(BookType)
....
<other group related fields>
def __str__(self):
return '%s group: %s' % (self.name, self.book_type)
class UserBookStats(models.Model):
reading_group = models.ForeignKey(ReadingGroup)
user = models.ForeignKey(settings.AUTH_USER_MODEL)
alias = models.CharField()
total_books_read = models.IntegerField(default=0)
num_books_owned = models.IntegerField(default=0)
fastest_read_time = models.IntegerField(default=0)
average_read_time = models.IntegerField(default=0)
serializers.py
class ReadingGroupSerializer(serializers.ModelSerializer):
users = UserSerializer(many = True,read_only=True)
owner = UserSerializer(read_only=True)
class Meta:
model = ReadingGroup
fields = ('url', 'id','owner', 'users')
#staticmethod
def setup_eager_loading(queryset):
#select_related for 'to-one' relationships
queryset = queryset.select_related('owner')
#prefetch_related for 'to-many' relationships
queryset = queryset.prefetch_related('users')
return queryset
class UserBookStatsSerializer(serializers.HyperlinkedModelSerializer):
reading_group = ReadingGroupSerializer()
user = UserSerializer()
awards = AwardSerializer(source='award_set', many=True)
class Meta:
model = UserBookStats
fields = ('url', 'id', 'alias', 'total_books_read', 'num_books_owned',
'average_read_time', 'fastest_read_time', 'awards')
#staticmethod
def setup_eager_loading(queryset):
#select_related for 'to-one' relationships
queryset = queryset.select_related('user')
#prefetch_related for 'to-many' relationships
queryset = queryset.prefetch_related('awards_set')
#setup prefetching for nested serializers
groups = Prefetch('reading_group', queryset ReadingGroup.objects.prefetch_related('userbookstats_set'))
queryset = queryset.prefetch_related(groups)
return queryset
views.py
class ReadingGroupViewset(views.ModelViewset):
def get_queryset(self):
qs = ReadingGroup.objects.all()
qs = self.get_serializer_class().setup_eager_loading(qs)
return qs
class UserBookStatsViewset(views.ModelViewset):
def get_queryset(self):
qs = UserBookStats.objects.all()
qs = self.get_serializer_class().setup_eager_loading(qs)
return qs
I've optimized the prefetching for the ReadingGroup endpoint (I actually posted about eliminating duplicate queries for that endpoint here), and now I'm working on the UserBookStats endpoint.
The issue I'm having is that, with my current setup_eager_loading in the UserBookStatsSerializer, it doesn't appear to use the prefetching set up by the eager loading method in the ReadingGroupSerializer. I'm still a little hazy on the syntax for the Prefetch object - I was inspired by this excellent answer to try that approach.
Obviously the get_queryset method of UserBookStatsViewset doesn't call setup_eager_loading for the ReadingGroup objects, but I'm sure there's a way to accomplish the same prefetching.
prefetch_related() supports prefetching inner relations by using double underscore syntax:
queryset = queryset.prefetch_related('reading_group', 'reading_group__users', 'reading_group__owner')
I don't think Django REST provides any elegant solutions out of the box for fetching all necessary fields automatically.
An alternative to prefetching all nested relationships manually, there is also a package called django-auto-prefetching which will automatically traverse related fields on your model and serializer to find all the models which need to be mentioned in prefetch_related and select_related calls. All you need to do is add in the AutoPrefetchViewSetMixin to your ViewSets:
from django_auto_prefetching import AutoPrefetchViewSetMixin
class ReadingGroupViewset(AutoPrefetchViewSetMixin, views.ModelViewset):
def get_queryset(self):
qs = ReadingGroup.objects.all()
return qs
class UserBookStatsViewset(AutoPrefetchViewSetMixin, views.ModelViewset):
def get_queryset(self):
qs = UserBookStats.objects.all()
return qs
Any extra prefetches with more complex Prefetch objects can be added in the get_queryset method on the ViewSet.
I have a Requests Model which has a one-to-many relation to its RequestDetails. I also have a RequestFilter that has a one-to-one relation to the auth.user. What i'm trying to do is to display all Requests in a generic ListView, which have at least one RequestDetail that has a category enabled by the RequestFilter settings. I tried to accomplish this in different ways but still end up with a "Relation fields do not support nested lookups" Error. Here is what I did:
Given Models:
A User specifies the Requests (not django user.request) he wants to receive in a settings menu based on the following RequestFilter Model:
class RequestFilter(models.Model):
user = models.OneToOneField(User)
category1_enabled = models.BooleanField(...)
category2_enabled = models.BooleanField(...)
category3_enabled = models.BooleanField(...)
...
def get_categories_enabled(self):
returns a list of category names which are set to True e.g. ['category1', 'category3']
The Requests themselves contain basic information and a reference code:
class Requests(models.Model):
request_id = models.IntegerField(unique=True, ...)
reference = models.CharField(max_length=10)
some_general_info = models.CharField(...)
...
One Request can have many RequestDetails (like Orders which have many Products)
class RequestDetail(models.Model):
request = models.ForeignKey(Requests, to_field='request_id', related_name='details')
product_id = models.IntegerField()
category_id = models.IntegerField()
...
def get_category_name:
returns the name of the category of the RequestDetail e.g. 'category3'
I have a generic class based ListView which should display all Requests that contain at least one RequestDetail that is of a category which the User has set to enabled in the settings (RequestFilter).
views.py
class DashManagerRequestsView(generic.ListView):
template_name = 'dashboard/dashmanagerrequests.html'
context_object_name = 'request_list'
def get_categories_enabled(self):
return self.request.user.requestfilter.get_categories_enabled()
def get_queryset(self):
"""Returns all requests as an ordered list."""
requestfilter_list = self.get_categories_enabled()
return Requests.objects.order_by('-date_add').filter(details__get_category_name__in=requestfilter_list)
#method_decorator(login_required)
def dispatch(self, *args, **kwargs):
return super(DashManagerRequestsView, self).dispatch(*args, **kwargs)
I also tried using an F() expression but this doesn't work because it only compares the values of two different fields on the SAME model instance. The closest I got was to get a list of the enabled categories for the user and lookup if the category name is a part of this list with:
return Requests.objects.order_by('-date_add').filter(details__get_category_name__in=requestfilter_list)
but this raises a Relation fields do not support nested lookups Error.
thanks to schillingt, who pushed me into the right direction i came up with a solution.
since it is not possible to use model methods for filtering a QuerySet i generate a list of allowed ids during execution outside the filter function by relating to the RequestDetail model method using a second for loop. of course this can also be done using a list comprehension or something like that:
def get_queryset(self):
queryset = Requests.objects.order_by('-ps_date_add')
request_ids = []
for request in queryset:
for detail in request.details.all():
if detail.get_category_name() in self.get_categories_enabled():
request_ids.append(request.id)
q = q.filter(id__in=request_ids)
return q
this may not be the best solution in terms of efficiency if it comes to large amounts of data.