I'm using the admin search_fields functionality.
The problem: some of my tables are very big. So search is taking forever, and adding extra load on my production database.
As I'm having a follower of my production db, I though a good idea would be to use the follower as a read-only db, especially for those kind of requests.
So I decided to add a 'read-only' db in settings.DATABASES and surcharge ModelAdmin.get_search_results in my admin classes:
def get_search_results(self, request, queryset, search_term):
queryset, use_distinct = super(ReadOnlyDatabaseAdmin, self)\
.get_search_results(request, queryset, search_term)
queryset = queryset.using('read-only')
return queryset, use_distinct
After this update, I started to get some router errors when trying to set some object as foreign key related object of another object:
Cannot assign "...": the current database router prevents this relation
NB: the read-only database was the same as the default one when I tested and got the aforementioned error, I didn't use the follower yet. I just have set a 'read-only' key in settings.DATABASES, pointing to the same dict as DATABASES['default'].
So the problem is not coming from using a different database, but strictly from the database router.
To give more detail: this error is notably coming from admin actions that are performed when in a admin-search-results page (/admin/app/obj/?q=...).
I figured it's maybe because I replace the queryset object in the method. Maybe this object is actually re-used somewhere else notably in admin actions...? I am currently looking into this.
So I'm interested in:
finding the reason of the error
and/or finding another way of performing admin search requests on a follower database to offload the main database
I guess the answer to the error is to do instead:
if request.method == 'GET':
queryset = queryset.using('read-only')
Indeed, the search results are dont with a GET, while the admin actions are done with a POST.
I will have to check this
This is not exactly you are looking for How to improved query performance in Django admin search on related fields (MySQL), but it can help to optimize the queries.
Related
I am struggling to understand how permissioning in DRF is meant to work. Particularly when/why should a permission be used versus when the queryset should be filtered and the difference between has_object_permission() & has_permission() and finally, where does the serializer come in.
For example, with models:
class Patient(models.Model):
user = models.OneToOneField(User, related_name='patient')
class Appointment(models.Model):
patient = models.ForeignKey(Patient, related_name='appointment')
To ensure that patients can only see/change their own appointments, you might check in a permission:
class IsRelevantPatient(BasePermission):
def has_object_permission(self, request, view, obj):
if self.request.user.patient == obj.appointment.patient:
return True
else:
return False
But, modifying the queryset also makes sense:
class AppointmentViewSet(ModelViewSet):
...
def get_queryset(self):
if self.request.user.is_authenticated:
return Appointment.objects.filter(patient=self.request.user.patient)
What's confusing me is, why have both? Filtering the queryset does the job - a GET (retrieve and list) only returns that patient's appointments and, a POST or PATCH (create or update) only works for that patient's appointments.
Aside from this seemingly redundant permission - what is the difference between has_object_permission() & has_permission(), from my research, it sounds like has_permission() is for get:list and post:create whereas has_object_permission() is for get:retrieve and patch:update. But, I feel like that is probably an oversimplification.
Lastly - where does validation in the serializer come in? For example, rather than a permission to check if the user is allowed to patch:update an object, You can effectively check permissions by overriding the update() method of the serializer and checking there.
Apologies for the rambling post but I have read the docs and a few other question threads and am at the point where I am probably just confusing myself more. Would really appreciate a clear explanation.
Thanks very much.
First, difference between has_object_permission() and has_permission() :
has_permission() tells if the user has the permission to use the view or the viewset without dealing with any object in the database
has_object_permission() tells if the user has the permission to use the view or the viewset based on a specific object in the database.
The important note thaw is that DRF wont perform the test itself in the case of object level permission, but you have to do it explicitly by calling check_object_permission() somewhere in your view (doc here).
The second important note is that DRF will not filter the result of the query based on object permission. If you want the query to be filtered, then you have to do it yourself (by overriding get_queryset() like you did or using a filter backend), that's the difference.
The serializer has nothing to do with permission neither with filtering. It handles objects one by one, applying validation (not permission) on each field of each objects.
I am using Django oscar(2.0.2), where I have made foreign key relation of merchant ids with product table AbstractProduct with data, (https://prnt.sc/s1ssxx) so that I can fetch merchant-specific products from the database. By default oscar returns all the products in the search results, I want to only return the products specific to a merchant's site.
I am using Haystack simple search as suggested in oscar documentation, I have tried overriding the search app like all other apps, I have overridden the search_indexes.py file, but it seems that it never gets called from the FacetedSearchView. I also tried to override the search handlers, but it was also not getting called.
I tried understanding oscar's search functionality, but on the shell, I get a warning,
UserWarning: The model is not registered for search.
warnings.warn('The model %r is not registered for search.' % (model,))
Model '' not handled by the routers. Model class oscar_apps.catalogue.models.Product not handled by the routers.
How can I register the Product model for search?
where will I have to override the query like that:
Product.objects.filter(user_id=1), to return only merchant-specific products while searching for a product?
I know, how to override apps, but could someone give an overview and explain to me the steps that will be required to override the search app, and get the basic sorting functionality working?
if my question is not clear, let me know in the comments so I can improve it.
#yajant, I'm not sure whether my answer might be useful to you, but I have faced the same issue and struggled to find out the solution. Hope this might help someone facing the same issue. Thanks
fork the search app and import oscar search and overwrite FacetedSearchView class get_results method as below
Override the get_results method in your forked search/views.py
def get_results(self):
# We're only interested in products (there might be other contenttypes
# in the Solr index).
qs = super().get_results().models(Product)
qs = qs.filter(user_id=1)
return qs
I'm just start working with django and DRF, and occure a problem, that is looks like DRF cache responses. I mean - I can change object, create new, or delete it - and DRF keep response, thats nothing is changed. For example, I create an object, but modelViewSet still return data where this object does not presented. But if I directly request it object - it show that it's created. And so with any another actions. I can't find topic about caching in DRF, and look like I have not any django chaching middlewares, so I have no idea what is going on.
Only one thing that helps - restart server ( I'm using default dev-server).
One more thing - all data is ok when it's rendered by django views, not DRF views.
Here is one of the serializers/modelViewSets that I'm using. It simple as it possible. And also - I'm not using django cache backends. At least - I have not any in my settings.
class WorkOperationSerializer(serializers.ModelSerializer):
class Meta:
model = WorkOperation
class WorkOperationAPIView(viewsets.ModelViewSet):
serializer_class = WorkOperationSerializer
queryset = WorkOperation.objects.all()
def get_queryset(self):
return self.queryset
You can read here about django queryset caching. The best advice seems to be: re-run the .all() method to get fresh results. Using object.property may give you cached results.
(Django 1.8) I have a table which has 4 many-to-many relationship to other tables.
Two of these tables have so many entries and that is causing the admin page to load very slowly because it is trying to load all the entries in the lists.
Is there a way to avoid the internal admin page query for loading all the entries of the big tables to speed up the admin page load?
I think the best way is to only list the selected values but I'm not sure how.
I'm not sure how to use limit_choices_to in here:
class Data(models.Model):
pass # stuff here
class Report(models.Model):
data= models.ManyToManyField(Data)
I also tried adding this to my admin.py but it did not help at all. It's not limiting for some reason:
def queryset(self, request):
qs = super(MyModelAdmin, self).queryset(request)
if len(qs) > 10:
qs = qs[:10]
return qs
If you still want to use limit_choices_to, then please refer to the docs. You basically just supply the filters in a dictionary object.
To speed up the admin, my suggestions include:
1. Using raw_id_fields in your ModelAdmin. This gives you a little searchbox instead of a selectbox and avoids the overhead of listing all related objects.
2. If you are handling forward ForeignKey relationships, you can use list_select_related in your ModelAdmin as well. In your case, you are handling many-to-many relationships, so you can try overriding the get_queryset method of the ModelAdmin, and use prefetch_related like in the code below.
from django.contrib import admin
class TestModelAdmin(admin.ModelAdmin):
def get_queryset(self, request):
test_model_qs = super(TestModelAdmin, self).get_queryset(request)
test_model_qs = test_model_qs.prefetch_related('many-to-many-field')
return test_model_qs
If you really like to get your hands dirty, I highly recommend you use django-debug-toolbar. It really gives you visibility on how many and what SQL statements are being run. If you can read SQL, you can deduce what you need to input to select_related and prefetch_related.
I have a table 'Comments' and inside field the 'user', I would get in addition to the profile Profile in the same query. Currently I have something like that
comments = models.Comment.objects.select_related('author__profile').filter(post=article)
Unfortunately I can not retrieve information about profile, I can only get to it through
comment.author._profile_set_cache
Any ideas to make it look nice like?
comment.author.profile
If the 'author' is from the contrib.auth User model, then you don't have a FK to the UserProfile. It is a "reverse one-to-one". Fortunately, django is able to navigate a reverse one-to-one using "select_related", so the query is actually retrieving the fields (you can check it by using
print models.Comment.objects.select_related('author__profile').filter(post=article).query
The way to get the profile of a user is with the get_profile() method:
print comment.author.get_profile()
As the profile data is already cached (that's why the _profile_set_cache is for), getting the object means no additional query.