(Django 1.8) I have a table which has 4 many-to-many relationship to other tables.
Two of these tables have so many entries and that is causing the admin page to load very slowly because it is trying to load all the entries in the lists.
Is there a way to avoid the internal admin page query for loading all the entries of the big tables to speed up the admin page load?
I think the best way is to only list the selected values but I'm not sure how.
I'm not sure how to use limit_choices_to in here:
class Data(models.Model):
pass # stuff here
class Report(models.Model):
data= models.ManyToManyField(Data)
I also tried adding this to my admin.py but it did not help at all. It's not limiting for some reason:
def queryset(self, request):
qs = super(MyModelAdmin, self).queryset(request)
if len(qs) > 10:
qs = qs[:10]
return qs
If you still want to use limit_choices_to, then please refer to the docs. You basically just supply the filters in a dictionary object.
To speed up the admin, my suggestions include:
1. Using raw_id_fields in your ModelAdmin. This gives you a little searchbox instead of a selectbox and avoids the overhead of listing all related objects.
2. If you are handling forward ForeignKey relationships, you can use list_select_related in your ModelAdmin as well. In your case, you are handling many-to-many relationships, so you can try overriding the get_queryset method of the ModelAdmin, and use prefetch_related like in the code below.
from django.contrib import admin
class TestModelAdmin(admin.ModelAdmin):
def get_queryset(self, request):
test_model_qs = super(TestModelAdmin, self).get_queryset(request)
test_model_qs = test_model_qs.prefetch_related('many-to-many-field')
return test_model_qs
If you really like to get your hands dirty, I highly recommend you use django-debug-toolbar. It really gives you visibility on how many and what SQL statements are being run. If you can read SQL, you can deduce what you need to input to select_related and prefetch_related.
Related
In my django website, I have 3 classes: Thing, Category and SubCategory.
Thing has 2 ForeignKeys: "Category" and "SubCategory" (such as Car and Ferrari).
SubCategory has 1 ForeighKey: "Category" (Ferrari is in the category Car)
When I create an instance of Thing in the Admin part and when I choose a Category, I would like that the "SubCategory" field only shows the SubCategories linked to the Category I chose. Is that possible?
I saw the possibility to change the AdminForm like:
class ThingFormAdmin(forms.ModelForm):
def __init__(self,Category,*args,**kwargs):
super (ThingFormAdmin,self ).__init__(*args,**kwargs) # populates the post
self.fields['sub_category'].queryset = SubCategory.objects.filter(category= ... )
But I don't know what to write on the ...
Thanks for the help!
in general always this solution would work:
you need some javascript to catch what has been selected for first selection. then do filtering agin using javascript.
but in django admin, there is autocomplete_fields available. using this would create a kind of selection-input that uses ajax to do some magic filtering on choices when user types some characters. it uses the get_search_results method of the related models admin.ModelAdmin class. overriding that method and giving some extra data to that method could help. but it's the longest way to walk.
Thanks! I will look in the 1st answer after the 2nd, because the fact that I don't have to write JS is very nice, as I am very bad in it.
I manage to use the autocomplete_field to work, but I am stuck with the redefinition of the get_search_results method. If I understood correctly the doc, it will be something like:
def get_search_results(self, request, queryset, search_term):
queryset, use_distinct = super().get_search_results(request, queryset, search_term)
try:
cat = search_term
except ValueError:
queryset |= self.model.objects.all()
else:
queryset |= self.model.objects.filter(category=cat)
return queryset, use_distinct
But I don't understand from where this search_term comes from, and how I can specify it. Any ideas?
https://simpleisbetterthancomplex.com/tutorial/2018/01/29/how-to-implement-dependent-or-chained-dropdown-list-with-django.html
This tutorial will walk you through every step of doing whatever I presume you need to do.
I have 2 models, company and Product.
class Product(Meta):
company = models.ForeignKey(Company, related_name='products', on_delete=models.CASCADE)
form the database I'm trying to get the Company data and the corresponding products.
From the products I want to get only the name and to be ordered descending by updated_at, created_at.
I'm working with Prefetch object and prefetch_related and definitively I have multiple misunderstandings how they work.
def get_queryset(self):
qs = Company.objects.prefetch_related(
Prefetch('products', queryset=Product.objects.only('name').order_by('-updated_at', '-created_at'))).get()
return qs
The error that I receive is:
get() returned more than one Company
Because I closed the prefetch_related method/function with ))) :
I thought get() will act over the Company object and get it using the pk/slug from the url(as get do by default in DetailView). Seems that is not the case.
I'm already using 'products' the related name in the Prefetch object, why in queryset is necessary to tell again the model queryset=Product.objects.... ?
I was looking at the following example in django documentation:
Question.objects.prefetch_related(Prefetch('choice_set')).get().choice_set.all()
If there is 'choice_set' in Prefetch object why is called at the end choice_set.all() ?
Isn't Django attached to the quesryset in prefetch_related the products to the queryset (question.choice_set) ?
I think my problem is that I don't understand the order of execution, and I'm confused how methods are chained, even if are closed by ')'
queryset.get() will only work if the queryset has a single object. If it contains zero or more than one objects, you'll get an error.
You should return a queryset from the get_queryset object. In the class based view, the code that filters on the pk/slug is in get_object.
The prefetch_related method is useful if you want to fetch the products for multiple countries. The way that the Django docs use get() is confusing in my opinion - if the queryset has a single item then prefetch_related is over-complicated.
If you have a single company, then there is no advantage, and the code will be simpler if you fetch the countries separately, e.g. in get_context_data.
def get_context_data(self, **kwargs):
context = super(MyView, self).get_context_data(**kwargs)
context['products'] = Product.objects.filter(company=self.object).Product.objects.only('name').order_by('-updated_at', '-created_at')))
return context
I've removed the only('name') call. It's an optimisation that you probably don't need.
If you really want to use prefetch_related, then remove the get().
qs = Company.objects.prefetch_related(
Prefetch('products', queryset=Product.objects.order_by('-updated_at', '-created_at')))
By specifying the queryset above, you are able to change the order (you could filter it if you like). If you don't want to customize the queryset you can simply do:
Company.objects.prefetch_related('products')
When you use Question.objects.prefetch_related(...), the queryset is still a list of questions. You need to call choice_set.all() on the individual instances to access their choices. This won't cause any additional queries, because Django has already prefetched the choices.
queryset = Question.objects.prefetch_related(Prefetch('choice_set'))
for question in queryset:
print(question) # the question
print(question.choice_set.all()) # the related choices
One of my models is particularily complex. When I try to edit it in Django Admin it performs 1042 queries and takes over 9 seconds to process.
I know I can replace a few of the drop-downs with raw_id_fields, but I think the bigger bottleneck is that it's not performing a select_related() as it should.
Can I get the admin site to do this?
you can try this
class Foo(admin.ModelAdmin):
list_select_related = (
'foreign_key1',
'foreign_key2',
)
https://docs.djangoproject.com/en/dev/ref/contrib/admin/#django.contrib.admin.ModelAdmin.list_select_related
Although dr jimbob's answer makes sense, for my needs, I was able to simply override the get_queryset() method with a one-liner, even selecting a foreign key's foreign key. Maybe this could be helpful to someone.
class MyModelAdmin(admin.ModelAdmin):
model = MyModel
...
def get_queryset(self, request):
return super(MyModelAdmin, self).get_queryset(request).select_related(
'foreign_key1', 'foreign_key2__fk2_foreign_key')
For my particular model, the particularly slow aspect is going through ForeignKeys when they were being displayed in forms, which aren't called using select_related, so that's the part I'm going to speed up.
Looking through the relevant django source, you see in django/contrib/admin/options.py that the method formfield_for_foreignkeys takes each FK db_field and calls the ForeignKey class's formfield method, which is defined in django/db/models/fields/related/ like:
def formfield(self, **kwargs):
db = kwargs.pop('using', None)
defaults = {
'form_class': forms.ModelChoiceField,
'queryset': self.rel.to._default_manager.using(db).complex_filter(self.rel.limit_choices_to),
'to_field_name': self.rel.field_name,
}
defaults.update(kwargs)
return super(ForeignKey, self).formfield(**defaults)
From this, we see if we provide the db_field with a kwargs['queryset'] we can define a custom queryset that will be use select_related (this can be provided by formfield_for_foreignkey).
So basically what we want to do is override admin.ModelAdmin with SelectRelatedModelAdmin and then make our ModelAdmin subclasses of SelectRelatedModelAdmin instead of admin.ModelAdmin
class SelectRelatedModelAdmin(admin.ModelAdmin):
def formfield_for_foreignkey(self, db_field, request, **kwargs):
if 'queryset' in kwargs:
kwargs['queryset'] = kwargs['queryset'].select_related()
else:
db = kwargs.pop('using', None)
kwargs['queryset'] = db_field.rel.to._default_manager.using(db).complex_filter(db_field.rel.limit_choices_to).select_related()
return super(SelectRelatedModelAdmin, self).formfield_for_foreignkey(db_field, request, **kwargs)
This code sample doesn't cover admin Inlines or ManyToManyFields, or foreign_key traversal in functions called by readonly_fields or custom select_related queries, but a similar approach should work for those cases.
In Django 2.0+, a good way to improve performance of ForeignKey and ManyToMany relationships is to use autocomplete fields.
These fields don't show all related objects and therefore load with many fewer queries.
For the admin edit/change a specific item page, foreign key select boxes may take a long time to load, to alter the way django queries the data for the foreign key:
Django docs on Using formfield_for_foreignkey
Say I have a field called foo on my Example model, and I wish to select ralated bar objects:
class ExampleAdmin(admin.ModelAdmin):
def formfield_for_foreignkey(self, db_field, request, **kwargs):
if db_field.name == "foo":
kwargs["queryset"] = Example.objects.select_related('bar')
return super().formfield_for_foreignkey(db_field, request, **kwargs)
For the sake of completeness, I would like to add another option that was the most suitable for my use case.
As others have pointed out, the problem is often loading the data for select boxes. list_select_related does not help in this case.
In case you don't actually want to edit the foreign key field via admin, the easiest fix is making the respective field readonly:
class Foo(admin.ModelAdmin):
readonly_fields = ('foreign_key_field1','foreign_key_field2',)
You can still display these fields, there will simply not be a select box, hence Django does not need to retrieve all the select box options from the database.
So, the extra field in a queryset can be used to add additional columns to your select query, which in turn can be set as the default ordering. I have so far been able to achieve this: created an extra field and then set it as default ordering.
qs = qs.extra(select={'natname':"concat('0', nat, name)"}, order_by=['natname'])
Now, in my admin interface, I have my other fields {name, nat, location, space, ....} and the results from the table are ordered by natname when the page is loaded...perfect.
But now I wish to enable ordering on the name field, but instead of ordering by name, I want it to order it by natname. Is this possible?
So even though natname is an extra field, I want to somehow bind the column name with natname when it comes to ordering.
Right now, if I do qs.query.__str__(), I get the sql query with order by natname. When I click on the column name, order by changes to name, but just for this special case, I wish it to order it by natname. Is this possible?
I have looked at how Django generates the headers and the views for these automated admin pages under <django-installation-path>/contrib/admin but it only references the list_display set defined in the ModelAdmin for this model. If I make any changes there, the columns displayed get changed in the admin-view.
It might sound a bit confusing. If you require particular details, please feel free to ask.
Thanks.
Yes! this is possible (at least in Django 1.2.3):
In your admin.ModelAdmin subclass:
from django.contrib import admin
from .models import ClassA
class ModelAdminA(admin.ModelAdmin):
list_display = ('natname',)
def natname(self, obj):
return obj.name
natname.admin_order_field = 'natname'
natname.short_description = 'name'
def queryset(self, request):
qs = super(ModelAdminA, self).queryset(request)
qs = qs.extra(select={ 'natname': "concat('0', nat, name)" })
return qs
admin.site.register(ClassA, ModelAdminA)
I actually want to accomplish the same thing a user in
Remembering checked checkboxes across pages - what's the best way?
asked. But not for php, I want it for django. Since django is just great :-)
I`m using pagination and want to remember the checked checkbox while the user is navigating over the pages provided by pagination.
Now I'm searching for a best practice how I could accomplish this. I do not really have a good idea how i could accomplish this. My idea would include javascript, but I'm sure there is somewhere out there a solution without js.
I can't think of any way to do this with a paginator. But this is a good spot for a formwizard and dynamic forms. The general idea is to create a dynamic form for each "page" and then a formwizard to hold them all together. This allows for saving the checkboxes across multiple pages and lets you go backwards and forwards easily. The best thing is that it takes virtually no code!
Something like this should be able to deal with everything:
from django.contrib.formtools.wizard import FormWizard
from django import forms
from django.forms.extras.widgets import CheckboxSelectMultiple
from django.core.paginator import Paginator
# In your forms.py --------------
def MPageFormMaker(paginator):
"""
Create a "paginated" group of forms based on a queryset and num-per-page item.
"""
def Sform(this_q):
"""
Create this page's items
"""
class _PageForm(forms.Form):
items = forms.ModelMultipleChoiceField(queryset = this_q,
widget = forms.CheckboxSelectMultiple)
return _PageForm
for i in range(paginator.num_pages):
yield Sform(paginator.page(i).object_list)
class MpageForm(FormWizard):
def done(self, request, formlist):
#do something with the list of forms
#----- In your views.py
def MpageChecker(request):
qset = Item.objects.all()
paginator = Paginator(qset, 30)
formwizard = MPageForm(list(MPageFormMaker(paginator)))
#then deal with it like a normal formwizard
Essentially just instantiate the formwizard class and then let it take care of everything. Since it uses a paginator class to make the forms you can use any sort of personalization you'd like.
BTW: I haven't tested this code so it may have a few typos but it should be enough to get you on your way.
EDIT ... fix the ordering problem, now all ordering should be preserved correctly across pages!