Django admin cache m2m relation - django

I have two models:
models.py
class City(models.Model):
title = models.CharField(max_length=255)
show = models.BooleanField(default=True)
class Company(models.Model)
title = models.CharField(max_length=255)
cities = models.ManyToManyField(City, null=True, blank = True)
admin.py
class CompanyAdmin(admin.ModelAdmin):
search_fields = ('title',)
filter_horizontal = ('cities',)
It is about 23000 cities in the database.
When i edit the Company detail in Admin it loads forever!!! Just accessing admin/myapp/company/12/ takes 2-3 minutes - this is horrible.
How can i speed up things and cache City model queryset?

You can use raw_id_fields
class CompanyAdmin(admin.ModelAdmin):
raw_id_fields = ("cities",)

23000 is actually "nothing" for a database.
First, you need to identify the "culprit" of a slowdown, which query is slow. This is where django-debug-toolbar would help a lot. Then, you can run the queries manually with EXPLAIN and see what can be improved - for example, adding indexes could help.
Also, consider using django-cache-machine or johnny-cache packages.
Also see:
Tweaks for making django admin faster

Related

Slow Django many to many query

I have 9 Million records.
It will be 200M soon.
It can take 15min + to fetch this:
class Follower():
hashtags = models.ManyToManyField(
"instagram_data.Hashtag", verbose_name=_("hashtags_name"))
class Hashtag(models.Model):
name = models.CharField(_("HashtagName"), max_length=150, null=True, blank=True, unique=True)
def __str__(self):
return self.name
Ubunto htop:
I think it reverse lookup for all values.
I think it will have maybe 2000 records found.
What am I doing wrong ?
The image you post looks like the model admin, if that is the case try adding the field hashtags to the raw_id_fields [Django docs] of the model admin (although then you won't get the select tag and will have to manually enter the id/pk) or the autocomplete_fields [Django docs] which would use select2 to load the choices asychronously. This would work something like:
class HashtagAdmin(admin.ModelAdmin):
ordering = ['name']
search_fields = ['name']
class FollowerAdmin(admin.ModelAdmin):
autocomplete_fields = ['hashtags']
admin.site.register(Follower, FollowerAdmin)
admin.site.register(Hashtag, HashtagAdmin)
If this is not in the model admin you can change the widget you use in the form on the form class (you will need to build a custom widget or look for some package that provides widget that can allow comma separted values), or you can use the package Django-Select2 to give the user a searchable select tag (again using select2).

Can I use unique_together on ManyToMany field?

I have a model OrderPage which is manytomany to Site. In Django admin, I want to restrict the selection of sites(Sites which belong to existing OrderPage can not be selected again). Can I do it with unique_together ? I get an error with following model ManyToManyFields are not supported in unique_together
class OrderPage(models.Model):
description = models.CharField(max_length=255, blank=False)
sites = models.ManyToManyField(Site)
class Meta:
unique_together = (('id', 'sites'),)
class Order(models.Model):
order_page = models.ForeignKey(OrderPage)
class OrderPageAdmin(admin.ModelAdmin):
filter_horizontal = ('sites',)
admin.site.register(OrderPage, OrderPageAdmin)
If an Site can have only one OrderPage, you don't need to worry about unique_together.
Ideally you should subclass Site and use a ForeignKey from that to OrderPage. That would natively give you what you're looking for: each site would be able to have one OrderPage, and each OrderPage multiple Sites. This would be the cleanest but you would have to use your subclass throughout the program in place of the original Site which might be more work than you want right now.
class BetterSite(Site):
order_page = models.ForeignKey('OrderPage')
The dirtier way is to keep your M2M and just set the site as unique, since there should only ever be one entry on each site in the M2M table. You would use a 'through' table so you could set the custom uniqueness value:
class OrderPage(models.Model):
description = models.CharField(max_length=255, blank=False)
sites = models.ManyToManyField(Site, through='OrderPageToSite')
class OrderPageToSite(models.Model):
order_page = models.ForeignKey(OrderPage)
site = models.ForeignKey(Site, unique=True)
(Note that I've left these simple but in your FK fields you should also consider setting on_delete and related_name)

Django admin list_display not showing several objects

I have just begun to play around with Django admin views, and to start off, I am trying to do something very simple: showing several fields in the listing of objects using list_display as explained here: https://docs.djangoproject.com/en/dev/ref/contrib/admin/
This is my dead simple code:
class ArticleAdmin(admin.ModelAdmin):
list_display = ('title', 'category')
Unfortunately, the list_display option is causing the columnar view to appear, but only some of the objects (40 out of 85) are now displaying in the listing. I cannot deduce why certain objects are showing over the others - their fields look like they are filled similarly. It's clearly not paginating, because when I tried it on an admin of another model, it showed only 2 objects out of about 70 objects.
What might be going on here?
[UPDATE] Article Model:
class Article(models.Model):
revision = models.ForeignKey('ArticleRevision', related_name="current_revision")
category = models.ForeignKey('meta.Category')
language = models.ForeignKey('meta.Language', default=get_default_language)
created = models.DateTimeField(auto_now_add=True, editable=False)
changed = models.DateTimeField(auto_now=True, editable=False)
title = models.CharField(max_length=256)
resources = models.ManyToManyField('oer.Resource', blank=True)
image = models.ManyToManyField('media.Image', blank=True)
views = models.IntegerField(editable=False, default=0)
license = models.ForeignKey('license.License', default=get_default_license)
slug = models.SlugField(max_length=256)
difficulty = models.PositiveIntegerField(editable=True, default=0)
published = models.NullBooleanField()
citation = models.CharField(max_length=1024, blank=True, null=True)
Before adding list_display:
After adding list_display:
[UPDATE] This behaviour occurs only when ForeignKey fields are included in list_display tuple. Any of them.
[UPDATE] Category model code:
class Category(models.Model):
title = models.CharField(max_length=256)
parent = models.ForeignKey('self')
project = models.NullBooleanField(default=False)
created = models.DateTimeField(auto_now_add=True, editable=False)
slug = models.SlugField(max_length=256, blank=True)
def __unicode__(self):
return self.title
This behavior is caused by a foreign key relation somewhere that is not declared as nullable, but nonetheless has a null value in the database. When you have a ManyToOne relationship in list_display, the change list class will always execute the query using select_related. (See the get_query_set method in django.contrib.admin.views.ChangeList).
select_related by default follows all foreign keys on each object, so any broken foreign key found by this query will cause data to drop out when the query is evaluated. This is not specific to the admin; you can interactively test it by comparing the results of Article.objects.all() to Article.objects.all().select_related().
There's no simple way to control which foreign keys the admin will look up - select_related takes some parameters, but the admin doesn't expose a way to pass them through. In theory you could write your own ChangeList class and override get_query_set, but I don't recommend that.
The real fix is to make sure your foreign key model fields accurately reflect the state of your database in their null settings. Personally, I'd probably do this by commenting out all FKs on Article other than Category, seeing if that helps, then turning them back on one by one until things start breaking. The problem doesn't have to be with a FK on an article itself; if a revision, language or category has a broken FK that will still cause the join to miss rows. Or if something they relate to has a broken FK, etc etc.

Stacked Inline with many rows crashing in Django

When trying to navigate to 'Add a presentation' in Django admin, I have to wait ~1 minute for the response to render. The problem is that I have ~500 slides in the database and the admin is selecting all the slides three different times to fill in the menues. I am obviously doing something wrong with my model definitions, as I wouldn't expect this amount of data to bring my server to its knees. Any visibility into why I am experiencing this issue with the way I have defined relationships or am using the django admin?
class PresentationTitle(models.Model):
title = models.CharField(max_length=255)
order_number = models.IntegerField(default=0)
def __unicode__(self):
return self.title
class PresentationUser(models.Model):
user = models.OneToOneField(User)
authorized_modules = models.ManyToManyField(PresentationTitle)
class Presentation(models.Model):
title = models.ForeignKey(PresentationTitle)
user = models.ForeignKey(PresentationUser)
presentation_date = models.DateTimeField()
def __unicode__(self):
return self.title.title
class Slide(models.Model):
....
submodule = models.ForeignKey(Submodule)
presentation = models.ManyToManyField(Presentation, through='PresentationSlide')
...
class Meta:
order_with_respect_to = 'submodule'
ordering = ['order']
class PresentationSlide(models.Model):
presentation = models.ForeignKey(Presentation)
slide = models.ForeignKey(Slide)
slide_order = models.IntegerField()
Additionally, my admin contains:
class PresentationSlideInline(admin.StackedInline):
model = PresentationSlide
class PresentationAdmin(admin.ModelAdmin):
inlines = [PresentationSlideInline]
admin.site.register(Presentation, PresentationAdmin)
Understandably, removing just having PresentationAdmin from the admin.site.register makes it load very responsively.
Resolved my issue -- there was another culprit at play: django-debug-toolbar
After removing this from my setup, the admin panel for the Presentation effectively loads.
Thanks to everyone on the interest and the support, be wary of profiling add-ons when you are experience performance issues.

Django - many-to-one relationship - how to select parent models that have children only

Say I have this model relationship in Django 1.2...
class Section(models.Model):
title = models.CharField(max_length=200)
class Page(models.Model):
section = models.OneToOneField(Section)
title = models.CharField(max_length=200)
I want to select Sections that have one or more Pages associated with them, how would I achieve this in a query? Or would I have to select all sections and then filter out one's that do not have pages manually?
Change to:
class Section(models.Model):
title = models.CharField(max_length=200)
page = models.ForeignKey(Page, related_name="section")
class Page(models.Model):
title = models.CharField(max_length=200)
select Sections that have one or more Pages associated:
result_q = Section.objects.filter(page__isnull=False)
I would second sza's answer if was 100% sure that isnull works in all such cases - but I haven't checked that so I'm not sure (even though I occasionaly use it for such purposes) :-)
What I'm sure of is this:
from django.db import models
Section.objects.annotate(page_num=models.Count('page')).filter(page_num__gt=0)
-- and while counting might be an overkill for your case wih OneToOneField relationship, it definitely works.