I've a model with approximately 150K rows.
It takes 1.3s to render the ListView for this model.
When I click the change link in the ListView I takes almost 2 minutes to render the change view.
Other models have normal render times for the edit view.
Any ideas how to speed this up?
Your best bet is to limit the number of returned rows and implement some type of pagination in your application.
Django conveniently implements a type of pagination
First of all, ask yourself these questions:
Do you have much work with your data in templates?
Can I do this work in a backend and in a template only render it?
Do I use pagination?
As I know pagination in Django implemented with LIMIT and OFFSET sql statements, which work not so quickly when you're having many pages. In our projects, we wrote a row SQL for this purpose which works a little bit faster.
Also, you can install Django Debug Toolbar which can show you what statements django ORM is executing and measure time.
Related
This might sound like a stupid question so apologies in case I'm wasting your time. I have a tons or results coming from my data in my Django project. It's a table with many columns and almost 4000 rows. I am using Datatables for pagination, filtering, horizontal scrolling, sorting the columns.
Client-side (I meant server-side) I am also using django-filter for querying the database.
My problem is that the loading of the initial data (non filtered via django-filter) takes a lot of time. Shall I implement pagination on the client-side (I meant server-side)? If so, how does this work with Datatables? Will Datatables paginate/display only the (first page of) paginated data coming from the server-side query? Is there a way for the two to work together?
Thanks.
How can I make Django Rest Frameworks browsable UI fast with RelatedField?
I'm aware this has already been asked here: Django REST Framework: slow browsable UI because of large related table but the answer is no longer valid for new versions of DRF
Including two PrimaryKeyRelatedFields gives me a 5s+ load time, removing them takes me back down to under .3
I've tried setting html_cutoff=100 or even html_cutoff=1but it seems to make no difference to load times.
Any ideas? currently on DRF '3.3.2'
Edit: tables involved have 12000 to 120 records - but it would be great to handle much larger amounts
Since DRF version 3.4.4, it's possible to limit number of relationships being displayed by using selected fields cutoffs.
From DRF documentations:
When rendered in the browsable API relational fields will default to only displaying a maximum of 1000 selectable items. If more items are present then a disabled option with "More than 1000 items…" will be displayed.
...
You can also control these globally using the settings HTML_SELECT_CUTOFF and HTML_SELECT_CUTOFF_TEXT.
This question is similar or duplicate of this one Django REST Framework: slow browsable UI because of large related table.
In essence it's N+1 Problem and in context of Django it can be fixed by eager loading of data by calling prefetch_related() or select_related() on QuerySet. Check this answare
Not quite the answer I am looking for, but currently it looks like there is activity around this already on github - https://github.com/tomchristie/django-rest-framework/issues/3329 with a little luck, one of those patches will get merged soon
The use case is I want to statically render a view daily. It seems like there should be a pretty standard way to take a view/template and render static contents daily without simply saying "write a custom admin command" or a relatively simple command template that populates a static file.
The reason is to remove a large volume of database queries to make a site lightening quick, even on a lightweight vps by only touching the database daily instead of on every page view.
If there's a better way to do it, I'm open to that. It just seems like the best way to do it is rendering static views on a regular basis and cache-ing the crap out of it before it even touches django.
There are several ways I know to solve this:
1. You can use Varnish (as described in this blog post). Yet this solution takes a bit more time to get into because it's side technology you'll have to deal with. Also it takes more efforts to maintain it.
2. More "django-side" solution is to use django-celery for daily rendering your view and storing it in cache. You can move all your static view logic into task and render it there once a day. In your view you can just get rendered response from cache and return it to user.
3. Also you can use django per-view cache and create task in celery to clear cache daily.
I would like to filter on some columns that only have a couple of possible values, with only a couple of values the side filter wastes a lot of screen real estate I need for data. So I was wondering if there was some way to have a filter in the top bar next to the date selector.
Consider using Grappelli. It is a very popular django app, it modifies the admin interface and the filter section (to be pop-up instead of fixed to right)
If you want that, you are essentially talking about modifying django admin template behavior. For that you would need to create a admin template in you own django app and you can change the way the template is rendered...
I am a django noob and am trying to figure out how to get the admin module to do something slightly different than the normal operation on a single model. Essentially what I need is to run a query and display the results of the query as a view page and then allow the link to the edit page take the user to an existing model's edit view. 2 of the 3 tables in my query are related, but not all 3.
Example:
select a.foo, a.second_field, b.bar, c.unrelated_field
from a, b, c
where a.primary_key = b.foreign_key
and a.some_value = c.some_value
Note that a and c are not defined as related tables.
I would like to have a view of this query output and have a link to the edit view of the b model as a whole when selected.
I have created a view in the DB for this query and simply created a new model which makes it easy to get the view, but I'm not sure this is even the right approach to start with...but from there I can't seem to figure out how to make this link to the edit page for the B table.
Any pointers or advice on how best to accomplish something like this with django admin would be appreciated!
Using Django 1.3.1 by the way.
Cheers!
You can override change_view in your ModelAdmin so it will construct a list of dicts with all your needed data. Then override change_list.html template to display this data correctly and link it with change_form view for correct model. So it will flawlessly integrate in Django's admin site.
And I don't like DB views as long as it's possible to solve the problem without it. If data can be constructed in Python without massive performance gaps and lots of magic code, it should be processed in Python.