I'm started using the Django REST framework in preparation for production, but unfortunately, it is performing quite slowly.
I am calling an array of 500 dictionaries, each with 5 key-value pairs each. In the shell, the call-time is not noticeable at all - you press enter, and it's done. Previously, when I was serving the same content directly without the REST framework, there was also no noticeable delay. However, with the REST framework, it takes about 1 - 2 seconds after the page has rendered for the content to display.
I do not think this is due to javascript as hitting the same details through the browseable API results in a similar delay.
Also, I am NOT cacheing at the moment.
There's no way anyone else is going to be able to debug this for you from the details given in the question.
Are you reusing an existing generic view or writing your own view?
Are you serializing the data, if so what does your serializer definition look like?
Does the issue manifest when rendering to JSON, or just when rendering to the Browsable API?
You mention serving the content without REST framework - what did the views looks like before, and what do they look like after?
The REST framework views are trivial, so use a profiling tool, or simply override them and add some timing. Likewise the renderers are trivial - subclass whatever renderer you're using at the moment, override .render() and add a couple of timing calls either side of calling the parent's .render() method.
If you think you've narrowed down a problem to a specific area then throw together a minimal test case and submit it as an issue.
The serialization itself is unlikely to be an issue, I've used the same serialization engine to replicate Django's fixture dumping and there was no significant degradation.
It's feasible that if you're doing lookups across model relationships you might need to add .select_related() or .prefetch_related() calls when constructing the queryset, exactly as you would with any other Django view.
Edit: Note that following on from this post there were significant serializer speed improvements made, as noted in this ticket.
Related
I have a fairly large django project consisting of several individual apps. I am farming out some of the front-end work (CSS, HTML tweaks) to people who aren't over-familiar with django. To that end I'd like to generate a list of templates for each URL pattern any given engineer is working on. This will save much time that would otherwise be spent manually tracking down the templates used during a view's render phase.
For example, if Bob is working on URLs beginning with /accounts/ then I'd like to generate a list of all the templates used by any view that handles requests to those URLs.
My initial thought is to use something in the test framework since that has access to the templates rendered during a request. However, I can't guarantee that all URLs or views will be exercised (sadly I don't have 100% test coverage), and a missed template is unlikely to be noticed. I don't mind writing a set of tests that simply exercise each view, but don't want to duplicate existing efforts. Also certain views require POSTed data or authentication to function correctly - although I suspect that's an issue I'll have to face no matter what approach is used.
Are there any utilities or snippets that will do what I need?
django-debug-toolbar is a must for developing with Django, It includes a panel detailing all templates used during a request.
I've found the SQL panel is the most helpful for improving page load times as it details slow and duplicate queries.
It can slow down requests when enabled, disabling all panels but those that you use helps.
How can I make Django Rest Frameworks browsable UI fast with RelatedField?
I'm aware this has already been asked here: Django REST Framework: slow browsable UI because of large related table but the answer is no longer valid for new versions of DRF
Including two PrimaryKeyRelatedFields gives me a 5s+ load time, removing them takes me back down to under .3
I've tried setting html_cutoff=100 or even html_cutoff=1but it seems to make no difference to load times.
Any ideas? currently on DRF '3.3.2'
Edit: tables involved have 12000 to 120 records - but it would be great to handle much larger amounts
Since DRF version 3.4.4, it's possible to limit number of relationships being displayed by using selected fields cutoffs.
From DRF documentations:
When rendered in the browsable API relational fields will default to only displaying a maximum of 1000 selectable items. If more items are present then a disabled option with "More than 1000 items…" will be displayed.
...
You can also control these globally using the settings HTML_SELECT_CUTOFF and HTML_SELECT_CUTOFF_TEXT.
This question is similar or duplicate of this one Django REST Framework: slow browsable UI because of large related table.
In essence it's N+1 Problem and in context of Django it can be fixed by eager loading of data by calling prefetch_related() or select_related() on QuerySet. Check this answare
Not quite the answer I am looking for, but currently it looks like there is activity around this already on github - https://github.com/tomchristie/django-rest-framework/issues/3329 with a little luck, one of those patches will get merged soon
I have a site which uses the standard Django comments (well, a subclass of that, but mostly the same). I want to cache the list of comments that's rendered on each page, as that's quite a big and slow query. But, while I know how to cache individual querysets, I can't see how best to do it for the comments app.
It looks like the querysets for these lists of comments are generated in the BaseCommentNode templatetag. So I can't see an easy way to see if there's a cached version of that QS and return that if so... what's my best way of nicely caching this query?
(I'm also caching every page for all logged-out users, with a 5 minute expiry, but think my site would benefit from caching queries like this for a longer period.)
I'm an advanced beginner at django and python. I'm writing an app to handle registration and abstract submission for a conference, and I'm trying to use class-based views. Users get an emailed link that includes their registration code in the url. Starting at this url, users move through a series of views that collect all the necessary info.
The complication comes from the fact that users often stop half way through, and then want to complete the process several days or weeks later. This means that they might continue from the current page, or they might just click that original link. In addition, after several weeks they might have missed certain deadlines, so, e.g., they can no longer submit an abstract (but they can still register). Along the way, they have checked or unchecked various options that also influence the path they should take through the app.
My question is: where is the best place to put the logic that determines if the user is currently allowed to view that page, and if not, the best url to redirect them too? I thought I would create a custom view class that, e.g., overrides the dispatch method to include global checks (e.g., is conference registration open?), and then subclasses could add additional checks (e.g., has the user entered all the necessary info for her abstract?). The problem I ran into was that the checks were run in the wrong order (I want base class checks run first). I then started investigating custom view decorators or custom middleware. At this point I realized I could use some expert advice about which approach to take. (If it matters, I am not using the django authentication system.) Thanks in advance.
Maybe the form wizard could help you managing the viewing sequence.
In general django greybeards advocate keeping row-wise logic in Models, and table-wise logic in Managers, so it seems appropriate to keep complex view logic in a master view class.
The wizard class can help maintain the order of the views, but to resume an out-dated session you may need to do some model saves (which could get too complex very quickly) or some cookie handling.
In the past, when presented with a similar situation, I took the simplest route and separated user registration and the task that the user wants to perform (event registration). The user registers once but if they fluff up the event registration, they just have to log back in at a later date and do it again (their hassle - not yours!).
Should I be writing application logic in my view code? For example, on submission of a form element, I need to create a user and send him an activation email. Is this something to do from a view function, or should I create a separate function to make it easier to test down the road? What does Django recommend here?
I found it very hard to figure out where everything goes when I started using django. It really depends on the type of logic you are writing.
First start with the models: model-methods and managers are a good place to perform row-level logic and table level logic i.e. a model manager would be a good place to write code to get a list of categories that are associated with all blogposts. A model method is a good place to count the characters in a particular blogpost.
View level logic should deal with bringing it all together - taking the request, performing the necessary steps to get to the result you need (maybe using the model managers) and then getting it ready for the template.
If there is code that doesn't fit in else where, but has a logical structure you can simply write a module to perform that. Similarly if there are scraps of code that you don't think belong, keep a utils.py to hold them.
You shouldn't perform any logic really in your templates - instead use template tags if you have to. These are good for using reusable pieces of code that you you neither want in every request cycle nor one single request cycle - you might want them in a subset (i.e. displaying a list of categories while in the blog section of your website)
If you do want some logic to be performed in every request cycle, use either context processors or middleware. If you want some logic to be performed only in one single request cycle, the view is probably the place.
TLDR: Writing logic in your view is fine, but there are plenty of places that might be more appropriate
Separating the registration code into it's own function to make testing easier is a good reason. If you allowed admins to register users in a separate, private view, then a registration function would be more DRY. Personally, I don't think a little application logic in the code will do to much harm.
You might find it instructive to have a look at the registration view in the django-registration app -- just to see how it's written, I'm not saying you should or have to use it. It has encapsulated the user registration into it's own function (there's a level of indirection as well, because the registration backends are pluggable).