I have a page on my site that is dedicated to group comments. Users can create posts (i.e. questions or comments), and other users can comment on them. Comment hierarchy is identical to Facebook; there is no comment threading like reddit. Using Heroku.
I'm using django-debug-toolbar to optimize this page. Right now the page loads in 5-6 seconds; that's just the time that my browser waits for a response from the server (doesn't include JS/CSS/IMG load times). There are 30 SQL queries take between 80-100ms to load (am using prefetch_related on relevant m2m fields). The size of the page returned is 344KB, so not a monster page at all. The comments are paginated and I'm only returning 10 posts + all comments (in test each post only has 3-4 comments).
I can't figure out why it takes such a long time to load the page when the SQL queries are only taking up to 100ms to complete.
Below is the relevant code. I am getting back the 'posts' object and using a for loop to render each object. Within that 'posts' for loop I'm doing another for loop for the 'comments'.
What else can I do to further optimize this/reduce page load time?
# views.py
def group_app(request, course_slug):
# get the group & plan objects
group = Group.objects.get(slug=course_slug)
# load the page with a new Post and comment form
group_post_form = newGroupPost()
post_comment_form = newPostComment(auto_id=False)
# gather all the Posts that have already been created
group_posts = GroupPost.objects.filter(group=group).prefetch_related('comments', 'member', 'group')
paginator = Paginator(group_posts, 10)
page = request.GET.get('page')
try:
posts = paginator.page(page)
except PageNotAnInteger:
page = 1
posts = paginator.page(page)
except EmptyPage:
posts = paginator.page(paginator.num_pages)
variables = RequestContext(request, {
'group': group,
'group_post_form': group_post_form,
'posts': posts,
'next_page': int(page) + 1,
'has_next': posts.has_next(),
'post_comment_form': post_comment_form
})
if request.is_ajax():
return render_to_response('group-post.html', variables)
else:
return render_to_response('group-app.html', variables)
# group-app.html
{% extends 'base.html' %}
{% block canvas %}
<div id="group-body" class="span8">
{% include "group-body.html" %}
</div>
{% endblock canvas %}
# group-body.html
{% for post in posts %}
{% include 'group-post-item.html' %}
{% endfor %}
# group-post-item.html
{% with post_avatar=post.member.get_profile.medium_image.url %}
<div class="groupPost" id="{{ post.id }}">
<div class="avatarContainer">
{% if post_avatar %}
<img class="memberAvatar" src="{{ post_avatar }}" alt="">
{% else %}
<img class="memberAvatar" src="{{ STATIC_URL }}img/generic-avatar.png">
{% endif %}
</div>
<div class="postContents">
<div class="groupPostComment">
<h6 class="comment-header">SHOW/ADD COMMENTS</h6>
<div class="comments">
{% for comment in post.comments.all %}
{% with comment_avatar=comment.member.get_profile.medium_image.url %}
<div class="commentText">
<p class="comment-p">{{ comment.comment }}</p>
</div>
{% endwith %}
{% endfor %}
</div>
</div>
</div>
</div>
{% endwith %}
As I understand this is related to avatar image and not SQL queries. Working with remote storages like S3 is a bit slow. But it should not hamper the user experience as you have faced above.
I didn't get what you are using to get 'avatar' image, but I would suggest to use some third party packages like.
Easy_thumbnails - It is really nice and has lots of options. I recently shifted to this you. In case of remote storages, you could pregenerate avatars with celery and serve saved thumbnail
sorl-thumbnail - I used this for my previous project. It doesn't have option to pregenerate, so first request is slow. But once thumbnail is created, next request will be fast. It has smart caching. I tried to fork it here to use with celery https://github.com/neokya/sorl-thumbnail-async
I hope it helps.
Related
I'm making a portfolio project where I'm using the Google Books API to do a books search, and the Django Paginator class to paginate the results. I've been able to get search results using a CBV FormView and a GET request, but I can't seem to figure out how to get pagination working for the API response.
The solution I can think of is to append &page=1 to the url of the first search, then pull that param on every GET request and use that to paginate. The problem is, I can't figure out how to append that param on the first search, and I don't know how I'd increment that param value when clicking the pagination buttons.
Here's what I've got now:
Form:
class SearchForm(forms.Form):
search = forms.CharField(label='Search:', max_length=150)
View:
class HomeView(FormView):
template_name = "home.html"
form_class = SearchForm
pageIndex = 0
def get(self, request, *args, **kwargs):
# get submitted results in view and display them on results page. This will be swapped out for an AJAX call eventually
if "search" in request.GET:
# getting search from URL params
search = request.GET["search"]
kwargs["search"] = search
context = super().get_context_data(**kwargs)
# Rest API request
response = requests.get(
f'https://www.googleapis.com/books/v1/volumes?q={search}&startIndex={self.pageIndex}&key={env("BOOKS_API_KEY")}'
)
response = response.json()
items = response.get("items")
# pagination...needs work
paginator = Paginator(items, 2)
page_obj = paginator.get_page(1)
context["results"] = page_obj
return self.render_to_response(context)
else:
return self.render_to_response(self.get_context_data())
Template:
{% extends "base.html" %}
{% block content %}
<form action="/">
{{ form }}
<input type="submit" value="Submit">
</form>
<h1>Books</h1>
<ul>
{% for result in results %}
<li>{{ result.volumeInfo.title }} : {{result.volumeInfo.authors.0}}</li>
{% empty %}
<li>Search to see results</li>
{% endfor %}
</ul>
{% if results %}
<div class="pagination">
<span class="step-links">
{% if results.has_previous %}
« first
previous
{% endif %}
<span class="current">
Page {{ results.number }} of {{ results.paginator.num_pages }}
</span>
{% if results.has_next %}
next
last »
{% endif %}
</span>
</div>
{% endif %}
{% endblock content %}
I also looked at Django REST Framework for this, but the Google Books API response doesn't contain any info on next page, previous page, etc. I've done this kind of pagination in React and it's not difficult, I'm just having trouble adjusting my mental model for how to do this to Django. If anyone could offer some advice on how to make this work, I'd be very grateful.
I have a django app hosted on heroku, and I have a 'family album' page that loads a bunch of image thumbnails which link to a larger image detail page. (And the set of pictures can be different per user).
Each of these thumbnails uses an image_link.html template. At the most there can be up to ~1100 of these thumbnails on the family album page, and it loads peachy in production.
The problem: now I've added a little hover overlay (grabbed from w3schools here), so that when you mouse over a thumbnail you can see an overlay of who the picture contains. This list of is returned by a method on the Image class- it's not an attribute, so it's not returned with the Image objects.
Adding this has made the template really slow to load: 4-5 seconds locally, and ~22 seconds in production (Heroku, using 1 professional Dyno). I think usually this sort of thing would be made better by pagination, but in this case I like having one long page. (Though I'd be fine with having the top part load first and then the rest fill in afterward).
So I've done a few things:
added a 'get_image_index_data' function in views.py to loop through, get the pictured_list results, make an array with what I'll need, and cache it. I'm calling this in a receiver function (listening for signal that user logged in) so it'll be set by the time the user clicks the Family Album link, and views.py gets it from the cache
I added template fragment caching on the family album page
Issues: even with the data cached (and I confirmed that there wasn't a miss), this can still be slow-loading (until the template fragment caching kicks in? investigating...)
Here's the code:
I have a receiver function that calls this code to save the data:
def get_image_index_data(accessible_branches, profile):
image_list = Image.objects.none()
for branch in accessible_branches:
name = branch.display_name
image_list = image_list.union(Image.objects.filter(branches__display_name__contains=name).order_by('year'))
sorted_list = image_list.order_by('year')
# Save time on Family album page (aka image_index) by calling ahead for pictured_list. Elsewhere the template will retrieve it
family_album_data = []
for image in sorted_list:
family_album_data.append([image, image.pictured_list])
image_cache_name = 'images_' + str(profile.user)
cache.set(image_cache_name, family_album_data, 60 * 30) # save this for 30 minutes
return family_album_data
Then here's the function to render the family album:
#login_required(login_url=login_url)
def image_index(request):
profile = get_display_profile(request).first()
accessible_branches = get_valid_branches(request)
image_cache_name = 'images_' + str(profile.user)
family_album_data = cache.get(image_cache_name)
if not family_album_data:
family_album_data = get_image_index_data(accessible_branches, profile)
context = {'image_list': family_album_data, 'accessible_branches': accessible_branches, 'branch2_name': branch2_name,
'profile': profile, 'user_person': profile.person, 'media_server': media_server, 'user': profile.user}
return render(request, 'familytree/image_index.html', context)
And here's the family album template (image_index.html):
{% extends 'familytree/base.html' %}
{% block title %} - family album{% endblock title %}
{% block content %}
{% load cache %}
{% cache 1800 album profile %}
<h1>Family Album</h1>
{% if image_list %}
{% for image in image_list %}
{% include "familytree/image_link.html" with image=image height=150 show_hover=True pictured_list=pictured_list%}
{% endfor %}
{% else %}
<p>No images are available.</p>
{% endif %}
{% endcache %}
{% endblock content %}
image_link.html (the new bit is the overlay span in the 'if show_hover' block):
<div style="display: inline-block; margin-bottom:10px"; class="image_container">
<a href="{% url 'image_detail' image.id %}">
{% if image.little_name %}
<img src="{{ media_server }}/image/upload/h_{{ height }},r_20/{{ image.little_name }}" class="image"/>
{% else %}
<img src="{{ media_server }}/image/upload/h_{{ height }},r_20/{{ image.big_name }}" class="image"/>
{% endif %}
{% if show_hover %}
<span class="overlay">
<span class="text">Pictured: {{pictured_list}}</span>
</span>
{% endif %}
<div style="padding-left: 8px">
{% if image.year %}
({{ image.year }})
{% endif %}
</div>
</a>
</div>
Some suggestions:
Delegate the cache building on a background process handler like celery
If you want to display a long page, have a look at doing a facebook-like infinite scroll pagination, to give the illusion that you have a long page but still paginating in the background.
I am using django as backend for website
There will be 10 qwestions
i want to paginate them (one qwestion's ans is given then go to next so on ) and atlast give marks calculated based on all the answers given
Any hints on how to do it?
I found somthing similar but dont know how to implement in django
What's the most efficient way to calculate a running total/balance when using pagination (PHP, MySQL)
Have you looked to Django Pagination docs?
https://docs.djangoproject.com/en/2.1/topics/pagination/
From that page
Your view:
from django.core.paginator import EmptyPage, PageNotAnInteger, Paginator
from django.shortcuts import render
def listing(request):
contact_list = Contacts.objects.all()
paginator = Paginator(contact_list, 25) # 25 is the items per page, in Your case would be 1
page = request.GET.get('page')
contacts = paginator.get_page(page)
return render(request, 'list.html', {'contacts': contacts})
Your template:
{% for contact in contacts %}
{# Each "contact" is a Contact model object. #}
{{ contact.full_name|upper }}<br>
...
{% endfor %}
<div class="pagination">
<span class="step-links">
{% if contacts.has_previous %}
« first
previous
{% endif %}
<span class="current">
Page {{ contacts.number }} of {{ contacts.paginator.num_pages }}.
</span>
{% if contacts.has_next %}
next
last »
{% endif %}
</span>
</div>
To go to a page you should add to the url ?page=1 where 1 is the page number.
I am a beginner in Django, and I am really thoughtful about how things really work under the Django hood. I am currently implementing pagination in my web app.
Take a look at this view.py file:
def post_list(request):
object_list = Post.published.all(); # '.published' is a manager.
paginator = Paginator(object_list, 3); # 3 posts in each page.
page = request.GET.get("page");
try:
posts = paginator.page(page);
except PageNotAnInteger:
posts = paginator.page(1);
except EmptyPage:
posts = paginator.page(paginator.num_pages);
return render(request, "post/list.html", {"page": page, "posts": posts});
Isn't request.GET a dictionary object containing all the the GET request parameters that are in the url, and the .get() method used to return the value
for the given key inside the parameter? As my URL is currently just localhost:8000 when I start the application, why does it work if I pass the key "page"?
My list.html file:
{% extends "base.html" %}
{% block title %}My Blog{% endblock %}
{% block content %}
<h1>My Blog</h1>
{% for post in posts %}
<h2>{{ post.title }}</h2> <!-- How does this absurl work?-->
<p class="date">Published {{ post.publish }} by {{ post.author }}</p> <!-- What does '.publish' print?-->
{{ post.body|truncatewords:30|linebreaks }}
{% endfor %}
{% include "pagination.html" with page=posts %}
<!-- The statement above is the little menu: "Page 1 of 2. Next" -->
<!-- It also sends the 'page' variable as a GET parameter. -->
{% endblock %}
My pagination.html file:
<!-- This pagination template expects a paginator object. -->
<div class="pagination">
<span class="step-links">
{% if page.has_previous %}
Previous
{% endif %}
<span class="current">
Page {{ page.number }} of {{ page.paginator.num_pages }}. <!-- what!? -->
</span>
{% if page.has_next %}
Next
{% endif %}
</span>
</div>
When there are no parameters in the request (when you hit http://localhost:8000 directly), the value of page will be None. That is the default behaviour of request.GET.get() when it can't find the key you're asking for - the same as a normal Python dictionary (because GET extends it).
# page will be None
page = request.GET.get("page")
This means that None is passed to paginator.page():
try:
# Passing None here
posts = paginator.page(page)
except PageNotAnInteger:
Which likely means (although we can't see the code of paginagor) that a PageNotAnInteger exception is raised, and thus a value of 1 is passed to paginagor.page():
try:
posts = paginator.page(page) # Raises PageNotAnInteger because None passed
except PageNotAnInteger:
# Posts are retrieved for page 1
posts = paginator.page(1)
The posts from the above call, and the value of page (still None) are then passed to the template.
return render(request, "post/list.html", {"page": page, "posts": posts});
The template list.html then iterates the posts and displays them.
Rather confusingly, when the pagination.html template is included, it defines a context variable called page to the current value of posts:
<!-- Pass the value of posts using a variable name of page -->
{% include "pagination.html" with page=posts %}
So the places where the pagination.html template refers to page, it is actually using the value of posts.
<!-- Really posts.number and posts.paginator.num_pages -->
Page {{ page.number }} of {{ page.paginator.num_pages }}
Hope that helps to explain things.
One other thing, you don't need to add a semi-colon to the end of every line in Python.
I am using django pagination, as told in documentation:
view part is :
def list(request):
job_list = Job.objects.all()
paginator = Paginator(job_list, 25) # Show 25 jobs per page
page = request.GET.get('page',1)
try:
jobs = paginator.page(page)
except PageNotAnInteger:
# If page is not an integer, deliver first page.
jobs = paginator.page(1)
except EmptyPage:
# If page is out of range (e.g. 9999), deliver last page of results.
jobs = paginator.page(paginator.num_pages)
return render_to_response('jobs/list.html', {"jobs": jobs})
and template is:
<div>
{% for job in jobs %}
{# Each "contact" is a Contact model object. #}
{{ job.title|upper }}<br />
{% endfor %}
<div class="pagination">
<span class="step-links">
{% if contacts.has_previous %}
previous
{% endif %}
<span class="current">
Page {{ contacts.number }} of {{ contacts.paginator.num_pages }}.
</span>
{% if contacts.has_next %}
next
{% endif %}
</span>
</div>
</div>
But it gives the error saying:
In template d:\programming\django_projects\kaasib\templates\jobs\list.html, error at line 32
Caught TypeError while rendering: 'Page' object is not iterable
I am new in django and this error seems general but very strange. Because in loop there is some other variable, not job. So please tell if anyone have any idea about it.
thanks
The error should be clear - the variable you've called jobs actually contains a Page object from the paginator. Which is as it should be, as you assigned jobs to paginator.page(x). So obviously, it contains a Page.
The documentation shows what to do:
{% for job in jobs.object_list %}
etc.