Django: template loading slowly even using data from cache - django

Question

I have a django app hosted on heroku, and I have a 'family album' page that loads a bunch of image thumbnails which link to a larger image detail page. (And the set of pictures can be different per user).
Each of these thumbnails uses an image_link.html template. At the most there can be up to ~1100 of these thumbnails on the family album page, and it loads peachy in production.
The problem: now I've added a little hover overlay (grabbed from w3schools here), so that when you mouse over a thumbnail you can see an overlay of who the picture contains. This list of is returned by a method on the Image class- it's not an attribute, so it's not returned with the Image objects.
Adding this has made the template really slow to load: 4-5 seconds locally, and ~22 seconds in production (Heroku, using 1 professional Dyno). I think usually this sort of thing would be made better by pagination, but in this case I like having one long page. (Though I'd be fine with having the top part load first and then the rest fill in afterward).
So I've done a few things:
added a 'get_image_index_data' function in views.py to loop through, get the pictured_list results, make an array with what I'll need, and cache it. I'm calling this in a receiver function (listening for signal that user logged in) so it'll be set by the time the user clicks the Family Album link, and views.py gets it from the cache
I added template fragment caching on the family album page
Issues: even with the data cached (and I confirmed that there wasn't a miss), this can still be slow-loading (until the template fragment caching kicks in? investigating...)
Here's the code:
I have a receiver function that calls this code to save the data:
def get_image_index_data(accessible_branches, profile):
image_list = Image.objects.none()
for branch in accessible_branches:
name = branch.display_name
image_list = image_list.union(Image.objects.filter(branches__display_name__contains=name).order_by('year'))
sorted_list = image_list.order_by('year')
# Save time on Family album page (aka image_index) by calling ahead for pictured_list. Elsewhere the template will retrieve it
family_album_data = []
for image in sorted_list:
family_album_data.append([image, image.pictured_list])
image_cache_name = 'images_' + str(profile.user)
cache.set(image_cache_name, family_album_data, 60 * 30) # save this for 30 minutes
return family_album_data
Then here's the function to render the family album:
#login_required(login_url=login_url)
def image_index(request):
profile = get_display_profile(request).first()
accessible_branches = get_valid_branches(request)
image_cache_name = 'images_' + str(profile.user)
family_album_data = cache.get(image_cache_name)
if not family_album_data:
family_album_data = get_image_index_data(accessible_branches, profile)
context = {'image_list': family_album_data, 'accessible_branches': accessible_branches, 'branch2_name': branch2_name,
'profile': profile, 'user_person': profile.person, 'media_server': media_server, 'user': profile.user}
return render(request, 'familytree/image_index.html', context)
And here's the family album template (image_index.html):
{% extends 'familytree/base.html' %}
{% block title %} - family album{% endblock title %}
{% block content %}
{% load cache %}
{% cache 1800 album profile %}
<h1>Family Album</h1>
{% if image_list %}
{% for image in image_list %}
{% include "familytree/image_link.html" with image=image height=150 show_hover=True pictured_list=pictured_list%}
{% endfor %}
{% else %}
<p>No images are available.</p>
{% endif %}
{% endcache %}
{% endblock content %}
image_link.html (the new bit is the overlay span in the 'if show_hover' block):
<div style="display: inline-block; margin-bottom:10px"; class="image_container">
<a href="{% url 'image_detail' image.id %}">
{% if image.little_name %}
<img src="{{ media_server }}/image/upload/h_{{ height }},r_20/{{ image.little_name }}" class="image"/>
{% else %}
<img src="{{ media_server }}/image/upload/h_{{ height }},r_20/{{ image.big_name }}" class="image"/>
{% endif %}
{% if show_hover %}
<span class="overlay">
<span class="text">Pictured: {{pictured_list}}</span>
</span>
{% endif %}
<div style="padding-left: 8px">
{% if image.year %}
({{ image.year }})
{% endif %}
</div>
</a>
</div>

Answer 1

Some suggestions:
Delegate the cache building on a background process handler like celery
If you want to display a long page, have a look at doing a facebook-like infinite scroll pagination, to give the illusion that you have a long page but still paginating in the background.