How to query in Django with best efficiency? - django

Question

I recently found that too much SQL query optimization issue. django-debug-tool reported hundreds of similar and duplicate queries. So, I'm trying to figure out the best efficiency of Django ORM to avoid unnecessary Queryset evaluation.
As you see the below Store model, a Store model has many Foreign key and ManyToManyFields. Due to that structure, there are many code snippets doing the blow on HTML template files such as store.image_set.all or store.top_keywords.all. Everything starts with store. In each store detail page, I simply pass a cached store object with prefetch_related or select_related. Is this a bad approach? Should I cache and prefetch_related or select_related each Foreign key or ManyToManyField separately on views.py?
HTML templates
{% for img in store.image_set.all %}
{{ img }}
{% endfor %}
{% for top_keyword in store.top_keywords.all %}
{{ top_keyword }}
{% endfor %}
{% for sub_keyword in store.sub_keywords.all %}
{{ sub_keyword }}
{% endfor %}
views.py
class StoreDetailView(View):
def get(self, request, *args, **kwargs):
cache_name_store = 'store-{0}'.format(store_domainKey)
store = cache.get(cache_name_store, None)
if not store:
# query = get_object_or_404(Store, domainKey=store_domainKey)
query = Store.objects.all().prefetch_related('image_set').get(domainKey=store_domainKey)
cache.set(cache_name_store, query)
store = cache.get(cache_name_store)
context = {
'store': store,
}
return render(request, template, context)
models.py
class Store(TimeStampedModel):
categories = models.ManyToManyField(Category, blank=True)
price_range = models.ManyToManyField(Price, blank=True)
businessName = models.CharField(unique=True, max_length=40,
verbose_name='Business Name')
origin = models.ForeignKey(Origin, null=True, on_delete=models.CASCADE, blank=True)
ship_to = models.ManyToManyField(ShipTo, blank=True)
top_keywords = models.ManyToManyField(Keyword, blank=True, related_name='store_top_keywords')
sub_keywords = models.ManyToManyField(SubKeyword, blank=True, related_name='store_sub_keywords')
sponsored_stores = models.ManyToManyField(
'self', through='Sponsorship', symmetrical=False, related_name='sponsored_store_of_store')
similar_stores = models.ManyToManyField(
'self', through='Similarity', symmetrical=False, related_name='similar_store_of_store')
shortDesc = models.TextField(blank=True, verbose_name='Short Description')
longDesc = models.TextField(blank=True, verbose_name='Long Description')
returnPol = models.TextField(verbose_name='Return Policy', blank=True)
returnUrl = models.CharField(max_length=255, null=True, blank=True, verbose_name='Return Policy URL')
likes = models.ManyToManyField(settings.AUTH_USER_MODEL, blank=True, editable=False)
created_by = models.ForeignKey(settings.AUTH_USER_MODEL, editable=False, on_delete=models.CASCADE,
related_name='stores_of_created_by', null=True, blank=True)
updated_by = models.ForeignKey(settings.AUTH_USER_MODEL, editable=False, on_delete=models.CASCADE,
related_name='stores_of_updated_by', null=True, blank=True)

Answer 1

I really wouldn't advise custom caching/performance optimisation, unless it's a very last resort. Django has great docs on querysets and optimisation - if you follow those, it should be rare for you to experience major performance issues that require custom workarounds.
I think the issue here is that you're printing your objects in a template and hence calling their str() method. There's nothing wrong with this, but I'd check what variables you're using in your str() methods. I suspect you're referencing other models? I.e. the str() method in your image model (or whatever) is doing something like image.field.other_field. In this case, your query should look like:
queryset = Store.objects.prefetch_related('image_set__field')
Your final queryset may look like:
queryset = Store.objects.prefetch_related('image_set__field1', 'image_set__field2', 'top_keywords__field3', ...)
Note that you can still pass this into get_object_or_404 like so:
get_object_or_404(queryset, pk=<your_stores_id>)
Hope this helps.