django count specific rows in queryset - django

class Order(models.Model):
name = models.CharField(max_length=100)
# other fields..
user = models.ForeginKey(User)
old = models.BooleanField(default=False)
I want to display all the orders of a specific user, but I want to split them those which are "old" and the ones who are not.
So, currently I do in views.py:
orders = Order.objects.filter(user=user)
In template:
First table:
<table>
{% for order in orders %}
{% if not order.old %}
<tr>
<td>... </td>
</tr>
{% endif %}
{% endfor %}
</table>
And another table:
{% for order in orders %}
{% if order.old %}
<tr>
<td>...</td>
<tr>
{% endif %}
{% endfor %}
This way have some drawbacks, first, now I want to count how many of the orders are "old", to display this number in the template. I can't, unless I do another query.
Is it possible to annotate(number_of_old=Count('old'))? Or I have to do another query?
So what would be the best?
1. Do two queries, one with old=False, another with old=True, and pass two querysets to the template. And use |len filter on the querysets
2. Do one query like this and split them somehow in python? That will be less convenient as I have a similar structures which I want to split like that.
And should I call the DB .count() at all?
EDIT:
If I would write my model like this:
class Order(models.Model):
name = models.CharField(max_length=100)
# other fields..
user = models.ForeginKey(User)
old = models.BooleanField(default=False)
objects = CustomManager() # Custom manager
class CustomQueryset(models.QuerySet):
def no_old(self):
return self.filter(old=False)
class CustomManager(models.Manager):
def get_queryset(self):
return CustomQuerySet(model=self.model, using=self._db)
Is this template code produce one or two queries ?
{% if orders.no_old %}
{% for order orders.no_old %}
...
{% endfor %}
{% endif %}

You can't do any annotations, and there is no need to make .count() since you already have all the data in memory. So its really just between:
orders = Order.objects.filter(user=user)
old_orders = [o for o in orders if o.old]
new_orders = [o for o in orders if not o.old]
#or
old_orders = Order.objects.filter(user=user, old=True)
new_orders = Order.objects.filter(user=user, old=False)
In this specific scenario, I don't think there will be any performance difference. Personally I will choose the 2nd approach with the two queries.
A good read with tips about the problem: Django Database access optimization
Update
About the custom Manager which you introduce. I don't think you are doing it correctly I think what you want is this:
class CustomQueryset(models.QuerySet):
def no_old(self):
return self.filter(old=False)
class Order(models.Model):
name = models.CharField(max_length=100)
# other fields..
user = models.ForeginKey(User)
old = models.BooleanField(default=False)
#if you already have a manager
#objects = CustomManager.from_queryset(CustomQueryset)()
#if you dont:
objects = CustomQueryset.as_manager()
So having:
orders = Order.objects.filter(user=user)
If you do {% if orders.no_old %} will do another query, because this is new QuerySet instance which has no cache..
About the {% regroup %} tag
As you mention, in order to use it, you need to .order_by('old'), and if you have another order, you can still use it, just apply your order after the old, e.g. .order_by('old', 'another_field'). This way you will use only one Query and this will save you one iteration over the list (because Django will split the list iterating it only once), but you will get less readability in the template.

Related

Prefetching model with GenericForeignKey

I have a data structure in which a Document has many Blocks which have exactly one Paragraph or Header. A simplified implementation:
class Document(models.Model):
title = models.CharField()
class Block(models.Model):
document = models.ForeignKey(to=Document)
content_block_type = models.ForeignKey(to=ContentType)
content_block_id = models.CharField()
content_block = GenericForeignKey(
ct_field="content_block_type",
fk_field="content_block_id",
)
class Paragraph(models.Model):
text = models.TextField()
class Header(models.Model):
text = models.TextField()
level = models.SmallPositiveIntegerField()
(Note that there is an actual need for having Paragraph and Header in separate models unlike in the implementation above.)
I use jinja2 to template a Latex file for the document. Templating is slow though as jinja performs a new database query for every Block and Paragraph or Header.
template = get_template(template_name="latex_templates/document.tex", using="tex")
return template.render(context={'script': self.script})
\documentclass[a4paper,10pt]{report}
\begin{document}
{% for block in chapter.block_set.all() %}
{% if block.content_block_type.name == 'header' %}
\section{ {{- block.content_block.latex_text -}} }
{% elif block.content_block_type.name == 'paragraph' %}
{{ block.content_block.latex_text }}
{% endif %}
{% endfor %}
\end{document}
(content_block.latex_text() is a function that converts a HTML string to a Latex string)
Hence I would like to prefetch script.blocks and blocks.content_block. I understand that there are two methods for prefetching in Django:
select_related() performs a JOIN query but only works on ForeignKeys. It would work for script.blocks but not for blocks.content_block.
prefetch_related() works with GenericForeignKeys as well, but if I understand the docs correctly, it can only fetch one ContentType at a time while I have two.
Is there any way to perform the necessary prefetching here? Thank you for your help.
Not really an elegant solution but you can try using reverse generic relations:
from django.contrib.contenttypes.fields import GenericRelation
class Paragraph(models.Model):
text = models.TextField()
blocks = GenericRelation(Block, related_query_name='paragraph')
class Header(models.Model):
text = models.TextField()
level = models.SmallPositiveIntegerField()
blocks = GenericRelation(Block, related_query_name='header')
and prefetch on that:
Document.objects.prefetch_related('block_set__header', 'block_set__paragraph')
then change the template rendering to something like (not tested, will try to test later):
\documentclass[a4paper,10pt]{report}
\begin{document}
{% for block in chapter.block_set.all %}
{% if block.header %}
\section{ {{- block.header.0.latex_text -}} }
{% elif block.paragraph %}
{{ block.paragraph.0.latex_text }}
{% endif %}
{% endfor %}
\end{document}
My bad, I did not notice that document is an FK, and reverse FK can not be joined with select_related.
First of all, I would suggest to add related_name="blocks" anyway.
When you prefetch, you can pass the queryset. But you should not pass filters by doc_id, Django's ORM adds it automatically.
And if you pass the queryset, you can also add select/prefetch related call there.
blocks_qs = Block.objects.all().prefetch_related('content_block')
doc_prefetched = Document.objects.prefetch_related(
Prefetch('blocks', queryset=blocks_qs)
).get(uuid=doc_uuid)
But if you don't need extra filters or annotation, the simpler syntax would probably work for you
document = (
Document.objects
.prefecth_related('blocks', 'blocks__content_block')
.get(uuid=doc_uuid)
)

How to correctly display data from two related models in Django ListView

I have two models as below:
class Loo(models.Model):
loo_type = models.CharField(max_length=3, default="LOO"...)
loo_from = models.ForeignKey(Harr, on_delete=models.CASCADE, ...)
loo_fac = models.DecimalField(max_digits=7, decimal_places=.....)
class Woo(models.Model):
woo_item = models.AutoField(primary_key=True, ...)
woo_loo = models.ForeignKey(Loo, on_delete=models.CASCADE, ...)
woo_dt = models.DateField(null=True, ...)
woo_rate = models.DecimalField(max_digits=7, decimal_places=.....)
I am trying to display data from the models using the following listview:
class WhoLooView(ListView):
template_name = "who_loo_list.html"
context_object_name = 'wholoos'
model = Loo
def get_context_data(self, **kwargs):
context = super(WhoLooView, self).get_context_data(**kwargs)
context.update({
'woo_item_list': Woo.objects.order_by('-woo_dt'),
})
return context
def get_queryset(self):
return Loo.objects.order_by('loo_from')
Note that there can be more than one "woo_item" per instance of Loo (id), so in the listview there will be occasions when for the same Loo id it will have two instances of Woo/s, and thus need to be displayed separately (preferably in two distinct rows).
What I have tried so far creates extra (adjacent) columns for each Loo id and whichever Loo/s have a single instance of Woo, are shown in the normal fashion as expected.
How does one take care of such a situation. Can we have a nested row for cases where there are more than one instance of Woo?
Edit
What I have tried (based on your code sample):
{% for obj in wholoos %} <!-- wholoos : context object name -->
{{ obj.loo_type }}
{% for item in obj.woo_set.all %}
{{ item.woo_dt }}
{% endfor %}
{% endfor %}
But now I am not getting anything from the second model Woo.
Edit 2
I am getting the same result as earlier with my original code. Check the image below:
If you notice (in the image above), objects # 31 and 34 have one each of child objects (sub-objects). # 32 and 33 have two each. I want them to be in separate rows and not columns. The reason is, in case there are a number of items for each parent item (which my DB design makes it imperative to be), I would end up with an enormous number of extra columns for the sub-objects (and that too without any column header).
you can loop the instances of Loo as shown, in your templates, don't have to override the get_context_data.
{% for obj in object_list %}
{{ obj.loo_type }}
{% for item in obj.woo_set.all %}
{{ item.woo_dt }}
{% endfor %}{% endfor %}

Regroups a list of products by order with regroup template tag

I'm having problems using {% regroup %} Django template tag.
A brief summary: I succeed listing all my orders with their products ordered in the same template. So all seems to works fine doing the following:
Create the order
Create products ordered
Assing those products to the order
Display the daily orders with their products in the same template (filtered by date also) in dailyorders.html
Here my codes and only I'll show the code which allows me to display the orders and modify one if I want (where I have the problem)
models:
class Productsordered (models.Model):
item = models.ForeignKey(Itemlist, on_delete=models.CASCADE)
order = models.ForeignKey(Orders, on_delete=models.CASCADE)
quantity = models.IntegerField(default=1)
def __str__(self):
return f"{self.quant} - {self.item.nombre_producto}"
class Orders(models.Model):
date = models.DateField(default=date.today)
client_name = models.CharField(max_length=30) (just name, not client id)
def __str__(self):
return self.client_name
class Itemlist(models.Model):
id = models.SlugField(primary_key = True, max_length =30)
name_item = models.CharField(max_length=30)
price = models.IntegerField()
Just daily orders and modify order views:
def show_daily_orders(request,**kwargs):
daily_products_ordered =
Productsordered.objects.filter(order__date__day=date.today().day,
order__date__month=date.today().month,
order__date__year=date.today().year)
return render(request,'dailyorders.html',{'daily_products_ordered':daily_products_ordered})
def add_products_ordered (request,pk_order,pk_item):
products_ordered=Productsordered.objects.filter(order=pk_order)
get_order=Orders.objects.get(id=pk_pedido)
list= Itemlist.objects.all()
#ask if the product is already there, if not create one
try:
product = products_ordered.get(item=pk_item)
product.quantity += 1
product.save()
except:
newproduct = Itemlist.objects.get(id=pk_item)
newproduct = Productsordered.objects.create(item=newproduct, order=get_order)
product.save()
return render(request,'modify.html',{'list':list,'products_ordered':products_ordered,'order':order})
dailyorders.html would be something like:
{% regroup daily_products_ordered by order as order_list %}
{% for order in order_list %}
<h2> {{order.grouper}}
{% for product in order.list %}
<li> {{product.quantity}} - {{ product.item.name_item }}</li>
<a class = 'btn' href="{% url 'go_to_modify_order' pk=order.grouper.id %}">Modify order</a>
{% endfor %}
{% endfor %}
modify.html
{% for prod in list %}
<a class="btn" href="{% url 'go_to_modify_order' pk_item=prod.id pk_order=order.id %}">{{prod.name_item}}</a>
{% endfor %}
<a class="btn" href="{% url 'go_to_orders' pk_item=prod.id pk_pedido=order.id %}"> Finish modifying order </a>
# Also there would be the choice to delete an added product, but no matter for the problem.
The problem is that when I try to modify an order and add a new product (simply clicking the buttons and adding them to the cart) when I return to the template where I get all orders, regroup counts this new product as if it were from another order. For example:
Laura (order_id=1):
Product A
Product B
Carlos (order_id=2):
Product X
Laura (order_id=1):
Product C (this would be the product added when modify order!)
In order to see what happened, I went to the admin section and noticed that this new product is even added to the order I modified (Product C belonging to order_id=1 for example). So, it seems to be a problem when passing from the modify template to the daily orders, or something else I haven't noticed yet.
I'd appreciate help in this topic.
Thank you all
I've already done!
Seems to be that regroup only works when you have your dictionary order by the attribute you want to regroup! That's because I was getting the orders separately (see the example Laura (order_id=1) twice! see django documentation about regroup built-in template

How to display related content of an object

I have a Contact class that can have many Conversations. But each conversation can belong to a single Contact.
So its a One-To-Many relationship.
class Contact(models.Model):
first_name = models.CharField()
last_name = models.CharField()
class Conversation(models.Model):
contact = models.ForeignKey(Contact)
notes = models.TextField()
def __unicode__(self):
return self.notes
Now when I pass in the contacts to the template, I would like to have one field that shows the last conversation for the contact.
contacts= profile.company.contact_set.all()[:10]
calls = []
for c in contacts:
calls.append(max(c.conversation_set.all()))
And I pass them in like this:
vars = {'contacts' : contacts, 'calls': calls}
variables = RequestContext(request, vars)
return render_to_response('main_page.html', variables)
In the template I came up with this magic:
{% for idx, val in contacts %}
<tr>
<td>...
</td>
<td>
{% if calls %}
{{ calls.idx }}
{% endif %}</td>
</tr>
{% endfor %}
This doesn't show anything for calls. But if I replaced calls.idx with calls.0 I get the first one displayed.
What am I doing wrong? Beside the fact that it could be done probably much easier than that. :) I am open for better ways of doing it.
You can't do this sort of indirect lookup in the template language - calls.idx will always refer to an attribute idx belonging to calls.
However I think a better solution is to add the call value directly onto the contact object - don't forget that in Python objects are dynamic, so you can annotate anything you like onto them.
for c in contacts:
c.max_call = max(c.conversation_set.all())
As an aside, does that max really work? I'm not sure what it would be doing the comparison based on. An alternative is to define get_latest_by in your Conversation model, then you can avoid this loop altogether and just call {{ contact.conversation_set.get_latest }} in your template for each contact through the loop.
(Note this will still be pretty inefficient, there are various ways of getting the whole set of max conversations in one go, but it will do for now.)
class Contact(models.Model):
# your code here
#property
def last_conversation(self):
try:
return self.conversation_set.order_by("-some_date_or_order_field")[0]
except IndexError:
return None
Then you don't have to care about this in your view and just need to call "c.last_contact" in your template.

Iterating over results, sorted by a m2m

My data model is simple:
class Neighborhood(models.Model):
name = models.CharField(max_length = 50)
slug = models.SlugField()
class Location(models.Model):
company = models.ForeignKey(Company)
alt_name = models.CharField()
neighborhoods = models.ManyToManyField(Neighborhood)
I would like to supply a page on my site that lists all locations by their neighborhood (s). If it were singular, I think {% regroup %} with a {% ifchanged %} applied to the neighborhood name would be all that I need, but in my case, having it be a m2m, I'm not sure how do this. A location may have multiple neighborhoods, and so I would like them to be redundantly displayed under each matching neighborhood.
I'm also aware of FOO_set but that's per Object; I want to load the entire data set.
The final result (in the template) should be something like:
Alameda
Crazy Thai
Castro
Kellys Burgers
Pizza Orgasmica
Filmore
Kellys Burgers
Some Jazz Bar
Mission
Crazy Thai
Elixir
...
The template syntax would (ideally?) look something like:
{% for neighborhood in neighborhood_list %}
{% ifchanged %}{{ neighborhood.name }}{% endifchanged %}
{% for location in neighborhood.restaurants.all %}
{{ location.name }}
{% endfor %}
{% endfor %}
I'd just do it the expensive way and cache the result over scratching my head. Your template example would work fine until performance becomes an issue in generating that one page per X cache timeout.
You could do it in python as well if the result set is small enough:
# untested code - don't have your models
from collections import defaultdict
results = defaultdict(list)
for location_m2m in Location.neighborhoods.through.objects.all() \ # line wrap
.select_related('neighborhood', 'location', 'location__company__name'):
if location_m2m.location not in results[location_m2m.neighborhood]:
results[location_m2m.neighborhood].append(location_m2m.location)
# sort now unique locations per neighborhood
map(lambda x: x.sort(key=lambda x: x.location.company.name), results.values())
# sort neighborhoods by name
sorted_results = sorted(results.items(), key=lambda x:x[0].name)
create another model that would define your m2m relationship:
class ThroughModel(models.Model):
location = models.ForeignKey('Location')
neighborhood = models.ForeignKey('Neighborhood')
and use it as the through model for your m2m field:
class Location(models.Model):
...
neighborhoods = models.ManyToManyField(Neighborhood, through='ThroughModel')
Then you can get all your neighbourhoods sorted:
ThroughModel.objects.select_related().order_by('neighborhood__name')
This is untested.
Or, if you cannot change the database, just do a raw SQL join.