I know I've seen this before, but I can't find now that I am ready to implement it.
I'm trying to list items on a page by category and subcategory, but I only want 1 category and then the subcategories in the category. Hopefully my code will make sense.
class Objects(models.Model):
# Main Checkbox.
category = models.CharField(
max_length=100,
blank=True,
)
# Checkboxes under Main Checkbox.
subcategory = models.CharField(
max_length=100,
blank=True,
)
So my objects are stored as:
category1, subcategory1
category1, subcategory2
category2, subcategory1
category2, subcategory2
And when displayed should give me:
category1
subcategory1
subcategory2
category2
subcategory1
subcategory2
How do I set my query so that my results show me each "different" category?
First off you should look to normalise your objects, meaning you could keep your subcategories as a separate object and use models.foreignKey to link them together. See Willem Van Onsem's answer for this.
However, to solve the current problem you should be able to utilize the regroup template tag in order to aggregate the categories.
In your case it would look something like:
{% regroup objects by category as categories %}
{% for category in categories %}
{{ category.grouper }}
{% for subcategory in category.list %}
{{ subcategory }}
{% endfor %}
{% endfor %}
The data duplication anti-pattern
Many computer scientists see this as bad design, since it introduces data duplication. Imagine that you later want to change the name of a category, then that means you need to find all occurrences of that category, and rename them. If you only use this for the Objects model, then that is perhaps doable, but if all sorts of elements belong to Categorys, then this easily gets out of hand.
Furthermore it also restricts categories: two different categories can never have the same name (which might here be reasonable), nor can we attach much properties to the Category: imagine that we want to add a description for the category, then that description needs to be repeated over all rows, or if we decide to store it only in one row, then it will be hard to find that specific row. Furthermore if there are two rows with different descriptions, then what description to pick?
The database will also be very huge: each row should repeat the same category. If a category on average takes 15 characters, that means that we will - depending on the encoding - easily waste 8 bytes per row (well the row contains 16 bytes for a string given it is UTF-8 encoding, and only ASCII characters, but a ForeignKey will frequently use 8 bytes). If we would add a description that has on average 63 characters, then we would again waste another 64 bytes per row. For a small amount of rows, that is not a problem, but the problem easily scales problematic. The above are of course only estimates on what problems might arise, do not see this as the "real numbers": the size a database takes is determined by a lot of parameters that are here either ignored, or estimated.
Yes all these problems probably can be solved, but instead of solving the problems ad-hoc, it is better to normalize the database.
Normalizing the models
Normalization typically means that we introduce extra tables that store for example one record per Category, and use ForeignKeys to refer to that record. For you example a normalized variant would be:
class Category(models.Model):
name = models.CharField(max_length=100)
class SubCategory(models.Model):
name = models.CharField(max_length=100)
category = models.ForeignKey(Category, on_delete=models.CASCADE)
class Object(models.Model):
subcategory = models.ForeignKey(
SubCategory,
null=True,
on_delete=models.SET_NULL
)
So we store Categorys and SubCategorys in dedicated tables, and link the models together with ForeignKeys.
Rendering lists of (Sub)Categorys
Now that we normalized the models, we can effectively render the Categorys with:
# app/views.py
def some_view(request):
categories = Category.objects.prefetch_related('subcategory_set')
return render(request, 'app/some_template.html', {'categories': categories})
and in the app/templates/some_template.html we then can render it as:
<ul>
{% for cat in categories %}
<li>{{ cat.name }}</li>
<ul>
{% for subcat in cat.subcategory_set %}
<li>{{ subcat.name }}</li>
{% endfor %}
</ul>
{% endfor %}
</ul>
We thus iterate over all categories, and for every cat, we iterate over the subcategory_set.
Related
class Order(models.Model):
name = models.CharField(max_length=100)
# other fields..
user = models.ForeginKey(User)
old = models.BooleanField(default=False)
I want to display all the orders of a specific user, but I want to split them those which are "old" and the ones who are not.
So, currently I do in views.py:
orders = Order.objects.filter(user=user)
In template:
First table:
<table>
{% for order in orders %}
{% if not order.old %}
<tr>
<td>... </td>
</tr>
{% endif %}
{% endfor %}
</table>
And another table:
{% for order in orders %}
{% if order.old %}
<tr>
<td>...</td>
<tr>
{% endif %}
{% endfor %}
This way have some drawbacks, first, now I want to count how many of the orders are "old", to display this number in the template. I can't, unless I do another query.
Is it possible to annotate(number_of_old=Count('old'))? Or I have to do another query?
So what would be the best?
1. Do two queries, one with old=False, another with old=True, and pass two querysets to the template. And use |len filter on the querysets
2. Do one query like this and split them somehow in python? That will be less convenient as I have a similar structures which I want to split like that.
And should I call the DB .count() at all?
EDIT:
If I would write my model like this:
class Order(models.Model):
name = models.CharField(max_length=100)
# other fields..
user = models.ForeginKey(User)
old = models.BooleanField(default=False)
objects = CustomManager() # Custom manager
class CustomQueryset(models.QuerySet):
def no_old(self):
return self.filter(old=False)
class CustomManager(models.Manager):
def get_queryset(self):
return CustomQuerySet(model=self.model, using=self._db)
Is this template code produce one or two queries ?
{% if orders.no_old %}
{% for order orders.no_old %}
...
{% endfor %}
{% endif %}
You can't do any annotations, and there is no need to make .count() since you already have all the data in memory. So its really just between:
orders = Order.objects.filter(user=user)
old_orders = [o for o in orders if o.old]
new_orders = [o for o in orders if not o.old]
#or
old_orders = Order.objects.filter(user=user, old=True)
new_orders = Order.objects.filter(user=user, old=False)
In this specific scenario, I don't think there will be any performance difference. Personally I will choose the 2nd approach with the two queries.
A good read with tips about the problem: Django Database access optimization
Update
About the custom Manager which you introduce. I don't think you are doing it correctly I think what you want is this:
class CustomQueryset(models.QuerySet):
def no_old(self):
return self.filter(old=False)
class Order(models.Model):
name = models.CharField(max_length=100)
# other fields..
user = models.ForeginKey(User)
old = models.BooleanField(default=False)
#if you already have a manager
#objects = CustomManager.from_queryset(CustomQueryset)()
#if you dont:
objects = CustomQueryset.as_manager()
So having:
orders = Order.objects.filter(user=user)
If you do {% if orders.no_old %} will do another query, because this is new QuerySet instance which has no cache..
About the {% regroup %} tag
As you mention, in order to use it, you need to .order_by('old'), and if you have another order, you can still use it, just apply your order after the old, e.g. .order_by('old', 'another_field'). This way you will use only one Query and this will save you one iteration over the list (because Django will split the list iterating it only once), but you will get less readability in the template.
models.py:
class InjuredLocation(models.Model):
reportperson = models.ForeignKey(ReportPerson)
mark1 = models.BooleanField('Mark1', default=False)
mark2 = models.BooleanField('Mark2', default=False)
mark3 = models.BooleanField('Mark3', default=False)
class Report(models.Model):
user = models.ForeignKey(User, null=False)
report_number = models.CharField('report Number', max_length=100)
class ReportPerson(models.Model):
report = models.ForeignKey(Report)
action_type = models.CharField(max_length=100, choices=ACTION_TYPE)
name = models.CharField('Name', max_length=100)
This is my three models, I want to filter the data from InjuredLocation models.
Reportperson table contain report id and name field of that table could be multiple.Each report can have multiple names.
I want to filter the data from InjuredLocation table with reference to reportperson_id.
The filtered data should be for the equivalent report.
tried:
injury_list = []
reportperson = ReportPerson.objects.filter(report=report_id, action_type="involved")
injuary_mark = InjuredLocation.objects.filter(pk=reportperson)
for injuary in injuary_mark:
mark = InjuredLocation.objects.get(pk=injuary.id)
marklist={'mark':mark}
injury_list.append(marklist)
I am getting this error "(1242, 'Subquery returns more than 1 row')" in 5th line,if Reportperson table have more than one name.
update:
injuery_list = []
injuries = InjuredLocation.objects.filter(reportperson__report=report_id, reportperson__action_type="involved")
for reportperson in injuries:
injun = InjuredLocation.objects.get(pk=reportperson.id)
list_inju = {'person': injun}
injuery_list.append(list_inju)
Able to take the objects from InjuredLocation models,in template i rendered it but problem is "it should render against reportperson_id,instead it is rendering all" for example if InjuredLocation models have reportperson_id=1,mark1=0 & mark2=1 and for reportperson_id=2,mark1=1 & mark2=0 it is rendering all like this "1 1" for both reportperson_id.The expected output is 0 1 and 1 0.What ever selected are all comes to display for all reportperson_id.
template is
{%for injuary_mark in injuery_list%}
{%if injuary_mark.person.mark1 %}<img style="float: right; margin:5px 4px -35px 0;" src="{{BASE_URL}}/static/images/red-cross.png"> {% endif %}
{%if injuary_mark.person.mark2 %}<img style="float: right;margin:5px 8px -35px -8px;" src="{{BASE_URL}}/static/images/red-cross.png"> {% endif %}
{%endfor%}
Last update:
I want to show the details in InjuredLocation models against id in the Reportperson models.This is from a single report,see the Report models in the figure.
All the three models with data i pasted below.
What i required as output is,a tab will be dynamically created when a row in InjuredLocation model is created against id in Reportperson table.I want to show the mark from InjuredLocation table against their respective id from Reportperson table in their respective tabs.Now all the mark whatever created against id in Reportperson model are shown in all tabs.Assume mark for id=1 is empty and Marks for id=2 and id=3 are their in database,as per requirement tab1 should not show any data,but now tab1 is showing data of tab2 and tab3 from id=2 and id=3's data.Need help
Your problem is with this line:
injuary_mark = InjuredLocation.objects.filter(pk=reportperson)
The exception refers to a different line because that's where the queryset is actually evaluated.
There are two problems here.
The fatal one is that reportperson is not a single value - it's a queryset:
reportperson = ReportPerson.objects.filter(report=report_id, action_type="involved")
As you note, 'each report can have multiple names' - and this will find all of them that match the action_type, so it's not an appropriate value to use in an exact lookup.
Also, you almost certainly do not mean pk=reportperson - even if reportperson were a single value, you're filtering on the wrong field.
The fix is somewhat dependent on what exactly you want to do with the multiple names. If you just want to get all the InjuredLocation instances that relate to the report_id regardless of report name, this is a more concise expression:
injuries = InjuredLocation.objects.filter(reportperson__report_id=report_id, reportperson__action_type="involved")
If necessary you could use your original reportperson lookup and then an __in filter, but the version above using __ to filter on related values is more concise. In the database, __in uses a subquery while filtering using __ performs a join; the two can have different performance. The __in version would be:
reportpeople = ReportPerson.objects.filter(report=report_id, action_type="involved")
injuries = InjuredLocation.objects.filter(reportperson__in=reportpeople)
If you want to keep each InjuredLocation instance with its ReportPerson instance, say because you're going to group them in a template:
reportpeople = ReportPerson.objects.filter(report_id=report_id, action_type="involved")
for reportperson in reportpeople:
injuries = reportperson.injuredlocation_set.all()
# now do something with them
Edited:
if you given me a sample how to make the queryset and how to iterate in template will be a great help for me
Something like:
In the view:
reportpeople = ReportPerson.objects.filter(report_id=report_id, action_type="involved")
return render('mytemplate.html', {'reportpeople': reportpeople})
In the template:
{% for reportperson in reportpeople %}
<p>Report name: {{ reportperson.name }}</p>
<ul>
{% for injured_location in reportperson.injuredlocation_set.all %}
<li>
{% if injured_location.mark1 %}
<img style="float: right; margin:5px 4px -35px 0;" src="{{BASE_URL}}/static/images/red-cross.png"> {% endif %}
{% if injured_location.mark2 %}
<img style="float: right;margin:5px 8px -35px -8px;" src="{{BASE_URL}}/static/images/red-cross.png"> {% endif %}
</li>
{% endfor %}
</ul>
{% endfor %}
Or whatever HTML you want for each report name. The point is that you can get at the InjuredLocation instances related to a particular ReportPerson instance via the injuredlocation_set manager.
I have a Contact class that can have many Conversations. But each conversation can belong to a single Contact.
So its a One-To-Many relationship.
class Contact(models.Model):
first_name = models.CharField()
last_name = models.CharField()
class Conversation(models.Model):
contact = models.ForeignKey(Contact)
notes = models.TextField()
def __unicode__(self):
return self.notes
Now when I pass in the contacts to the template, I would like to have one field that shows the last conversation for the contact.
contacts= profile.company.contact_set.all()[:10]
calls = []
for c in contacts:
calls.append(max(c.conversation_set.all()))
And I pass them in like this:
vars = {'contacts' : contacts, 'calls': calls}
variables = RequestContext(request, vars)
return render_to_response('main_page.html', variables)
In the template I came up with this magic:
{% for idx, val in contacts %}
<tr>
<td>...
</td>
<td>
{% if calls %}
{{ calls.idx }}
{% endif %}</td>
</tr>
{% endfor %}
This doesn't show anything for calls. But if I replaced calls.idx with calls.0 I get the first one displayed.
What am I doing wrong? Beside the fact that it could be done probably much easier than that. :) I am open for better ways of doing it.
You can't do this sort of indirect lookup in the template language - calls.idx will always refer to an attribute idx belonging to calls.
However I think a better solution is to add the call value directly onto the contact object - don't forget that in Python objects are dynamic, so you can annotate anything you like onto them.
for c in contacts:
c.max_call = max(c.conversation_set.all())
As an aside, does that max really work? I'm not sure what it would be doing the comparison based on. An alternative is to define get_latest_by in your Conversation model, then you can avoid this loop altogether and just call {{ contact.conversation_set.get_latest }} in your template for each contact through the loop.
(Note this will still be pretty inefficient, there are various ways of getting the whole set of max conversations in one go, but it will do for now.)
class Contact(models.Model):
# your code here
#property
def last_conversation(self):
try:
return self.conversation_set.order_by("-some_date_or_order_field")[0]
except IndexError:
return None
Then you don't have to care about this in your view and just need to call "c.last_contact" in your template.
My data model is simple:
class Neighborhood(models.Model):
name = models.CharField(max_length = 50)
slug = models.SlugField()
class Location(models.Model):
company = models.ForeignKey(Company)
alt_name = models.CharField()
neighborhoods = models.ManyToManyField(Neighborhood)
I would like to supply a page on my site that lists all locations by their neighborhood (s). If it were singular, I think {% regroup %} with a {% ifchanged %} applied to the neighborhood name would be all that I need, but in my case, having it be a m2m, I'm not sure how do this. A location may have multiple neighborhoods, and so I would like them to be redundantly displayed under each matching neighborhood.
I'm also aware of FOO_set but that's per Object; I want to load the entire data set.
The final result (in the template) should be something like:
Alameda
Crazy Thai
Castro
Kellys Burgers
Pizza Orgasmica
Filmore
Kellys Burgers
Some Jazz Bar
Mission
Crazy Thai
Elixir
...
The template syntax would (ideally?) look something like:
{% for neighborhood in neighborhood_list %}
{% ifchanged %}{{ neighborhood.name }}{% endifchanged %}
{% for location in neighborhood.restaurants.all %}
{{ location.name }}
{% endfor %}
{% endfor %}
I'd just do it the expensive way and cache the result over scratching my head. Your template example would work fine until performance becomes an issue in generating that one page per X cache timeout.
You could do it in python as well if the result set is small enough:
# untested code - don't have your models
from collections import defaultdict
results = defaultdict(list)
for location_m2m in Location.neighborhoods.through.objects.all() \ # line wrap
.select_related('neighborhood', 'location', 'location__company__name'):
if location_m2m.location not in results[location_m2m.neighborhood]:
results[location_m2m.neighborhood].append(location_m2m.location)
# sort now unique locations per neighborhood
map(lambda x: x.sort(key=lambda x: x.location.company.name), results.values())
# sort neighborhoods by name
sorted_results = sorted(results.items(), key=lambda x:x[0].name)
create another model that would define your m2m relationship:
class ThroughModel(models.Model):
location = models.ForeignKey('Location')
neighborhood = models.ForeignKey('Neighborhood')
and use it as the through model for your m2m field:
class Location(models.Model):
...
neighborhoods = models.ManyToManyField(Neighborhood, through='ThroughModel')
Then you can get all your neighbourhoods sorted:
ThroughModel.objects.select_related().order_by('neighborhood__name')
This is untested.
Or, if you cannot change the database, just do a raw SQL join.
I have setup like so (changed for simplicity)
class Author(models.Model)
name = models.CharField(max_length=100)
...
class Document(models.Model):
title = models.CharField(max_length=200)
content - models.TextField()
author = models.ForeignKey("Author", related_name="documents")
date_published = models.DateTimeField()
categories = models.ManyToManyField("Category")
class Category(models.Model):
name = models.CharField(max_length=100)
I'm pulling in the Author records but I only want to pull in related document records for each author that match specific criteria -- say, date_published and category.
I know the easy way to do this would be to pull in the records as a list of dictionaries using Author.objects.values(), looping through each record and running:
author['documents']=Document.objects.filter(categories__in=[category_list], date_published__year=year)`
However, this is being generated for django-piston, and it seems to be happiest (particularly if you're defining your own fields!) if you return a QuerySet object.
Part of this may be because I made changes to the base django-piston code. Basically, the current version of the code here overwrites the fields value. I changed this code so that I could change the fields value for a Handler based on the request (so I could provide more details if the request was for a specific resource).
So I guess my question is three-fold:
Is there a way to filter or somehow limit the subrecords of a record (i.e. filter documents for each author.documents)
If not, what is a functional way of doing this that also works with django-piston?
Is there some easier, better way to do what I'm trying to do (display all the authors without their documents if an id is not given, but displaying the sub-records if filtering to just one author)?
Clarification
Okay, just to be clear, here is the pseudocode that I want:
def perhaps_impossible_view(request, categories=None, year=None):
authors = Author.objects.all()
authors.something_magical_happens_to_documents(category__in=[categories], date_published__year=year)
return render_to_response('blar.html', locals(), RequestContext(request))
So that if I were to put it in a template, this would work without any modifications:
{% for a in authors %}
{% for d in authors.documents.all %}
{{ d.title }} is almost certainly in one of these categories: {{ categories }} and was absolutely published in {{ year }}. If not, .something_magical_happens_to_documents didn't work.
{% endfor %}
{% endfor %}
something_magical_happens_to_documents has to run and actually change the contents of documents for each author record. It seems like this should be possible, but perhaps it isn't?
Edited... or better... replaced! :)
That's true the authors without document matching won't be in the queryset so you will have to add them back after (I couldn't find a better way but maybe someone knows how to not remove them without using raw sql).
You get the full documents count of the authors because you don't use the queryset to get the document counts:
qryset = Author.objects.all()
qryset = qryset.filter(documents__categories__name__in = category_list).distinct()
qryset = qryset.filter(documents__date_published__year=year)
print(qryset) gives [<Author: Author object>] (if only 1 author matched all categories) and qryset[0].documents.count() will return only the number of documents matched (not all documents from the author - 2 in the case I tested and the author had 4 but only 2 matching all conditions).
If you use dict (.values()) instead of querysets in the steps above, you can't do that (I think) because dict won't have a documents field so:
qryset_dict = Author.objects.values()
qryset_dict = qryset_dict.filter(documents__categories__name__in = category_list).distinct().values()
qryset_dict = qryset_dict.filter(documents__date_published__year=year).values()
when you issue qryset_dict[0].documents.count() you receive an error:
AttributeError: 'dict' object has no attribute 'documents'
Now to add the filtered authors back you can do:
res = []
for a in Author.objects.all():
if a in qryset:
res.append([a,a.documents.count()])
else:
res.append([a,0])
and res will be a list with <authors> in 1st column and count of documents matching in 2nd column.
I know this is far from perfect... but if you are interested only in the count() of matching documents per author, I think you could find a better way using django aggregation and annotation or possibly make it with raw sql using a left join from authors to documents.
EDIT after Clarification in Question:
def possible_view(request, categories=None, year=None):
# you will pass these as parmeters of course
category_list = ['c2', 'c3']
year = 2010
qryset = Document.objects.filter(categories__name__in = category_list).distinct()
qryset = qryset.filter(date_published__year=year)
authors = Author.objects.all()
return render_to_response('blar.html', { 'result': qryset, 'authors': authors, 'categories': category_list, 'year': year }, RequestContext(request))
Template blar.html:
{% for a in authors %}
<b>{{a.name}}</b><br />
{% for d in result %}
{% if d.author.name == a.name %}
{{ d.title }} is almost certainly in one of these categories: {{ categories }} and was absolutely published in {{ year }}. If not, .something_magical_happens_to_documents didn't work.<br />
{% endif %}
{% endfor %}<br />
{% endfor %}
This will give you something not very pretty but with all authors and below each one, the list of their documents that fall within one of the category_list (OR condition, for AND, you need to filter the query for each category instead of using __in).
If the author has no document in the category_list, it wil be listed without documents below him.
Something like:
aut1
tit2 is almost certainly in one of these categories: ['c2', 'c3'] and was absolutely published in 2010. If not, .something_magical_happens_to_documents didn't work.
tit1 is almost certainly in one of these categories: ['c2', 'c3'] and was absolutely published in 2010. If not, .something_magical_happens_to_documents didn't work.
aut2
tit3 is almost certainly in one of these categories: ['c2', 'c3'] and was absolutely published in 2010. If not, .something_magical_happens_to_documents didn't work.
aut3