I have multiple related models and on a single page I need to access data from different models on a variety of conditions. Most of this data is accessed without a form as only a few fields need to be manipulated by the user.
To give you an idea of how interconnected my Models are:
class User(models.Model):
first = models.ManyToManyField(First)
second= models.ManyToManyField(Second)
class First(models.Model):
[...]
class Second(models.Model):
first = models.ManyToManyField(First)
class Third(models.Model):
first = models.ForeignKey(First)
class Fourth(models.Model):
user = models.ForeignKey(User)
second = models.ForeignKey(Second)
third = models.ForeignKey(Third)
class Fifth(models.Model):
fourth = models.ForeignKey(Fourth)
class Sixth(models.Model):
fifth = models.ForeignKey(Fifth)
I'm having a lot of difficulty passing this data to my template and displaying it in efficiently. I originally displayed it to the user in a horrendous way as I just wanted to see if my data was accessible/displaying properly. This is an example of some of the dodgy code I used to test that my data was being passed properly:
{% for second in user.second.all %}
{% for first in second.first.all %}
[...]
{% for fourth in user.fourth.all %}
[...]
{% if fourth.first_id == first.id and fourth.second_id == second.id %}
{% if fourth.checked == True %}
[...]
{% else %}
[...]
{% endif %}
{% endif %}
{% endfor %}
{% endfor %}
{% endfor %}
[...]
Isn't it an abomination, and it goes deeper. Now I am trying to reformat this. I know that my processing should be done within my view rather than my template. The thing is I can't just filter out a lot of the data in my views as I need a lot of it in the template at different times (eg I might have 20 pieces of data in a model and I need to access all 20, but some when the id match, some when the id and type match etc. It's a lot of template side logic - I think that's bad).
This is the kind of thing I have been trying to do so far but am having no luck:
second = Second.objects.filter(user__id=user.id) .select_related('First')
Any help with how to join/pass all these models to my view and access the data without the nested nested loops, or even just pointers on how to approach this would be appreciated. I's also unsure if I should be aiming to join as many models as possible then pass into my view or pass many separate models.
Thank you.
you are serializer the data use a drf serializer you will need to use a nested serializer
example for nested serializer
class HospitalsSerializer(serializers.ModelSerializer):
class Meta:
model = Hospitals
fields = '__all__'
class HealthProductSerializers(serializers.ModelSerializer):
hospital_list = HospitalsSerializer(many=True)
class Meta:
model = HealthProduct
exclude = ("created_at", "updated_at")
Related
I have a data structure in which a Document has many Blocks which have exactly one Paragraph or Header. A simplified implementation:
class Document(models.Model):
title = models.CharField()
class Block(models.Model):
document = models.ForeignKey(to=Document)
content_block_type = models.ForeignKey(to=ContentType)
content_block_id = models.CharField()
content_block = GenericForeignKey(
ct_field="content_block_type",
fk_field="content_block_id",
)
class Paragraph(models.Model):
text = models.TextField()
class Header(models.Model):
text = models.TextField()
level = models.SmallPositiveIntegerField()
(Note that there is an actual need for having Paragraph and Header in separate models unlike in the implementation above.)
I use jinja2 to template a Latex file for the document. Templating is slow though as jinja performs a new database query for every Block and Paragraph or Header.
template = get_template(template_name="latex_templates/document.tex", using="tex")
return template.render(context={'script': self.script})
\documentclass[a4paper,10pt]{report}
\begin{document}
{% for block in chapter.block_set.all() %}
{% if block.content_block_type.name == 'header' %}
\section{ {{- block.content_block.latex_text -}} }
{% elif block.content_block_type.name == 'paragraph' %}
{{ block.content_block.latex_text }}
{% endif %}
{% endfor %}
\end{document}
(content_block.latex_text() is a function that converts a HTML string to a Latex string)
Hence I would like to prefetch script.blocks and blocks.content_block. I understand that there are two methods for prefetching in Django:
select_related() performs a JOIN query but only works on ForeignKeys. It would work for script.blocks but not for blocks.content_block.
prefetch_related() works with GenericForeignKeys as well, but if I understand the docs correctly, it can only fetch one ContentType at a time while I have two.
Is there any way to perform the necessary prefetching here? Thank you for your help.
Not really an elegant solution but you can try using reverse generic relations:
from django.contrib.contenttypes.fields import GenericRelation
class Paragraph(models.Model):
text = models.TextField()
blocks = GenericRelation(Block, related_query_name='paragraph')
class Header(models.Model):
text = models.TextField()
level = models.SmallPositiveIntegerField()
blocks = GenericRelation(Block, related_query_name='header')
and prefetch on that:
Document.objects.prefetch_related('block_set__header', 'block_set__paragraph')
then change the template rendering to something like (not tested, will try to test later):
\documentclass[a4paper,10pt]{report}
\begin{document}
{% for block in chapter.block_set.all %}
{% if block.header %}
\section{ {{- block.header.0.latex_text -}} }
{% elif block.paragraph %}
{{ block.paragraph.0.latex_text }}
{% endif %}
{% endfor %}
\end{document}
My bad, I did not notice that document is an FK, and reverse FK can not be joined with select_related.
First of all, I would suggest to add related_name="blocks" anyway.
When you prefetch, you can pass the queryset. But you should not pass filters by doc_id, Django's ORM adds it automatically.
And if you pass the queryset, you can also add select/prefetch related call there.
blocks_qs = Block.objects.all().prefetch_related('content_block')
doc_prefetched = Document.objects.prefetch_related(
Prefetch('blocks', queryset=blocks_qs)
).get(uuid=doc_uuid)
But if you don't need extra filters or annotation, the simpler syntax would probably work for you
document = (
Document.objects
.prefecth_related('blocks', 'blocks__content_block')
.get(uuid=doc_uuid)
)
Using a GenericRelation to map Records to Persons, I have the following code, which works correctly, but there is a performance problem I am trying to address:
models.py
class RecordX(Record): # Child class
....
class Record(models.Model): # Parent class
people = GenericRelation(PersonRecordMapping)
def people_all(self):
# Use select_related here to minimize the # of queries later
return self.people.select_related('person').all()
class Meta:
abstract = True
class PersonRecordMapping(models.Model):
person = models.ForeignKey(Person)
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
content_object = GenericForeignKey('content_type', 'object_id')
class Person(models.Model):
...
In summary I have:
RecordX ===GenericRelation===> PersonRecordMapping ===ForeignKey===> Person
The display logic in my application follows these relationships to display all the Persons mapped to each RecordX:
views.py
#rendered_with('template.html')
def show_all_recordXs(request):
recordXs = RecordX.objects.all()
return { 'recordXs': recordXs }
The problem comes in the template:
template.html
{% for recordX in recordXs %}
{# Display RecordX's other fields here #}
<ul>
{% for map in record.people_all %}{# <=== This generates a query every time! #}
<li>{{ map.person.name }}</li>
{% endfor %}
</ul>
{% endfor %}
As indicated, every time I request the Persons related to the RecordX, it generates a new query. I cannot seem to figure out how prefetch these initially to avoid the redundant queries.
If I try selected_related, I get an error that no selected_related fields are possible here (error message: Invalid field name(s) given in select_related: 'xxx'. Choices are: (none)). Not surprising, I now see -- there aren't any FKs on this model.
If I try prefetch_related('people'), no error is thrown, but I still get the same repeated queries on every loop in the template as before. Likewise for prefetch_related('people__person'). If I try prefetch_related('personrecordmapping'), I get an error.
Along the lines of this answer, I considered doing trying to simulate a select_related via something like:
PersonRecordMapping.objects.filter(object_id__in=RecordX.objects.values_list('id', flat=True))
but I don't understand the _content_object_cache well enough to adapt this answer to my situation (if even the right approach).
So -- how can I pre-fetch all the Persons to the RecordXs to avoid n queries when the page will display n RecordX objects?
Thanks for your help!
Ack, apparently I needed to call both -- prefetch_related('people', 'people__person'). SIGH.
class Order(models.Model):
name = models.CharField(max_length=100)
# other fields..
user = models.ForeginKey(User)
old = models.BooleanField(default=False)
I want to display all the orders of a specific user, but I want to split them those which are "old" and the ones who are not.
So, currently I do in views.py:
orders = Order.objects.filter(user=user)
In template:
First table:
<table>
{% for order in orders %}
{% if not order.old %}
<tr>
<td>... </td>
</tr>
{% endif %}
{% endfor %}
</table>
And another table:
{% for order in orders %}
{% if order.old %}
<tr>
<td>...</td>
<tr>
{% endif %}
{% endfor %}
This way have some drawbacks, first, now I want to count how many of the orders are "old", to display this number in the template. I can't, unless I do another query.
Is it possible to annotate(number_of_old=Count('old'))? Or I have to do another query?
So what would be the best?
1. Do two queries, one with old=False, another with old=True, and pass two querysets to the template. And use |len filter on the querysets
2. Do one query like this and split them somehow in python? That will be less convenient as I have a similar structures which I want to split like that.
And should I call the DB .count() at all?
EDIT:
If I would write my model like this:
class Order(models.Model):
name = models.CharField(max_length=100)
# other fields..
user = models.ForeginKey(User)
old = models.BooleanField(default=False)
objects = CustomManager() # Custom manager
class CustomQueryset(models.QuerySet):
def no_old(self):
return self.filter(old=False)
class CustomManager(models.Manager):
def get_queryset(self):
return CustomQuerySet(model=self.model, using=self._db)
Is this template code produce one or two queries ?
{% if orders.no_old %}
{% for order orders.no_old %}
...
{% endfor %}
{% endif %}
You can't do any annotations, and there is no need to make .count() since you already have all the data in memory. So its really just between:
orders = Order.objects.filter(user=user)
old_orders = [o for o in orders if o.old]
new_orders = [o for o in orders if not o.old]
#or
old_orders = Order.objects.filter(user=user, old=True)
new_orders = Order.objects.filter(user=user, old=False)
In this specific scenario, I don't think there will be any performance difference. Personally I will choose the 2nd approach with the two queries.
A good read with tips about the problem: Django Database access optimization
Update
About the custom Manager which you introduce. I don't think you are doing it correctly I think what you want is this:
class CustomQueryset(models.QuerySet):
def no_old(self):
return self.filter(old=False)
class Order(models.Model):
name = models.CharField(max_length=100)
# other fields..
user = models.ForeginKey(User)
old = models.BooleanField(default=False)
#if you already have a manager
#objects = CustomManager.from_queryset(CustomQueryset)()
#if you dont:
objects = CustomQueryset.as_manager()
So having:
orders = Order.objects.filter(user=user)
If you do {% if orders.no_old %} will do another query, because this is new QuerySet instance which has no cache..
About the {% regroup %} tag
As you mention, in order to use it, you need to .order_by('old'), and if you have another order, you can still use it, just apply your order after the old, e.g. .order_by('old', 'another_field'). This way you will use only one Query and this will save you one iteration over the list (because Django will split the list iterating it only once), but you will get less readability in the template.
So I have a template:
{% for Chapter in latest_chapter_list %}
{% ifequal Chapter.manga|truncatechars:20 Manga.name|truncatechars:20 %}
{{Chapter.chapter}}
{% endifequal %}
{% endfor %}
models:
class Manga(models.Model):
name = models.CharField(max_length=20,help_text='Name of the comic/manga')
class Chapter(models.Model):
manga = models.ForeignKey(Manga)
chapter = models.IntegerField(default=1, help_text='Number of a chapter')
So what I want is that the template would only display 5 items instead of all items that pass the if. In normal code I would add a temp value that counts each addition and later on resets, but I'm new to django and I don't know how to approach this.
Also I can't figure out why my if only works if I cut both names to equal length, even though they should be the same length, shouldn't they?
Also my views:
def index(request):
latest_item_list = Manga.objects.all().order_by('-added_on')[:5]
latest_chapter_list = Chapter.objects.all().order_by('-chapter')
context = {'latest_item_list': latest_item_list,
'latest_chapter_list': latest_chapter_list
}
return render(request, 'Item/index.html', context)
any help or tips would be appreciated!
EDIT, SOLUTION: made new filtered list in views:
latest_chapter_list_short = Chapter.objects.filter(chapter__lt=6)
and iterated through it instead of the full list!
Django has limited functionality in its templates so it's easier to only iterate over the elements that you wish to display. For example, you can prepare the data in the view to only contain five elements:
latest_item_list = Manga.objects.all().order_by('-added_on')[:5]
chapters = Chapter.objects.all().order_by('-chapter')
latest_chapter_per_mange = compute_chapters_per_mange(chapters, latest_item_list)
where compute_chapters_per_mange determines the chapters per Manga model.
Your second question: Django is now comparing the string representation of a Manga object to Manga.name which may not be the same. This could explain why you need truncatechars although I don't fully see why. If you define a __unicode__ method on your Manga model then you can specify how an instance of that model should be displayed as a string (i.e., self.name).
I have a Contact class that can have many Conversations. But each conversation can belong to a single Contact.
So its a One-To-Many relationship.
class Contact(models.Model):
first_name = models.CharField()
last_name = models.CharField()
class Conversation(models.Model):
contact = models.ForeignKey(Contact)
notes = models.TextField()
def __unicode__(self):
return self.notes
Now when I pass in the contacts to the template, I would like to have one field that shows the last conversation for the contact.
contacts= profile.company.contact_set.all()[:10]
calls = []
for c in contacts:
calls.append(max(c.conversation_set.all()))
And I pass them in like this:
vars = {'contacts' : contacts, 'calls': calls}
variables = RequestContext(request, vars)
return render_to_response('main_page.html', variables)
In the template I came up with this magic:
{% for idx, val in contacts %}
<tr>
<td>...
</td>
<td>
{% if calls %}
{{ calls.idx }}
{% endif %}</td>
</tr>
{% endfor %}
This doesn't show anything for calls. But if I replaced calls.idx with calls.0 I get the first one displayed.
What am I doing wrong? Beside the fact that it could be done probably much easier than that. :) I am open for better ways of doing it.
You can't do this sort of indirect lookup in the template language - calls.idx will always refer to an attribute idx belonging to calls.
However I think a better solution is to add the call value directly onto the contact object - don't forget that in Python objects are dynamic, so you can annotate anything you like onto them.
for c in contacts:
c.max_call = max(c.conversation_set.all())
As an aside, does that max really work? I'm not sure what it would be doing the comparison based on. An alternative is to define get_latest_by in your Conversation model, then you can avoid this loop altogether and just call {{ contact.conversation_set.get_latest }} in your template for each contact through the loop.
(Note this will still be pretty inefficient, there are various ways of getting the whole set of max conversations in one go, but it will do for now.)
class Contact(models.Model):
# your code here
#property
def last_conversation(self):
try:
return self.conversation_set.order_by("-some_date_or_order_field")[0]
except IndexError:
return None
Then you don't have to care about this in your view and just need to call "c.last_contact" in your template.