I have been succesfully using this brilliant technique to keep my code DRY encapsulating ORM relations in querysets so that code in views is simple and not containing foreign key dependency. But this time I face the following issue best descriped by code:
View:
vendors_qs = vendors_qs.select_related().prefetch_related('agreement_vendors')
Model
class AgreementVendorsQuerySet(models.query.QuerySet):
def some_filter1(self, option):
result = self.filter(.....)
return result
def some_filter2(self, option):
result = self.filter(.....)
return result
And a template:
{% for vendor in vendors_qs %}
<tr>
...
<td>
{% for vend_agr in vendor.agreement_vendors.all %}
{{vend_agr.serial_number}}
{% endfor %}
<td>
</tr>
{% endfor %}
The question is, how and where do I apply the some_filter1 to vendor agreements given that it is fetched as prefetch_related relation. Do I have to apply the filter in the template somehow or in the view itself ?
If I didn't put the question clearly enough, I will ask your questions to clarify further...
UPDATE:
Anna's asnwer looks very much like the truth, but one detail remains unclear. What if I want to apply several filters based on if-condition. For exapmle, if the filters were to apply to vendors, then the code would simply look like:
if (condition_1)
vendors_qs = vendors_qs.filter1()
if (condition_2)
vendors_qs = vendors_qs.filter2()
If I clearly understand your question you need something like this
vendors_qs = vendors_qs.prefetch_related(models.Prefetch('agreement_vendors', queryset=some_filter, to_attr='agreement_vendors_list'))
And then in template you can call it like {% for vend_agr in vendor.agreement_vendors_list %}
Related
I am trying to figure out how to produce the output between two querysets that have similar fields. I have two different models where I keep identical fields, and at times I want to show the differences between these two models. I have researched and successfully used sets and lists in my python code, but I can't quite figure out how to leverage them or determine if I Can. Sets seems to strip out just the field values, and when I try to loop through my sets, it doesn't currently work because it's just the values without the keys.
Example:
class Team(models.Model):
player = models.Charfield
coach = models.Charfield
class TeamCommittee(models.Model):
player = models.Charfield
coach = models.Charfield
I want to be able to query both models at times and be able to exclude data from one model if it exists in the other. I have spent the afternoon trying to use sets, lists, and loops in my django templates but can't quite work this out. I have also tried various querysets using exclude....
I tried something like....
query1 = TeamCommittee.objects.filter(id=self.object.pk).values('coach','player')
query2 = Team.objects.filter(id=self.object.pk).exclude(id__in=query1)
When I use the approach above, I get TypeError: Cannot use multi-field values as a filter value.
I am wondering if I can do this via a query or if I need to dump my querysets and go down a path of manipulating a data dictionary? That seems extreme for what I am trying to do though. This does seem to be a bit more complicated because I am trying to cross reference two different models. If it was the same model this would be a lot easier but it's not an option for this particular use case.
Thanks in advance for any thoughts on the right way to approach this.
If you want to compare on the basis of the ID of both tables, probably you can use this:
teamID = list(Team.objects.all().values_list('id', flat=True))
query1 = TeamCommittee.objects.filter(id__in=teamID)
teamCommitteeID = list(TeamCommittee.objects.all().values_list('id', flat=True))
query2 = Team.objects.filter(id__in=teamCommitteeID)
I was super close...Instead I just did this...
query1 = TeamCommittee.objects.filter(id=self.object.pk).values('coach','player').distinct()
Then in my template I did a very simple....
{% for item in query1.all %}
item
{% endfor %}
Then when I wanted to get the values out of the other queryset I just did the same thing with the loop.
query2 = Team.objects.filter(id=self.object.pk).values('coach','player').distinct()
Then in my template I did a very simple....
{% for item in query2.all %}
item
{% endfor %}
Sometimes simplicity is hard.
The answer above after additional testing only partially worked. I ultimately abandoned that approach and instead incorporated the logic below into my template. I did not need to create separate queries, I just needed to loop through the fields that were already available to me as part of the DetailView I was using....
{% for author in author_detail.author_set.all %}
{% for book in book_detail.book_set.all %}
{% if author.author_name %}
{% if book.book_name in author.book_name %}
{% if author.book_name == book.book_name %}
{% elif author.book_name != book.book_name %}
{{ author.author_name }}
{% endif %}
{% endif %}
{% endif %}
{% endfor %}
{% endfor %}
Thank you to everyone who made suggestions to get me to this point.
I have a model specifications which are divided into categories. For the template, in order to display the category as an header in the table, I make a prefetch_related on the Category like this:
categories = Category.objects.distinct().prefetch_related('specifications').filter(filters)
Now I can loop over the categories and show the related specifications like:
{% for category in categories %}
<tr>
<th colspan="7">{{ category.name }} - ({{ category.abbr }})</th>
</tr>
{% for specification in category.specifications.all %}
...
I also want to use the paginator, but now the paginator only counts the categories and not the related specifications. Is it possible to paginate on the specifications with the given query or should I change the query to retrieve all the specifications?
Use Prefetch
categories = Category.objects.distinct().filter(filters)
category_ids = categories.values_list('id', flat=True) # category ids on page
categories = categories.prefetch_related(
Prefetch(
'specifications',
queryset=Specialization.objects.filter(category_id__in=category_ids)
)
)
Here it creates another db request (to fetch category ids) but it will cost less than prefetch all specializations I think. It depends on your data structure but it definitely one of solutions.
Have you tried django-tables2?
With it you could just simply render the table with something like:
Create a CategoryTable class and add it to your view:
class CategoryTable(tables.Table):
class Meta:
model = Category
def your_view(request):
...
categories = Category.objects.distinct()
.prefetch_related('specifications')
.filter(filters)
table = CategoryTable(categories)
table.paginate(page=request.GET.get("page", 1), per_page=25)
return render(request, "category_template.html", {"table": table})
Then, in your template just put:
{% load django_tables2 %}
{% render_table table %}
What you are trying to achieve is an anti-pattern of prefetch_related. Prefetch is to fetch "all" the related rows, but the use case is to paginate the specifications.
Prefetch would be good in cases where the number of related rows per parent row is ~5 (or upto 10, remember that you are wasting DB network bandwidth growing prefetched child rows. So if you are considering pagination, it is best to avoid prefetch)
Note: Child rows = specifications, Parent rows = categories, in your use case.
My answer would be to avoid using prefetch in this case. Just use the following
categories = Category.objects.distinct().filter(filters)
--
{% for category in categories %}
<tr>
<th colspan="7">{{ category.name }} - ({{ category.abbr }})</th>
</tr>
# Use some table lib like django-tables
...
If this is a simple internal project, do go ahead with prefetch, no problems, otherwise you are going to hit DB performance issues.
I have a model, say:
from model_utils import Choices
class Day(models.Model):
WEATHER = Choices(
('r', 'rainy', 'Rainy'),
('s', 'sunny', 'Sunny'),
)
weather = models.CharField(max_length=1, choices=WEATHER)
Is there a way, in the presentation, to use the logical keywords e.g.:
{% ifequal monday.weather Day.WEATHER.rainy %}
It rained monday
{% endifequal %}
I am aware of the get_FOO_display() method, but would rather not test against a hardcoded string (e.g., monday.get_weather_display == 'Rainy'). Is there a template tag or inclusion or some way to refer to this decleration in the model? Or at least a best practices suggestion for doing choices-related logic in the template? I have found little evidence in the docs, can't really wrap my head around the source, and all questions I've found on here are people looking for the get_FOO_display method.
Thanks.
I would write a property method in the model
class Day(models.Model):
[...]
#property
def is_rainy(self):
return self.weather == self.WEATHER.rainy
And in my template I would check:
{% if monday.is_rainy %}
It rained monday
{% endif %}
I (foolishly) had already implemented this in my code and had forgotten about it! One can use the instance to refer to the choice possibilities, per my example:
{% if monday.weather == monday.WEATHER.rainy %}
[...]
I would delete the question, but the fact remains it is not explained very clearly in the docs and does not appear to be a duplicate of anything I was able to find on stackoverflow.
I am trying to understand the best way to structure queries in django to avoid excessive database hits.
This is similar to the question: Django best practice with foreign key queries, but involves greater 'depth' in the queries.
My situation:
models.py
class Bucket(models.Model):
categories = models.ManyToManyField('Category')
class Category(models.Model):
name = models.CharField(max_length=50)
class SubCategory(models.Model):
category = models.ForeignKey(Category)
class SubSubCategory(models.Model):
subcat = models.ForeignKey(SubCategory)
views.py
def showbucket(request, id):
bucket = Bucket.objects.prefetch_related('categories',).get(pk=id)
cats = bucket.categories.prefetch_related('subcategory_set__subsubcategory_set',)
return render_to_response('showbucket.html', locals(), context_instance=RequestContext(request))
and relevant template:
{% for c in cats %}
{{c}}
<ul>
{% for d in c.subcategory_set.all %}
<li>{{d}}</li>
<ul>
{% for e in d.subsubcategory_set.all %}
<li>{{e}}</li>
{% endfor %}
</ul>
{% endfor %}
</ul>
{% endfor %}
Despite the use of prefetch_related(), I seem to be hitting the database each time the top two for statements are evaluated, e.g. {% for c in cats %}, (at least I believe so from reading the debug_toolbar). Other ways I've tried have ended up with (C x D x E) number of database hits. Is this something inherently wrong with my use of prefetch, queries, or models? What is the best way in Django to access database objects with a "depth > 1" so-to-speak?
Use select_related() instead:
https://docs.djangoproject.com/en/dev/ref/models/querysets/#django.db.models.query.QuerySet.select_related
bucket = Bucket.objects.select_related('categories',).get(id=id)
cats = bucket.categories.select_related('subcategory_set__subsubcategory_set',)
So, i found out there's a couple things going on here:
First, my current understanding on select_related vs prefetch_related:
select_related() follows foreign-key relationships, causing larger result sets but means that later use of FK won't require additional database hits. It is limited to FK and one-to-one relationships.
prefetch_related() does a separate lookup for each relationship and joins them in python, and is means to be used for many-to-many, many-to-one, and GenericRelation and GenericForeignKey.
By the book, I should be using prefetch(), as I was not 'following' the Foreign Keys.
That's what I had understood going into this, but my template seemed to be causing additional queries when evaluating the given for loops in the template, even when I added the use of {with} tags.
At first, I had thought I had discovered something similar to this issue, but I am unable to replicate when I built out my simplified example. I switched from using the debug toolbar to direct checking using the following template code (in the article Tracking SQL Queries for a Request using Django by Karen Tracey, I would link but am link-limited):
{% with sql_queries|length as qcount %}
{{ qcount }} quer{{ qcount|pluralize:"y,ies" }}
{% for qdict in sql_queries %}
{{ qdict.sql }} ({{ qdict.time }} seconds)
{% endfor %}
{% endwith %}
Using this method, I am only seeing 5 queries for using pre-fetch() (7 with debug_toolbar), and queries grow linearly when using select_related() (with +2 for debug_toolbar), which I believe is expected.
I will gladly take any other advice/tools on getting a handle on these types of issues.
Sorry about the strange title, but I couldn't explain the situation in a few words. Let me articulate:
I have a Jobs model whose objects I show in a template. Against each job, I also want to show if the user has already applied for that job.
I have these models
class Job(models.Model):
is_valid = models.BooleanField()
description = models.CharField()
def has_user_applied(self, user):
return jobapplication_set.filter(applicant=user).exists()
class JobApplication(models.Model):
applicant = models.ForeignKey(User)
job = models.ForeignKey(Job)
cover_letter = models.CharField()
And a view in which I fetch all the Jobs:
jobs = Job.objects.filter(is_valid=True)
return HttpResponse( ... {'jobs': jobs} ... )
And a template in which I list them:
{% for j in jobs %}
{{ j.description }} {% if j.has_applied %} (You've already applied) {% endif %}
{% endfor %}
However the "has_applied" function takes "user" as argument and passing arguments in templates is not allowed.
Now I have two issues:
Can I create a "context" so that some functions can assume that a particular user is in question rather than it being passed explicitly and limiting its usage in templates? If it is not possible, what is the elegant way of annotating this information in the model objects in the views?
Secondly, even If I'm able to do this, for each Job object, I still have to execute a separate query to determine if user has already applied. I know this is possible in raw SQL using outer joins, but can I do it using django's ORM?
Easiest and most verbose way of doing it will be just:
view
jobs_applied_by_user = set(JobApplication.objects
.filter(applicant=user)
.distinct()
.values_list('job', flat=True))
template
{% for job in jobs %}
{% if job.pk in jobs_applied_by_user %}
…
One simple solution is to write the custom filter.
{% for j in jobs %}
{{ j.description }} {% if j|has_applied:user %} (You've already applied) {% endif %}
{% endfor %}
Here has_applied is custom filter and it will take user as parameter.