1.
1.1. query:
QuestionRequestAnswerModel.objects.filter(request_no__exact=index['id']).select_related('answer_user').values('answer_user', 'amount', 'select_yn')
2.
2.1. query:
QuestionRequestAnswerModel.objects.filter(request_no__exact=index['id']).select_related('answer_user')[0].answer_user
description
I want to get answer_user's last_name.
but when I use django orm by select_realated('answer_user'),
the result give me the answer_user's id only.
how can I get another column, except id, on ORM join?
You can follow foreign key relationships in value queries by using the double underscore notation
QuestionRequestAnswerModel.objects.filter(
request_no__exact=index['id']
).select_related(
'answer_user'
).values(
'answer_user__last_name',
'amount',
'select_yn'
)
Related
Django search_fields adding DUPLICATE left outer join with appliance table to Django query as it have .annotate(appliances_count=Count('appliances'))
View
class AppliancePoolViewSet(VneCommonViewSet):
serializer_class = vne_serializers.AppliancePoolSerializer
search_fields = ('pk', 'name', 'notes', 'appliances__name')
# filter appliance pools by customer
def get_queryset(self):
customer = getattr(self.request.user, 'customer', None)
if not customer:
return models.AppliancePool.objects.none()
return models.AppliancePool.objects.filter(
customer=1).prefetch_related('appliances'
).annotate(appliance_count=Count('appliances'))
First it is adding join for annotate, and later search_field is also adding the same join again as it have field 'appliances__name', which is causing incorrect data for appliance_count.
How can I restrict addition of duplicate join which is done by Django filters using search_fields attribute?
There are some note when using Django ORM:
First, the table join will be duplicated by:
https://docs.djangoproject.com/en/3.0/topics/db/queries/#spanning-multi-valued-relationships
I also found that it will affect when we using same relation in different place like: filter, annotate
Django will consider all those are single and should not merge
After deeper look on Django code, I found this:
https://docs.djangoproject.com/en/3.0/ref/models/querysets/#filteredrelation-objects
This will help to avoid spanning relations, we will have only 1 table join.
Sorry that my post will NOT help on the case: prefetch_related
Hope this will help other, as I found this one when search the solution for https://code.djangoproject.com/ticket/18437
I found an work around for this.
I replaced annotate with .extra('Query to get appliance count').
I did not used join in query inside extra. So final query have only single join, no duplicate join.
My models are similar to the following:
class Reporter(models.Model):
def gold_star(self):
return self.article_set.get().total_views >= 100000
class Article(models.Model):
reporter = models.ForeignKey(Reporter, on_delete=models.CASCADE)
total_views = models.IntegerField(default=0, blank=True)
Then in one of the templates I have this line:
{% if r.gold_star %}<img src="{% static 'gold-star.png' %}">{% endif %}
Obviously django sends as many queries as there are reporters on the page... Ideally this could be just one query, which would select reporters by criteria and join appropriate articles. Is there a way?
EDIT
Neither select_related nor prefetch_related doesn't seem to work as I'm selecting on the Reporter table and then use RelatedManager to access related data on the Article.
In other words django doesn't know what to prefetch until there's non empty queryset.
Because an article can only have one reporter it's for sure possible to join these tables together and then apply filter to subquery, I just can't find how it's done in django query language.
There's alternative - select on the Article table and filter by Reporter fields, but there's a problem with such approach. If I deleted all the articles of some reporter then I wouldn't be able to include that reporter in the list as from the Article point of view such reporter doesn't exist and yet reporter is in the Reporter table.
EDIT2
I tried what people suggested in the comments. The following generates desired query:
reporters = Reporter.objects.filter(**query).select_related().annotate(
gold_star=Case(
When(article__total_views__gte=0, then=Value(1)),
default=Value(0),
output_field=IntegerField()
)
)
Query generated by django:
SELECT
`portal_reporter`.`id`,
...,
CASE WHEN `portal_article`.`total_views` >= 0 THEN 1 ELSE 0 END AS `gold_star`
FROM
`portal_reporter`
LEFT OUTER JOIN `portal_article`
ON (`portal_reporter`.`id` = `portal_article`.`reporter_id`)
WHERE
...
Now I just need to work out a way how to produce similar query but without Case/When statements.
EDIT3
If I chose slightly different strategy, then django selects wrong join type:
query['article__id__gte'] = 0
reporters = Reporter.objects.filter(**query).select_related()
This code produce similar query but with the INNER JOIN instead of desired LEFT OUTER JOIN:
SELECT
`portal_reporter`.`id`,
...,
FROM
`portal_reporter`
INNER JOIN `portal_article`
ON (`portal_reporter`.`id` = `portal_article`.`reporter_id`)
WHERE
...
You can use select_related (https://docs.djangoproject.com/en/1.11/ref/models/querysets/#select-related) to do a join on the related table.
There's also prefetch_related (https://docs.djangoproject.com/en/1.11/ref/models/querysets/#prefetch-related) which uses an IN clause to fetch the related objects with an extra query. The difference is explained in the docs, but is reproduced below:
select_related works by creating an SQL join and including the fields of the related object in the SELECT statement. For this reason, select_related gets the related objects in the same database query. However, to avoid the much larger result set that would result from joining across a ‘many’ relationship, select_related is limited to single-valued relationships - foreign key and one-to-one.
prefetch_related, on the other hand, does a separate lookup for each relationship, and does the ‘joining’ in Python. This allows it to prefetch many-to-many and many-to-one objects, which cannot be done using select_related, in addition to the foreign key and one-to-one relationships that are supported by select_related. It also supports prefetching of GenericRelation and GenericForeignKey, however, it must be restricted to a homogeneous set of results. For example, prefetching objects referenced by a GenericForeignKey is only supported if the query is restricted to one ContentType.
Try annotating the new field gold_star and set it to 1 if reporter has an article that has more than 100000 total_views like this:
from django.db.models import Case, When, Value, IntegerField
reporters = Reporter.objects.annotate(
gold_star=Case(
When(article__total_views__gte=100000, then=Value(1)),
default=Value(0),
output_field=IntegerField()
)
)
You can leave the template code as it is.
i want to use django models feature to excute this sql query.
SELECT COUNT(DISTINCT ques_id), title FROM contest_assignment WHERE grp_id = 60 GROUP BY title;
i tried this but it did not give me proper result:
from django.db.models import Count
assignment.objects.values_list('title').annotate(count=Count('ques')).values('title', 'count')
how can i use django model?
You shouldn't use both .values() and .values_list(). The count field is implicitly added to the set of fields that is returned.
Django supports a COUNT(DISTINCT <field>) query by using .annotate(count=Count('<field>', distinct=True)). That would leave you with the following query:
(assignment.objects.filter(grp_id=60).values('title')
.annotate(count=Count('ques_id', distinct=True)))
I need to get list of all companies and join the company user with minimal companyuser id.
There are two models:
class Company(models.Model):
name = models.CharField(max_length=255)
kind = models.CharField(max_length=255)
class CompanyUser(models.Model):
company = models.ForeignKey('Company')
email = models.EmailField(max_length=40, unique=True)
#other fields
I've tried something like this:
companies = Company.objects.all().select_related(Min('companyuser__email'))
but It doesn't work. How can I do this with Django ORM? Is there any way to do it without raw SQL?
from django.db.models import Min
Company.objects.annotate(lowest_companyuser_id=Min("companyuser__id"))
Explanation
select_related() can be used for telling Django which related tables should be joined to the resulting queryset for reducing the number of queries, namely solving the dreaded "N+1 problem" when looping over a queryset and accessing related objects in iteration. (see docs)
With using Min() you were on the right track, but it ought to be used in conjunction with the annotate() queryset method. Using annotate() with aggregate expressions like Min(), Max(), Count(), etc. translates in an SQL query using one of the aforementioned aggregate expressions with GROUP BY. (see docs about annotate() in Django, about GROUP BY in Postgres docs)
As Burhan said - do not rely on the pk, but if u must...
companies = Company.objects.all().order_by('pk')[0]
need to get a queryset with the first book (by a date field) for each author (related to by foreign key) ...is there a Django ORM way to do this (without custom SQL preferred but acceptable)
*Edit: Please note that an answer that works using only a modern open source backend like Postgresql is acceptable ..still ORM based solution preferred over pure custom sql query)
Models
class Book(Model):
date = Datefield()
author = ForeignKey(Author)
class Author(Model):
name = CharField()
Book.objects.filter(??)
If you use PostgreSQL or another DB backend with support for DISTINCT ON there is a nice solution:
Books.objects.order_by('author', '-date').distinct('author')
Otherwise I don't know a solution with only one query. But you can try this:
from django.db.models import Q, Max
import operator
books = Book.objects.values('author_id').annotate(max_date=Max('date'))
filters = reduce(operator.or_, [(Q(author_id=b['author_id']) &
Q(date=b['max_date'])) for b in books])
queryset = Books.objects.filter(filters)
With the combination of .values() and .annotate() we group by the author and annotate the latest date of all books from that author. But the result is a dict and not a queryset.
Then we build a SQL statement like WHERE author_id=X1 AND date=Y1 OR author_id=X2 AND date=Y2.... Now the second query is easy.