I'm trying to do a conditional update based on the value of a field on a foreign key. Example:
Model Kid: id, parent (a foreign key to Parent), has_rich_parent
Model Parent: id, income
So say I have a query set of A. I wanna update each item's has_guardian in A based on the value of age on the Kid's parent in one update. What I was trying to do is
queryset_of_kids.update(
has_rich_parent=Case(
When(parent__income__gte=10, then=True)
default=False
)
)
But this is giving me an error Joined field references are not permitted in this query. Which I am understanding it as joined fields / pursuing the foreignkey relationships aren't allowed in updates.
I'm wondering if there's any other way to accomplish the same thing, as in updating this queryset within one update call? My situation has a couple more fields that I'd like to verify instead of just income here so if I try to do filter then update, the number of calls will be linear to the number of arguments I'd like to filter/update.
Thanks in advance!
Here are the models that I assume you're using:
from django.db import models
class Kid(models.Model):
parent = models.ForeignKey('Parent', on_delete=models.CASCADE)
has_rich_parent = models.BooleanField(default=False)
class Parent(models.Model):
income = models.IntegerField()
You can use a Subquery to update the has_rich_parent field.
The subquery filters on the primary key pk of the surrounding query using .filter(pk=OuterRef('pk')).
It uses a Q query object to obtain whether the parent income is >= 10.
from .models import Kid, Parent
from django.db.models import Q, Subquery, OuterRef
Kid.objects.update(has_rich_parent=Subquery(
Kid.objects.filter(pk=OuterRef('pk'))
.values_list(Q(parent__income__gte=10))))
That command produces the following SQL query:
UPDATE "more_kids_kid"
SET "has_rich_parent" = (
SELECT (U1."income" >= 10) AS "q1"
FROM "more_kids_kid" U0
INNER JOIN "more_kids_parent" U1 ON (U0."parent_id" = U1."id")
WHERE U0."id" = ("more_kids_kid"."id")
)
This query isn't as efficient as a SELECT-then-UPDATE query. However, your database may be able to optimize it.
I'm developing a Django project. I need to make many queries with the following pattern:
I have two models, not related by a FK, but that can be related by some fields (not their PKs).
I need to query the first model, and annotate it with results from the second model, joined by that field that is not de PK.
I can do it with a Subquery and an OuterRef function.
M2_queryset = M2.objects.filter(f1 = OuterRef('f2'))
M1.objects.annotate(b_f3 = Subquery(M2_queryset.values('f3')))
But if I need to annotate two columns, I need to do this:
M2_queryset = M2.objects.filter(f1 = OuterRef('f2'))
M1.objects.annotate(b_f3 = Subquery(M2_queryset.values('f3'))).annotate(b_f4 = Subquery(M2_queryset.values('f4')))
It's very inefficient because of the two identical subqueries.
It would be very interesting doing something like this:
M2_queryset = M2.objects.filter(f1 = OuterRef('f2'))
M1.objects.annotate(b_f3, b_f4 = Subquery(M2_queryset.values('f3','f4')))
or more interesting something like this and avoiding subqueries:
M1.objects.join(M2 on M2.f1 = M1.f2)...
For example in this model:
db
I need to do this regular query:
select m1.id,m1.f5, sum(m2.f2), sum(m2.f3)
from M1, M2
where M1.f1 = M2.f2
group by 1,2
without a fk between f1 and f2.
I have an Article model like this
from django.contrib.contenttypes.fields import GenericRelation
from django.db import models
from hitcount.models import HitCountMixin, HitCount
class Article(models.Model):
title = models.CharField(max_length=250)
hit_count_generic = GenericRelation(
HitCount, object_id_field='object_pk',
related_query_name='hit_count_generic_relation')
when I do Article.objects.order_by('hit_count_generic__hits'), I am getting results.but when I do
articles_by_id = Article.objects.filter(id__in=ids).annotate(qs_order=models.Value(0, models.IntegerField()))
articles_by_name = Article.objects.filter(title__icontains='sports').annotate(qs_order=models.Value(1, models.IntegerField()))
articles = articles_by_id.union(articles_by_name).order_by('qs_order', 'hit_count_generic__hits')
getting error
ORDER BY term does not match any column in the result set
How can i achieve union like this? I had to use union instead of AND and OR because i need to preserve order. ie; articles_by_id should come first and articles_by_name should come second.
using Django hitcount for hitcount https://github.com/thornomad/django-hitcount. Hitcount model is given below.
class HitCount(models.Model):
"""
Model that stores the hit totals for any content object.
"""
hits = models.PositiveIntegerField(default=0)
modified = models.DateTimeField(auto_now=True)
content_type = models.ForeignKey(
ContentType, related_name="content_type_set_for_%(class)s", on_delete=models.CASCADE)
object_pk = models.TextField('object ID')
content_object = GenericForeignKey('content_type', 'object_pk')
objects = HitCountManager()
As suggested by #Angela tried prefetch related.
articles_by_id = Article.objects.prefetch_related('hit_count_generic').filter(id__in=[1, 2, 3]).annotate(qs_order=models.Value(0, models.IntegerField()))
articles_by_name = Article.objects.prefetch_related('hit_count_generic').filter(title__icontains='date').annotate(qs_order=models.Value(1, models.IntegerField()))
the query of the prefetch_related when checked is not selecting the hitcount at all see.
SELECT "articles_article"."id", "articles_article"."created", "articles_article"."last_changed_date", "articles_article"."title", "articles_article"."title_en", "articles_article"."slug", "articles_article"."status", "articles_article"."number_of_comments", "articles_article"."number_of_likes", "articles_article"."publish_date", "articles_article"."short_description", "articles_article"."description", "articles_article"."cover_image", "articles_article"."page_title", "articles_article"."category_id", "articles_article"."author_id", "articles_article"."creator_id", "articles_article"."article_type", 0 AS "qs_order" FROM "articles_article" WHERE "articles_article"."id" IN (1, 2, 3)
From Django's official documentation:
Further, databases place restrictions on what operations are allowed in the combined queries. For example, most databases don’t allow LIMIT or OFFSET in the combined queries.
So, make sure that your database allows combining queries like this.
ORDER BY term does not match any column in the result set
You are getting this error, because that's exactly what's happening. Your final result-set for articles does not contain the hits column from the hitcount table , due to which the result-set cannot order using this column.
Before delving into the answer, let's look at what's happening with your django querysets under the hood.
Retrieve a particular set of articles and include an extra ordering field qs_order set to 0.
articles_by_id = Article.objects.filter(id__in=ids).annotate(qs_order=models.Value(0, models.IntegerField()))
SQL Query for the above
Select id, title,....., 0 as qs_order from article where article.id in (Select ....) # whatever you did to get your ids or just a flat list
Retrieve another set of articles and include an extra ordering field qs_order set to 1
articles_by_name = Article.objects.filter(title__icontains='sports').annotate(qs_order=models.Value(1, models.IntegerField()))
SQL Query for the above
Select id, title, ...1 as qs_order from article where title ilike '%sports%'
Original queryset and order_by hit_count_generic__hits
Article.objects.order_by('hit_count_generic__hits')
This will actually perform an inner join and fetch the hitcount table to order by the hits column.
Query
Select id, title,... from article inner join hitcount on ... order by hits ASC
Union
So when you do your union, the result-set of the above 2 queries is combined and then ordered using your qs_order and then hits ...where it fails.
Solution
Use prefetch_related to get your hitcount table in the initial queryset filtering, so you can then use the hits column in the union to order.
articles_by_id = Article.objects.prefetch_related('hit_count_generic').filter(id__in=ids).annotate(qs_order=models.Value(0, models.IntegerField()))
articles_by_name = Article.objects.prefetch_related('hit_count_generic').filter(title__icontains='sports').annotate(qs_order=models.Value(1, models.IntegerField()))
Now as you have the desired table and its columns in both your SELECT queries, your union should work the way you have defined.
articles = articles_by_id.union(articles_by_name).order_by('qs_order', 'hit_count_generic__hits')
Just replacing prefetch_related with select_related works for me.
https://docs.djangoproject.com/en/3.2/ref/models/querysets/#select-related
I have 3 models (User-default model , Project, ClickedUsers-custom m2m model) and I want to perform left join with filter in ORM on ClickedUsers and Project models. I tried the solutions on Stack Overflow but when I print those queries I saw they performed Inner join.
Here are my models :
class Project(models.Model):
…
clicked_users = models.ManyToManyField(User,through='ClickedUsers',blank=True)
class ClickedUsers(models.Model):
user = models.ForeignKey(User,on_delete=models.CASCADE)
project = models.ForeignKey(Project,on_delete=models.CASCADE)
status = models.IntegerField(default=0)
And the query I want to perform :
select * from project LEFT JOIN clickedusers ON project.id = clickedusers.project_id WHERE clickedusers.user_id = 1;
How can I done this query with django orm?
A LEFT JOIN won't give you different results from an INNER JOIN in this query
SELECT *
FROM table_1
JOIN table_2 on table_2.table_1_id = table_1.id
WHERE table_2.field = some_value;
Since you have a field from table_2 in the where clause, you can only get results from table_1 when there is a record in table_2. A LEFT JOIN says to get records from table_1 even if there is no record in table_2 satisfying the join criteria.
The reason you aren't getting the query from the ORM that you want is that the ORM recognizes it can use an INNER JOIN instead of a LEFT JOIN without changing the results.
Look into the reverse relationships and related_name.
Project.objects.filter(clicked_users__id=1)
Or you use the through model:
Project.objects.filter(clickedusers_set__user_id=1)
Edit:
I missed the * from select * from. If you need all the fields, then you'll need to use:
ClickedUser.objects.filter(
user_id=1,
).select_related('project')
I have following models:
class Dictionary(models.Model):
word = models.CharField(unique=True)
class ProcessedText(models.Model):
text_id = models.ForeiginKey('Text')
word_id = models.ForeignKey('Dictionary')
class UserDictionary(models.Model):
word_id = models.ForeignKey('Dictionary')
user_id = models.ForeignKye('User')
I want to make query using django ORM same with next sql
SELECT * FROM ProcessedText, UserDictionary WHERE
ProcessedText.text_id = text_id
AND ProcessedText.word_id = UserDictionary.word_id
AND UserDictionary.user_id = user_id
How to do it in one query without using cycles?
This might help you:
How do I select from multiple tables in one query with Django?
And also you may have to restructure your models to enable select_related concept of django.