I'm trying to do a conditional update based on the value of a field on a foreign key. Example:
Model Kid: id, parent (a foreign key to Parent), has_rich_parent
Model Parent: id, income
So say I have a query set of A. I wanna update each item's has_guardian in A based on the value of age on the Kid's parent in one update. What I was trying to do is
queryset_of_kids.update(
has_rich_parent=Case(
When(parent__income__gte=10, then=True)
default=False
)
)
But this is giving me an error Joined field references are not permitted in this query. Which I am understanding it as joined fields / pursuing the foreignkey relationships aren't allowed in updates.
I'm wondering if there's any other way to accomplish the same thing, as in updating this queryset within one update call? My situation has a couple more fields that I'd like to verify instead of just income here so if I try to do filter then update, the number of calls will be linear to the number of arguments I'd like to filter/update.
Thanks in advance!
Here are the models that I assume you're using:
from django.db import models
class Kid(models.Model):
parent = models.ForeignKey('Parent', on_delete=models.CASCADE)
has_rich_parent = models.BooleanField(default=False)
class Parent(models.Model):
income = models.IntegerField()
You can use a Subquery to update the has_rich_parent field.
The subquery filters on the primary key pk of the surrounding query using .filter(pk=OuterRef('pk')).
It uses a Q query object to obtain whether the parent income is >= 10.
from .models import Kid, Parent
from django.db.models import Q, Subquery, OuterRef
Kid.objects.update(has_rich_parent=Subquery(
Kid.objects.filter(pk=OuterRef('pk'))
.values_list(Q(parent__income__gte=10))))
That command produces the following SQL query:
UPDATE "more_kids_kid"
SET "has_rich_parent" = (
SELECT (U1."income" >= 10) AS "q1"
FROM "more_kids_kid" U0
INNER JOIN "more_kids_parent" U1 ON (U0."parent_id" = U1."id")
WHERE U0."id" = ("more_kids_kid"."id")
)
This query isn't as efficient as a SELECT-then-UPDATE query. However, your database may be able to optimize it.
Related
How can I check whether a many-to-many relationship exists or another property is fulfilled? When I try the query, it returns some rows twice!
Given a model
from django.db import models
class Plug(models.Model):
name = models.CharField(primary_key=True, max_length=99)
class Widget(models.Model):
name = models.CharField(primary_key=True, max_length=99)
shiny = models.BooleanField()
compatible = models.ManyToManyField(Plug)
I have the following items in my database:
from django.db.models import Q
schuko = Plug.objects.create(name='F')
uk = Plug.objects.create(name='G')
Widget.objects.create(name='microwave', shiny=True).compatible.set([uk])
Widget.objects.create(name='oven', shiny=False).compatible.set([uk])
Widget.objects.create(name='pc', shiny=True).compatible.set([uk, schuko])
Now I want all names of widgets that are shiny and/or compatible with Schuko:
shiny_or_schuko = sorted(
Widget.objects.filter(Q(shiny=True) | Q(compatible=schuko))
.values_list('name', flat=True))
But to my surprise, this does not return ['microwave', 'pc']. Instead, 'pc' is listed twice, i.e. shiny_or_schuko is ['microwave', 'pc', 'pc'].
Is this a Django bug? If not, how can I set up the query that I get 'pc' just once?
Is this a Django bug?
No. You simply perform a LEFT OUTER JOIN with the many-to-many table. If two or more related objects match, it will be included multiple times. This can be wanted behavior, for example if you add extra annotations to the elements that takes values from these related objects.
You can make use of .distinct() [Django-doc] to return only distinct elements:
Widget.objects.filter(
Q(shiny=True) | Q(compatible=schuko)
).values_list('name', flat=True).distinct()
I have an Article model like this
from django.contrib.contenttypes.fields import GenericRelation
from django.db import models
from hitcount.models import HitCountMixin, HitCount
class Article(models.Model):
title = models.CharField(max_length=250)
hit_count_generic = GenericRelation(
HitCount, object_id_field='object_pk',
related_query_name='hit_count_generic_relation')
when I do Article.objects.order_by('hit_count_generic__hits'), I am getting results.but when I do
articles_by_id = Article.objects.filter(id__in=ids).annotate(qs_order=models.Value(0, models.IntegerField()))
articles_by_name = Article.objects.filter(title__icontains='sports').annotate(qs_order=models.Value(1, models.IntegerField()))
articles = articles_by_id.union(articles_by_name).order_by('qs_order', 'hit_count_generic__hits')
getting error
ORDER BY term does not match any column in the result set
How can i achieve union like this? I had to use union instead of AND and OR because i need to preserve order. ie; articles_by_id should come first and articles_by_name should come second.
using Django hitcount for hitcount https://github.com/thornomad/django-hitcount. Hitcount model is given below.
class HitCount(models.Model):
"""
Model that stores the hit totals for any content object.
"""
hits = models.PositiveIntegerField(default=0)
modified = models.DateTimeField(auto_now=True)
content_type = models.ForeignKey(
ContentType, related_name="content_type_set_for_%(class)s", on_delete=models.CASCADE)
object_pk = models.TextField('object ID')
content_object = GenericForeignKey('content_type', 'object_pk')
objects = HitCountManager()
As suggested by #Angela tried prefetch related.
articles_by_id = Article.objects.prefetch_related('hit_count_generic').filter(id__in=[1, 2, 3]).annotate(qs_order=models.Value(0, models.IntegerField()))
articles_by_name = Article.objects.prefetch_related('hit_count_generic').filter(title__icontains='date').annotate(qs_order=models.Value(1, models.IntegerField()))
the query of the prefetch_related when checked is not selecting the hitcount at all see.
SELECT "articles_article"."id", "articles_article"."created", "articles_article"."last_changed_date", "articles_article"."title", "articles_article"."title_en", "articles_article"."slug", "articles_article"."status", "articles_article"."number_of_comments", "articles_article"."number_of_likes", "articles_article"."publish_date", "articles_article"."short_description", "articles_article"."description", "articles_article"."cover_image", "articles_article"."page_title", "articles_article"."category_id", "articles_article"."author_id", "articles_article"."creator_id", "articles_article"."article_type", 0 AS "qs_order" FROM "articles_article" WHERE "articles_article"."id" IN (1, 2, 3)
From Django's official documentation:
Further, databases place restrictions on what operations are allowed in the combined queries. For example, most databases don’t allow LIMIT or OFFSET in the combined queries.
So, make sure that your database allows combining queries like this.
ORDER BY term does not match any column in the result set
You are getting this error, because that's exactly what's happening. Your final result-set for articles does not contain the hits column from the hitcount table , due to which the result-set cannot order using this column.
Before delving into the answer, let's look at what's happening with your django querysets under the hood.
Retrieve a particular set of articles and include an extra ordering field qs_order set to 0.
articles_by_id = Article.objects.filter(id__in=ids).annotate(qs_order=models.Value(0, models.IntegerField()))
SQL Query for the above
Select id, title,....., 0 as qs_order from article where article.id in (Select ....) # whatever you did to get your ids or just a flat list
Retrieve another set of articles and include an extra ordering field qs_order set to 1
articles_by_name = Article.objects.filter(title__icontains='sports').annotate(qs_order=models.Value(1, models.IntegerField()))
SQL Query for the above
Select id, title, ...1 as qs_order from article where title ilike '%sports%'
Original queryset and order_by hit_count_generic__hits
Article.objects.order_by('hit_count_generic__hits')
This will actually perform an inner join and fetch the hitcount table to order by the hits column.
Query
Select id, title,... from article inner join hitcount on ... order by hits ASC
Union
So when you do your union, the result-set of the above 2 queries is combined and then ordered using your qs_order and then hits ...where it fails.
Solution
Use prefetch_related to get your hitcount table in the initial queryset filtering, so you can then use the hits column in the union to order.
articles_by_id = Article.objects.prefetch_related('hit_count_generic').filter(id__in=ids).annotate(qs_order=models.Value(0, models.IntegerField()))
articles_by_name = Article.objects.prefetch_related('hit_count_generic').filter(title__icontains='sports').annotate(qs_order=models.Value(1, models.IntegerField()))
Now as you have the desired table and its columns in both your SELECT queries, your union should work the way you have defined.
articles = articles_by_id.union(articles_by_name).order_by('qs_order', 'hit_count_generic__hits')
Just replacing prefetch_related with select_related works for me.
https://docs.djangoproject.com/en/3.2/ref/models/querysets/#select-related
My goal is to run a query on one model but sort the results based on a field in another model fetched via prefetch_related.
Suppose I have two models:
class ModelA(models.Model):
...some fields...
class ModelB(models.Model):
...some fields...
model_a = models.ForeignKey(ModelA, db_column='id')
year = models.IntegerField()
I have tried:
ModelA.objects.filter(...).prefetch_related(
Prefetch(
'modelb_set',
queryset=ModelB.objects.filter().order_by('-year'),
to_attr="modelb_date"
)
).order_by('-modelb_date')
but this fails because modelb_date is not a field on ModelA, it's a list. What I want is to order the ModelA queryset according to the latest associated date field (from ModelB). That is, if instance One of ModelA has a modelb_date attribute = [x, y, z] where x.year = 2017 and instance Two of ModelA has a modelb_date attribute = [v, w] where v.year = 2018 then the query would order instance Two before One.
I am using Django 2.0, python 3.6, and Oracle 12c.
Can anybody help? Thanks!
If you want to order by some value of a related model, with a one-to-many relation, then you need some sort of way to first "fold" that related data: for example taking the minimum, maximum (and depending on the type of the data, sum, average, etc. might also be valid options).
If we for example want to sort the ModelAs depending on the minimum date of the related ModelB items, then we can use an .annotate(..) and .order_by(..) combination, like:
ModelA.objects.filter(...).prefetch_related(
Prefetch(
'modelb_set',
queryset=ModelB.objects.filter().order_by('-year'),
to_attr="modelb_date"
)
).annotate(
first_date=Min('modelb__date')
).order_by('-first_date')
So here we will sort the ModelAs with the first_date in descending order, and first_date, is the lowest date for a related ModelB.
class Customer(models.Model):
name = models.CharField(max_length=200)
# ..
class CustomerTicket(models.Model):
customer = models.OneToOneField(Customer)
date = models.DateTimeField(auto_now_add=True)
# ..
I want to query all customers. And, adding for each customer its ticket if it has one in the date range - so I will get the ticket object only if it is in the given date range, otherwise the ticket field would be null.
Try this:
from django.db import models
customers = Customer.objects.select_related('customerticket').annotate(
ticket=models.Case(models.When(models.Q(customerticket__date__gt=date1) & models.Q(customerticket__date__lt=date2), then=models.F('customerticket')))
)
And you will get ticket as a computed field. Note that when referencing relational fields such as ForeignKey or OneToOneField, F() returns the primary key value rather than a model instance, which means your ticket field will have value of the primary key.
To get customers with related tickets with one query you can use select_related. To make complex condition you can use Q
from django.db.models import Q
Customer.objects.select_related('customerticket').filter(Q(customerticket__date__range=["2018-04-11", "2018-04-12"]) | Q(customerticket__isnull=True))
This will filter data by customerticket date.
Use queryset.filter:
from django.utils import timezone
Customer.objects.exclude(customerticket=None).filter(customerticket__date=timezone.now())
I have two models with an explicit many-to-many relationship: a thing, auth.user, and a "favorite" model connecting the two. I want to be able to order my "thing"s by whether or not they are favorited by a particular user. In Sqlite3, the best query i've come up with is (roughly) this:
select
*, max(u.name = "john cleese") as favorited
from thing as t
join favorite as f on f.thing_id = t.id
join user as u on f.user_id = u.id
group by t.id
order by favorited desc
;
The thing tripping me up in my sql-to-django translation is the max(u.name = "john cleese") bit. As far as I can tell, Django has support for arithmatic but not equality. The closest I can come is a case statement that doesn't properly group the output rows:
Thing.objects.annotate(favorited=Case(
When(favorites__user=john_cleese, then=Value(True)),
default=Value(False),
output_field=BooleanField()
))
The other direction I've tried is to use RawSQL:
Thing.objects.annotate(favorited=RawSQL('"auth_user"."username" = "%s"', ["john cleese"]))
However, this won't work, because (as far as I'm aware) there's no way to explicitly join the favorite and auth_user tables I need.
Is there something I'm missing?
This will achieve what you (or anyone else googling there way here) wants to do:
Thing.objects.annotate(
favorited=Count(Case(
When(
favorites__user=john_cleese,
then=1
),
default=0,
output_field=BooleanField(),
)),
)
From what I read in a related ticket, you can use subquery with the Exists query expression.
Exists is a Subquery subclass that uses an SQL EXISTS statement. In many cases it will perform better than a subquery since the database is able to stop evaluation of the subquery when a first matching row is found.
Assuming the middle model in your case of ManyToMany is called Favorite
from django.db.models import Exists, OuterRef
is_favorited_subquery = Favorite.objects.filter(
thing_id = OuterRef('pk')
)
Thing.objects.annotate(favorited=Exists(is_favorited_subquery))
Then you can order by favorited attribute of the query.
I'm not exactly sure what you're trying to achieve, but I would start it like this way.
from django.db import models
from django.contrib.auth.models import User
class MyUser(models.Model):
person = models.OneToOneField(User)
class Thing(models.Model):
thingname = models.CharField(max_length=10)
favorited_by = models.ManyToManyField(MyUser)
And in your view:
qs = MyUser.objects.get(id=pk_of_user_john_reese).thing_set.all()
Will give you all Thing objects of the given user.
You should have a look in the Django Docs for ManyToMany
I'm using Django for some years now in several smaller and even bigger Projects, but I have never used the RawSQL features. Most times I thought about it, I have had a mistake in my model design.