Let's say I have 2 tables A and B. Table B has a JSON field named preferences which contains a field with id of table A called a_id.
I want to count number of B rows which refers to A table rows like this:
A.objects.annotate(count=Count(B.objects.filter(preferences__a_id=OuterRef('id'))))
However, I get an error that operator does not exist: jsonb = bigint. According to this answer, I should be able to refer to JSON field attributes using two underscores (__) between names. Why doesn't it work in my case?
Also, below the error there is a message:
HINT: No operator matches the given name and argument types. You
might need to add explicit type casts.
But I don't understand to what it actually refers to. I tried casting OuterRef(id) to integer through Cast() expression but that does not make any difference.
You have to explicitely cast preferences.a_id to an explicit type, because it is a jsonb on PostgreSQL not an int
solution
from django.db import models
from django.db.models.functions.comparison import Cast
A.objects.annotate(count=Count(B.objects.annotate(casted_id=Cast("preferences__a_id", models.IntegerField())).filter(casted_id=OuterRef('id'))))
used Django 4.1
Related
I have some table ports(switch_ip, slot_number, port_number, many, more, columns) and would like to achieve the following PostgreSQL query using Django:
SELECT switch_ip, array_agg((slot_number, port_number, many, more, columns) ORDER BY slot_number, port_number) info
FROM ports
GROUP BY switch_ip
ORDER BY switch_ip
Using django.contrib.postgres.aggregates here's what I got so far:
Port.objects \
.values('switch_ip') \
.annotate(
info=ArrayAgg('slot_number', ordering=('slot_number', 'port_number'))
) \
.order_by('switch_ip')
I am unable to include more than one column in the ArrayAgg. None of ArrayAgg(a, b, c), ArrayAgg((a, b, c)), ArrayAgg([a, b, c]) seem to work. A workaround could involve separate ArrayAggs for each column and each with the same ordering. I would despise this because I have many columns. Is there any nicer workaround, possibly more low-level?
I suspect this is no issue with ArrayAgg itself but rather with tuple expressions in general. Is there any way to have tuples at all in Django queries? For example, what would be the corresponding Django of:
SELECT switch_ip, (slot_number, port_number, many, more, columns) info
FROM ports
If this is not yet possible in Django, how feasible would it be to implement?
I have spent lot of time searching for a working solution and here is a full recipe with code example.
You need to define Array "function" with square brackets in template
from django.db.models.expressions import Func
class Array(Func):
template = '%(function)s[%(expressions)s]'
function = 'ARRAY'
You need to define output field format (it must be array of some django field). For example an array of strings
from django.contrib.postgres.fields import ArrayField
from django.db.models.fields import CharField
out_format = ArrayField(CharField(max_length=200))
Finally make an ArrayAgg expression
from django.db.models import F
annotate = {'2-fields': ArrayAgg(Array(F('field1'), F('field2'), output_field=out_format), distinct=True) }
model.objects.all().annotate(**annotate)
(Optional) If field1 or field2 are not CharFields, you may include Cast as an argument of Array
from django.db.models.functions import Cast
annotate = {'2-fields': ArrayAgg(Array(Cast(F('field1'), output_field=CharField(max_length=200)), F('field2'), output_field=out_format), distinct=True) }
Having done a bit more research I guess one could add the missing tuple functionality as follows:
Create a new model field type named TupleField. The implementation might look kind of similar to django.contrib.postgres.fields.ArrayField. TupleField would be rather awkward because I don't think any RDBMS allows for composite types to be used as column types so usage of TupleField would be limited to (possibly intermediate?) query results.
Create a new subclass of django.db.models.Expression which wraps multiple expressions on its own (like Func in general, so looking at Func's implementation might be worthwile) and evaluates to a TupleField. Name this subclass TupleExpression for example.
Then I could simply annotate with ArrayAgg(TupleExpression('slot_number', 'port_number', 'many', 'more', 'columns'), ordering=('slot_number', 'port_number')) to solve my original problem. This would annotate each switch_ip with correctly-ordered arrays of tuples where each tuple represents one switch port.
I am using Django, with mongoengine. I have a model Classes with an inscriptions list, And I want to get the docs that have an id in that list.
classes = Classes.objects.filter(inscriptions__contains=request.data['inscription'])
Here's a general explanation of querying ArrayField membership:
Per the Django ArrayField docs, the __contains operator checks if a provided array is a subset of the values in the ArrayField.
So, to filter on whether an ArrayField contains the value "foo", you pass in a length 1 array containing the value you're looking for, like this:
# matches rows where myarrayfield is something like ['foo','bar']
Customer.objects.filter(myarrayfield__contains=['foo'])
The Django ORM produces the #> postgres operator, as you can see by printing the query:
print Customer.objects.filter(myarrayfield__contains=['foo']).only('pk').query
>>> SELECT "website_customer"."id" FROM "website_customer" WHERE "website_customer"."myarrayfield_" #> ['foo']::varchar(100)[]
If you provide something other than an array, you'll get a cryptic error like DataError: malformed array literal: "foo" DETAIL: Array value must start with "{" or dimension information.
Perhaps I'm missing something...but it seems that you should be using .filter():
classes = Classes.objects.filter(inscriptions__contains=request.data['inscription'])
This answer is in reference to your comment for rnevius answer
In Django ORM whenever you make a Database call using ORM, it will generally return either a QuerySet or an object of the model if using get() / number if you are using count() ect., depending on the functions that you are using which return other than a queryset.
The result from a Queryset function can be used to implement further more refinement, like if you like to perform a order() or collecting only distinct() etc. Queryset are lazy which means it only hits the database when they are actually used not when they are assigned. You can find more information about them here.
Where as the functions that doesn't return queryset cannot implement such things.
Take time and go through the Queryset Documentation more in depth explanation with examples are provided. It is useful to understand the behavior to make your application more efficient.
I'm trying to achive an Aggregation Query and that's my code:
TicketGroup.objects.filter(event=event).aggregate(
total_group=Sum(F('total_sold')*F('final_price')))
I have 'total_sold' and 'final_price' in TicketGroup object and all what I want to do is sum and multiply values to get the total sold of all TicketGroups together.
All I get is this error:
Expression contains mixed types. You must set output_field
What I am doing wrong, since I'm calling 'total_group' as my output field?
Thanks!
By output_field Django means to provide field type for the result of the Sum.
from django.db.models import FloatField, F
total_group=Sum(F('total_sold')*F('final_price'), output_field=FloatField())
should do the trick.
I had to use something different in order to make my query work. Just output_field wont solve it. I needed a simple division between two aliases. These are output of two annotations.
from django.db.models import FloatField, ExpressionWrapper, F
distinct_people_with_more_than_zero_bill = Task.objects.filter(
billable_efforts__gt=0).values('report__title').annotate(
Count('assignee', distinct=True)).annotate(
Sum('billable_efforts'))
annotate(yy=ExpressionWrapper(F('billable_efforts__sum') / F('assignee__count'), output_field=FloatField()))
The key here is ExpressionWrapper.
Without this, you will get an error: received non-expression(s)
The hint came for Django documentation itself, which says:
If the fields that you’re combining are of different types you’ll need
to tell Django what kind of field will be returned. Since F() does not
directly support output_field you will need to wrap the expression
with ExpressionWrapper
Link: https://docs.djangoproject.com/en/2.2/ref/models/expressions/
Using Django model syntax, if I do this:
ThatModel.objects.filter(
last_datetime__lte=now + datetime.timedelta(seconds=F("interval")))
I get:
TypeError: unsupported type for timedelta days component: ExpressionNode
Is there a way to make this work with pure Django syntax (and not parsing all the results with Python)?
Just avoid timedelta's F-ignorance
filter knows about F, but timedelta does not.
The trick is to keep the F out of the timedelta argument list:
ThatModel.objects.filter(
last_datetime__lte=now + datetime.timedelta(seconds=1)*F("interval"))
This will work with PostgreSQL, but, alas, not with SQlite.
From django docs:
Django provides F expressions to allow such comparisons. Instances
of F() act as a reference to a model field within a query. These
references can then be used in query filters to compare the values of
two different fields on the same model instance.
That means you can use F() for comparing within queries. F() returns reference so when you use it as parameter for timedelta object, you get the error ExpressionNode. You can check the documentation. You might check the source code of F()
For your solution, you can check this: DateModifierNode, or just save the value of interval elsewhere and then pass it as parameter of timedelta.
I have a class Image with the following GenericRelation:
properties = models.GenericRelation(Property)
I'm trying to get all Images with certain properties, so I do this:
Image.objects.filter(properties__type = "foo", properties__user = request.user)
But this results in the following error:
DatabaseError: operator does not exist: integer = text
LINE 1: ...perties_property" ON ("myapp_image"."id" = "propert...
^
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.
Is it not possible to query that way? What can I do as an alternative?
If you need to do very complex queries over generic relations, then i would rather suggest you write the sql yourself and use raw queries rather. So i guess i would say raw query is an alternative.