I need to get queryset, which is similar to this in SQL:
select * from kraj
where kraj_id in (select kraj_id from klient_kraj where klient_id = 1)
As you can see, I work with klient_kraj model, which is filtered and 1 column is returned kraj_id, which is then used for another filtering.
I wasn't able to find way, how to obtain this queryset using ORM.
Thanks
Related
I need to build a query using Django ORM, that looks like this one in SQL:
select * from A where id not in (select a_id from B where ... )
I try to use such code:
ids = B.objects.filter(...)
a_objects = A.object.exclude(id__in=Subquery(ids.values('a__id'))).all()
The problem is that instead of nested select Django generates query that looks like
select * from A where id not in (1, 2, 3, 4, 5 ....)
where in clause explicitly lists all ids that should be excluded, making result sql unreadable when it is printed into logs. Is it possible to adjst this query, so nested select is used?
So I see that your goal is to get all the A's that have no foreign key relations from B's. If I'm right, then you can just use inverse lookup to do it.
So, when you define models like that:
class A:
pass
class B:
a = ForeignKey(to=a, related_name='bs')
You can filter it like this:
A.objects.filter(bs__isnull=True)
Also, if you don't define related_name, it will default to b_set, so you will be able to A.objects.filter(b_set__isnull=True)
to make a filter on B you can
ids = B.objects.filter(x=x).values_list('id',flat=true)
you get a list of ids then make
a_objects = A.object.exclude(id__in=ids)
as mentioned before if there is a relation
You don't need to do anything special, just use the queryset directly in your filter.
ids = B.objects.filter(...)
a_objects = A.object.exclude(id__in=ids).all()
# that should generate the subquery statement
select * from A where NOT (id in (select a_id from B where ... ))
I am trying to annotate User with the count of delayed leads objects. The calculation of delayed leads is complex (uses RawSQL) implemented using a custom model manager. Hence, I am trying to implement this using a subquery.
sq = Lead.delayed.filter(assigned_to_id=OuterRef('pk'))
User.objects.annotate(num=Count(Subquery(sq.count())))
However, I keep getting this error:
ValueError: This queryset contains a reference to an outer query and may only be used in a subquery.
UPDATE:
I tried adding only('id') so my code:
sq = Lead.delayed.filter(assigned_to_id=OuterRef('id')).only('id')
User.objects.annotate(num=Count(Subquery(sq)))
This generated the sql query:
SELECT `auth_user`.`id`, `auth_user`.`username`, `auth_user`.`first_name`,
`auth_user`.`last_name`, COUNT((SELECT U0.`id` FROM
`lead` U0 WHERE U0.`assigned_to_id` = (`auth_user`.`id`))) AS `count`
FROM `auth_user` GROUP BY `auth_user`.`id`;
This is throwing error:
ERROR 1242 (21000): Subquery returns more than 1 row
I would like to have my query generated as:
SELECT `auth_user`.`id`, `auth_user`.`username`, `auth_user`.`first_name`,
`auth_user`.`last_name`, (SELECT COUNT(U0.`id`) FROM `marketing_lead` U0 WHERE
(more complex conditions here) U0.`assigned_to_id` = (`auth_user`.`id`)) AS `count`
FROM `auth_user` GROUP BY `auth_user`.`id`;
How can I acheive that using django ORM?
Alternative question label might be How to use Count() not perform grouping (GROUP BY) or How to count all in a Subquery
Check this answer for custom Count function to just perform simple count on any queryset without grouping.
Unfortunately, so far haven't found native django option for this.
Though the answer from Oleg was quite close to my requirement but I was still getting an SQL error on the query generated by django. Hence, I ended up implementing using cursor.
I am trying to gather additional information based on previously filtered data, like this
count(distinct batch_id) as batches,
sum(files) as files
The result is influenced by the previous filtering eg.
.filter(batch_id__gte=165)
I tried to clone the QuerySet and annotate aboves SQL
.annotate(
batches=Count('batch_id', distinct=True),
batch_files=Sum('files')
)
but this doesn't work because then the SQL is appended to the existing SELECT query
Is there an easy way of getting a second query with a custom SELECT part while keeping the WHERE part?
You probably want to aggregate(..) [Django-doc] instead, like:
result = Model.objects.filter(batch_id__gte=165).aggregate(
batches=Count('batch_id', distinct=True),
batch_files=Sum('files')
)
result is then a dictionary that contains two keys: 'batches' and 'batch_files' that contain the Count('batch_id', distinct=True) and Sum('files') respectively.
Annotating means that you will calculate properties, per object, not over the entire queryset, whereas aggregating means you calculate the aggregate over the entire (filtered) queryset.
I want to execute a simple query like:
select *,count('id') from menu_permission group by menu_id
In Django format I have tried:
MenuPermission.objects.all().values('menu_id').annotate(Count('id))
It selects only menu_id. The executed query is:
SELECT `menu_permission`.`menu_id`, COUNT(`menu_permission`.`id`) AS `id__count` FROM `menu_permission` GROUP BY `menu_permission`.`menu_id`
But I need other fields also. If I try:
MenuPermission.objects.all().values('id','menu_id').annotate(Count('id))
It adds 'id' in group by condition.
GROUP BY `menu_permission`.`id`
As a result I am not getting the expected result. How I can get all all fields in the output but group by a single one?
You can try subqueries to do what you need.
In my case I have two tables: Item and Transaction where item_id links to Item
First, I prepare Transaction subquery with group by item_id where I sum all amount fields and mark item_id as pk for outer query.
per_item_total=Transaction.objects.values('item_id').annotate(total=Sum('amount')).filter(item_id=OuterRef('pk'))
Then I select all rows from item plus subquery result as total filed.
items_with_total=Item.objects.annotate(total=Subquery(per_item_total.values('total')))
This produces the following SQL:
SELECT `item`.`id`, {all other item fields},
(SELECT SUM(U0.`amount`) AS `total` FROM `transaction` U0
WHERE U0.`item_id` = `item`.`id` GROUP BY U0.`item_id` ORDER BY NULL) AS `total` FROM `item`
You are trying to achieve this SQL:
select *, count('id') from menu_permission group by menu_id
But normally SQL requires that when a group by clause is used you only include those column names in the select that you are grouping by. This is not a django matter, but that's how SQL group by works.
The rows are grouped by those columns so those columns can be included in select and other columns can be aggregated if you want them to into a value. You can't include other columns directly as they may have more than one value (since the rows are grouped).
For example if you have a column called "permission_code", you could ask for an array of the values in the "permission_code" column when the rows are grouped by menu_id.
Depending on the SQL flavor you are using, this could be in PostgreSQL something like this:
select menu_id, array_agg(permission_code), count(id) from menu_permissions group by menu_id
Similary django queryset can be constructed for this.
Hopefully this helps, but if needed please share more about what you need to do and what your data models are.
The only way currently that it works as expected is to hve your query based on the model you want the GROUP BY to be based on.
In your case it looks like you have a Menu model (menu_id field foreign key) so doing this would give you what you want and will allow getting other aggregate information from your MenuPermission model but will only group by the Menu.id field:
Menu.objects.annotate(perm_count=Count('menupermission__id')).values('perm_count')
Of course there is no need for the "annotate" intermediate step if all you want is that single count.
query = MenuPermission.objects.values('menu_id').annotate(menu_id_count=Count('menu_id'))
You can check your SQL query by print(query.query)
This solution doesn't work, all fields end up in the group by clause, leaving it here because it may still be useful to someone.
model_fields = queryset.model._meta.get_fields()
queryset = queryset.values('menu_id') \
.annotate(
count=Count('id'),
**{field.name: F(field.name) for field in model_fields}
)
What i'm doing is getting the list of fields of our model, and set up a dictionary with the field name as key and an F instance with the field name as a parameter.
When unpacked (the **) it gets interpreted as named arguments passed into the annotate function.
For example, if we had a "name" field on our model, this annotate call would end up being equal to this:
queryset = queryset.values('menu_id') \
.annotate(
count=Count('id'),
name=F("name")
)
you can use the following code:
MenuPermission.objects.values('menu_id').annotate(Count('id)).values('field1', 'field2', 'field3'...)
I was wondering if it is possible to filter a queryset result with a queryset object attribute.
Example:
clients = Client.objects.filter(name__contains=search)
This should return several objects
result = Invoice.objects.filter(client_id=clients.id)
Now I want all the data inside Invoice that corresponds to the clients.id found.
What is the most optimized way to do it? Since Django is a powerful framework, I was wondering if it has a good and fast way to do it without me having to add the primary result to a list and do a for loop.
You can do this by filtering directly Invoices using lookups
result = Invoice.objects.filter(client__name__contains=search)
Alternatively, you can find all clients, extract ids, and filter invoices by those id.
clients = Client.objects.filter(**your_crazy_search).values_list('id', flat=True).all()
result = Invoices.objects.filter(client_id__in=clients_id)
You don't even need to extract ID from clients, that will work just fine:
clients = Client.objects.filter(name__contains=search)
result = Invoices.objects.filter(client__in=clients)
It will result in SQL query:
SELECT * FROM invoices WHERE result.client_id IN (SELECT `id` FROM `client` WHERE ...)