Translating MySql query into Django ORM query - django

Question

I have a query in MySql that I need translated into Django ORM. It involves joining on two tables with two counts on one of the tables. I'm pretty close to it in Django but I get duplicate results. Here's the query:
SELECT au.id,
au.username,
COALESCE(orders_ct, 0) AS orders_ct,
COALESCE(clean_ct, 0) AS clean_ct,
COALESCE(wash_ct, 0) AS wash_ct
FROM auth_user AS au
LEFT OUTER JOIN
( SELECT user_id,
Count(*) AS orders_ct
FROM `order`
GROUP BY user_id
) AS o
ON au.id = o.user_id
LEFT OUTER JOIN
( SELECT user_id,
Count(CASE WHEN service = 'clean' THEN 1
END) AS clean_ct,
Count(CASE WHEN service = 'wash' THEN 1
END) AS wash_ct
FROM job
GROUP BY user_id
) AS j
ON au.id = j.user_id
ORDER BY au.id DESC
LIMIT 100 ;
My current Django query (which brings back unwanted duplicates):
User.objects.annotate(
orders_ct = Count( 'orders', distinct = True )
).annotate(
clean_ct = Count( Case(
When( job__service__exact = 'clean', then = 1 )
) )
).annotate(
wash_ct = Count( Case(
When( job__service__exact = 'wash', then = 1 )
) )
)
The above Django code produces the following query which is close but not right:
SELECT DISTINCT `auth_user`.`id`,
`auth_user`.`username`,
Count(DISTINCT `order`.`id`) AS `orders_ct`,
Count(CASE
WHEN `job`.`service` = 'clean' THEN 1
ELSE NULL
end) AS `clean_ct`,
Count(CASE
WHEN `job`.`service` = 'wash' THEN 1
ELSE NULL
end) AS `wash_ct`
FROM `auth_user`
LEFT OUTER JOIN `order`
ON ( `auth_user`.`id` = `order`.`user_id` )
LEFT OUTER JOIN `job`
ON ( `auth_user`.`id` = `job`.`user_id` )
GROUP BY `auth_user`.`id`
ORDER BY `auth_user`.`id` DESC
LIMIT 100
I could probably achieve it by doing some raw sql subqueries but I would like to remain as abstract as possible.

Answer 1

Based on this answer, you can write:
User.objects.annotate(
orders_ct = Count( 'orders', distinct = True ),
clean_ct = Count( Case(
When( job__service__exact = 'clean', then = F('job__pk') )
), distinct = True ),
wash_ct = Count( Case(
When( job__service__exact = 'wash', then = F('job__pk') )
), distinct = True )
)
Table (after joins):
user.id order.id job.id job.service your case/when my case/when
1 1 1 wash 1 1
1 1 2 wash 1 2
1 1 3 clean NULL NULL
1 1 4 other NULL NULL
1 2 1 wash 1 1
1 2 2 wash 1 2
1 2 3 clean NULL NULL
1 2 4 other NULL NULL
Desired output for wash_ct is 2. Counting distinct values in my case/when, we will get 2.

Answer 2

I think this will work, the chained annotation of job might have produced duplicate users.
If not can you elaborate on duplicates you are seeing.
User.objects.annotate(
orders_ct = Count( 'orders', distinct = True )
).annotate(
clean_ct = Count( Case(
When( job__service__exact = 'clean', then = 1 )
) ),
wash_ct = Count( Case(
When( job__service__exact = 'wash', then = 1 )
) )
)

Answer 3

Try adding values(), also when distinct=True you can combine Count()'s in one annotation().
Users.objects.values("id").annotate(
orders_ct = Count('orders', distinct = True)
).annotate(
clean_ct = Count(Case(When(job__service__exact='clean', then=1)),
distinct = True),
wash_ct = Count(Case(When(job__service__exact='wash',then=1)),
distinct = True)
).values("id", "username", "orders_ct", "clean_ct", "wash_сt")
Using values("id") should add GROUP BY 'id' for annotations and therefore prevent duplicates, see docs.
Also, there's Coalesce, but it doesn't look like it's needed, since Count() returns int anyway. And distinct, but again the distinct in Count() should be enough.
Not sure if Case needed inside Count() as it should count them anyway.