my problem is in django orderby clause.i have used two queryset one is state_filter and another is party_filter. without order by clause append the value correctly .using order_by clause then count value is not correctly
My view is:
def const(request):
states = Loksabha.objects.values('state_name').distinct('state_name')
constituency=Loksabha.objects.values('constituency_name').distinct('constituency_name').filter(state_name='MAHARASHTRA')
dataset = Loksabha.objects.all()
state_filter=Loksabha.objects.filter(state_name='MAHARASHTRA',constituency_name='Kolhapur').order_by('id')
party_filter=state_filter.values('party_name').annotate(Count('party_name'))
crime_filter=childcrime_type.objects.filter(state_name='MAHARASHTRA')
womencrime_fltr=womencrime_type.objects.filter(state='MAHARASHTRA')
xdata=[]
ydata=[]
for b in party_filter:
xdata.append(b['party_name'])
ydata.append(b['party_name__count'])
without order_by clause answer is:
[
{'party': 'Shiv Sena', 'party__count': 2},
{'party': 'Indian Nationlist Congress', 'party__count': 3},
{'party': 'Nationlist Congress Party', 'party__count': 1},
{'party': 'Republican Party of India(A)', 'party__count': 2},
{'party': 'Bharatiya Janata Party', 'party__count': 1},
{'party': 'Independent', 'party__count': 2}
]
with order_by clause answer is:
[
{'party': 'Shiv Sena', 'party__count': 1},
{'party': 'Shiv Sena', 'party__count': 1},
{'party': 'Indian Nationlist Congress', 'party__count': 1},
{'party': 'Indian Nationlist Congress', 'party__count': 1},
{'party': 'Indian Nationlist Congress', 'party__count': 1},
{'party': 'Nationlist Congress Party', 'party__count': 1},
{'party': 'Republican Party of India(A)', 'party__count': 1},
{'party': 'Republican Party of India(A)', 'party__count': 1},
{'party': 'Bharatiya Janata Party', 'party__count': 1},
{'party': 'Independent', 'party__count': 1},
{'party': 'Independent', 'party__count': 1}
]
plese give me proper solution
When you execute this :
party_filter=state_filter.values('party_name').annotate(Count('party_name'))
It will perform aggregation . so you get total count according to party_name
Now when you execute this :
party_filter=state_filter.values('party_name').annotate(Count('party_name')).order_by('party_name')
It will perform group by .
Note : aggregate is for the complete resultset, annotate for individual (grouped) rows.
Related
I have a model which which has fields like:
class Vehicle(models.Model):
car_name = models.CharField(max_length=255)
car_color = models.CharField(max_length=255)
This model has a lot of duplicate values too, I would like distinct to be shown
The queryset gives an output like:
<QuerySet [{'car_name': 'Audi', 'car_color': 'Red'}, {car_name': 'BMW', 'car_color': 'white'}]>
[edit]I want my output Queryset to have another field, which is like a counter for the ouput. If I get two objects out of the query, then a field called ordinal_count also be sent in queryset.
The output I am looking for example:
<QuerySet [{'car_name': 'Audi', 'car_color': 'Red', 'ordinal_count': 1}, {car_name': 'BMW', 'car_color': 'white', 'ordinal_count': 2}, {car_name': 'Jaguar', 'car_color': 'olive green', 'ordinal_count': 3}]>
This is the query which I wrote:
op = (Vehicle.objects.annotate(ordinal_count=Count("car_name", distinct="car_name")).filter(car_color='white', list_display=True).values("car_name","car_color", "ordinal_count"))
This is not giving me the desired input and is also messing up the filter. Should I even be using annotate?
Then I also wrote another query but it fails to run:
NotImplementedError: annotate() + distinct(fields) is not implemented.
Query is:
count = Vehicle.objects.filter(car_color='white', list_display=True).distinct(
"car_name").annotate(ordinal_count=Count("car_name")).values_list(
"car_name","car_color", "ordinal_count")
[edit 1] Tried the solution by #trigo, but it gives me an output like(where ordinal_count remains 2). I want ordinal_count to be like a counter for the objects which come in a queryset:
<QuerySet [{'car_name': 'Audi', 'car_color': 'Red', 'ordinal_count': 2}, {car_name': 'BMW', 'car_color': 'white', 'ordinal_count': 2}, {car_name': 'Jaguar', 'car_color': 'olive green', 'ordinal_count': 2}]>
[Update]:
qs = (
Vehicle.objects.filter(car_color="white", list_display=True)
.distinct("car_name")
.annotate(ordinal_count=Window(expression=RowNumber(), partition_by=None))
.values("slug", "car_name", "car_color","ordinal_count")
).order_by('car_name')
the only issue here is annotate works before filtering. So the ordinal_count is messing up. Is there a way to be able to filter first?
edit3
Suppose there are 5 objects in total and I have said I want all white color cars, with distinct car_name.
The output after applying filter, it seems like annotation happens after filter, because of the jumbled up ordinal_count:
<QuerySet [{'car_name': 'Audi', 'car_color': 'white', 'ordinal_count': 3}, {car_name': 'BMW', 'car_color': 'white', 'ordinal_count': 2}, {car_name': 'Jaguar', 'car_color': 'white', 'ordinal_count': 5}]>
I do not have a complete example but there is a Window expression (inside you can use RowNumber()) available in django models that could help you:
from django.db.models.expressions import Window
from django.db.models.functions import RowNumber
query = Vehicle.objects.values(...).annotate(row_number=Window(expression=RowNumber()))
https://docs.djangoproject.com/en/4.1/ref/models/expressions/#window-functions
[Assumption] : By duplicate you mean only on the attribute "car_name". However, if you mean duplicate on "car_name" and "color" combined, below queryset shall not work. Let me know if this is the case, shall provide other solution for it.
The below queryset shall give you your desired output.
Vehicle.objects.filter().values("car_name").annotate(ordinal_count=Count("id")).order_by("car_name")
[Note]: If your database is PostgreSQL, you can use distinct() however in other database this is hacky way to achieve the same.
In complex queries, you would need to write a sql query directly.
[EDIT 1]
Ok, so basically ordinal_count is NOT the number of times the car_name has occurred but just the counter. Right?
As per your expected output which you provided, i.e.
<QuerySet [{'car_name': 'Audi', 'car_color': 'Red', 'ordinal_count': 1}, {car_name': 'BMW', 'car_color': 'white', 'ordinal_count': 2}, {car_name': 'Jaguar', 'car_color': 'olive green', 'ordinal_count': 3}]>
As per your sample output above, by ordinal_count you only mean the counter (i.e.) index of the array + 1 ? That is the only conclusion I can reach as per your example. For this you may not need to annotate at all.
from django.db.models import Count, Window, F
Vehicle.objects.distinct('car_name').annotate(ordinal_count=Window(
expression=Count("id"),
partition_by=[F("car_name")]
)).values(
car_name,
car_color,
ordinal_count
).order_by('ordinal_count')
This query results following
[{"car_name": "BMW", "car_color": "black", "ordinal_count": 5}...]
I have a queryset of objects that could have any text value in their field and I would like a dictionary of the counts of the values in that field. For example:
I have a Test Drives Object which has a foreign key to vehicle, which has a make (Char field). I would like to know how many Test Drives each make had. I have actually solved it already, but I wonder if there is a better way using Django's inbuilt functionality.
My existing, working solution:
customer_test_drives_makes = customer_test_drives.values_list('vehicle__make', flat=True).order_by('vehicle__make').distinct()
customer_test_drives_makes_dictionary = {}
for make in customer_test_drives_makes :
customer_test_drives_makes_dictionary[make] = customer_test_drives.filter(vehicle__make=make).count()
print(customer_test_drives_makes_dictionary)
This prints:{'BMW': 1, 'Honda': 1, 'Hyundai': 1, 'Mazda': 2} Which is correct
There is. Try group by using annotate:
from django.db.models import Count
customer_test_drives.values('vehicle__make').annotate(count=Count('vehicle__make')).values()
Another example: Suppose your model looks like this:
class Cars(models.Model):
vehicle = models.ForeignKey(Vehicle, on_delete=models.DO_NOTHING)
Then you can run this queryset as well:
Vehicle.objects.annotate(car_count=Count('cars'))
Ruddra's answer got me most of the way, but I had to make the values distinct. Here are the final solutions:
Pre-existing way:
queryset_makes = queryset.values_list('vehicle__make', flat=True).order_by('vehicle__make').distinct()
queryset_makes_dictionairy = {}
for make in queryset_makes :
queryset_makes_dictionairy[make] = queryset.filter(vehicle__make=make).count()
print(queryset_makes_dictionairy)
Result:
{'BMW': 1, 'Honda': 4, 'Hyundai': 1, 'Mazda': 2, 'Mitsubishi': 1, 'Nissan': 2}
------------
Updated Way:
print(queryset.values('vehicle__make').order_by('vehicle__make').distinct().annotate(count=Count('vehicle__make')))
Result:
<QuerySet [{'vehicle__make': 'BMW', 'count': 1}, {'vehicle__make': 'Honda', 'count': 4}, {'vehicle__make': 'Hyundai', 'count': 1}, {'vehicle__make': 'Mazda', 'count': 2}, {'vehicle__make': 'Mitsubishi', 'count': 1}, {'vehicle__make': 'Nissan', 'count': 2}]>
I'm finding that while nulls_last=True works, nulls_last=False doesn't. Example below is in a Django shell.
In [10]: [x.date for x in Model.objects.all().order_by(F('date').asc(nulls_last=True))]
Out[10]:
[datetime.datetime(2020, 3, 10, 16, 58, 7, 768401, tzinfo=<UTC>),
datetime.datetime(2020, 3, 10, 17, 4, 51, 601980, tzinfo=<UTC>),
None,
]
[ins] In [11]: [x.last_run_created_at for x in Model.objects.all().order_by(F('date').asc(nulls_last=False))]
Out[11]:
[datetime.datetime(2020, 3, 10, 16, 58, 7, 768401, tzinfo=<UTC>),
datetime.datetime(2020, 3, 10, 17, 4, 51, 601980, tzinfo=<UTC>),
None,
]
In [12]:
I've tried this with both desc() and asc().
The mistake is assuming that the opposite of nulls_last=True is nulls_last=False. It isn't.
nulls_last=True does the following to the query:
SELECT ... ORDER BY ... ASC NULLS LAST
Whereas nulls_last=False just means use the DB default:
SELECT ... ORDER BY ... ASC
What you want instead is to use nulls_first=True OR nulls_last=True to explicitly get the order you want.
This is mentioned in the docs, but perhaps not as explicitly as it could be:
Using F() to sort null values
Use F() and the nulls_first or
nulls_last keyword argument to Expression.asc() or desc() to control
the ordering of a field’s null values. By default, the ordering
depends on your database.
I am new to Django. I am trying to make this query return count by a group. But it doesn't group data.
notification = AppointmentNotificationGroupAppointment.objects.filter(receiver__notification_group__group=group).values('receiver__notification_group__group', 'sender__status__name').annotate(pcount=Count('sender__status__name', distinct=True))
It returns:
{'receiver__notification_group__group': '841536_123856', 'sender__status__name': 'Pending', 'pcount': 1},
{'receiver__notification_group__group': '841536_123856', 'sender__status__name': 'Pending', 'pcount': 1},
{'receiver__notification_group__group': '841536_123856', 'sender__status__name': 'Confirmed', 'pcount': 1},
{'receiver__notification_group__group': '841536_123856', 'sender__status__name': 'Confirmed', 'pcount': 1}
What am I doing wrong? I want it to return distinct records with them counted by group
You need to call the order_by(...) too
AppointmentNotificationGroupAppointment.objects.filter(receiver__notification_group__group=group).values(
'receiver__notification_group__group',
'sender__status__name').annotate(pcount=Count('sender__status__name', distinct=True)
).order_by('receiver__notification_group__group')
I want to get last 100 records of MyModel order_by('-end_date') and do a SUM annotate on different winner types them
MyModel.objects.all()[:100].order_by('-end_game_time').values('winner').annotate(total=Count('winner'))
result query is as below and I don't have expected groups
<QuerySet [{'winner': 3, 'total': 1}, {'winner': 15, 'total': 1}, 'total': 1}, {'winner': 3, 'total': 1}, {'winner': 5, 'total': 1}, {'winner': 15, 'total': 1}, {'winner': 5, 'total': 1}, {'winner': 3, 'total': 1}, '...(remaining elements truncated)...']>
generated query is like
SELECT "game_mymodel"."winner", COUNT("game_mymodel"."winner") AS "total" FROM "game_mymodel" GROUP BY "game_mymodel"."winner", "game_mymodel"."end_game_time" ORDER BY "game_mymodel"."end_game_time" DESC LIMIT 100
but when I don't have the order_by the result is as I expected
MyModel.objects.all()[:100].values('winner').annotate(total=Count('winner'))
Out[52]: <QuerySet [{'winner': 5, 'total': 43}, {'winner': 1, 'total': 2}, {'winner': 15, 'total': 51}, {'winner': 2, 'total': 42}, {'winner': 3, 'total': 43}]>
and generated query group_by part is different
SELECT "game_mymodel"."winner", COUNT("game_mymodel"."winner") AS "total" FROM "game_mymodel" GROUP BY "game_mymodel"."winner" LIMIT 100
As far as I know it is not possible to achieve what you want to do in single query, what you want in SQL is:
SELECT "game_mymodel"."winner", COUNT("game_mymodel"."winner") AS "total" FROM "game_mymodel" GROUP BY "game_mymodel"."winner" ORDER BY "game_mymodel"."end_game_time" DESC LIMIT 100
which is not a valid sql query, so you need to have a sub-query to select 100 elements and then apply your aggregation on them.
First build the sub-query:
top_100_games = MyModel.objects.order_by('-end_game_time')[:100].only('id').all()
And then use it in main query:
MyModel.objects.filter(id__in=top_100_games).values('winner').annotate(total=Count('winner'))