I have the following query
a = Mainfee.objects.values('collected_by__username').
distinct().annotate(Sum('amount'))
The result looks like this
[{'collected_by__username': u'maindesk', 'amount__sum': 800}]
How can I rename the first key to a and second key to b?
I tried the following
m = Mainfee.objects.extra(select =
{'a':'collected_by__username'}).values('a').distinct().
annotate(Sum('amount'))
and received this
DatabaseError: no such column: collected_by__username
I also tried
m = Mainfee.objects.extra(select =
{'a':'collected_by__username'}).values('collected_by__username').distinct().
annotate(Sum('amount'))
and got
[{'collected_by__username': u'maindesk', 'amount__sum': 800}]
PS: I want to rename the second field too
You can change the dictionary key of the annotated value by using keyword arguments:
m = Mainfee.objects.values('collected_by__username').annotate(b=Sum('amount'))
[{'collected_by__username': u'maindesk', 'b': 800}]
There is no quick and easy way to rename a related field, though. You can convert it in Python, but you'll have to ask yourself if that is really necessary:
m = Mainfee.objects.values('collected_by__username').annotate(b=Sum('amount'))
m = [{'a': x['collected_by__username'], 'b': x['b']} for x in m]
Or maybe this is slightly faster:
m = Mainfee.objects.values_list('collected_by__username').annotate(Sum('amount'))
m = [{'a': x[0], 'b': x[1]} for x in m]
Both methods will of course force evaluation of the whole queryset and don't allow further filtering/ordering etc. through the queryset's methods.
Also note that the call to distinct() is redundant when you're using values() or values_list() together with annotate().
You can use an F() expression on collected_by__username:
An F() object represents the value of a model field or annotated
column. It makes it possible to refer to model field values and
perform database operations using them without actually having to pull
them out of the database into Python memory... F() can be used to create dynamic fields on your models...
If you do this inside an annotate clause, we can 'rename' the field to a. You don't need to use an F() expression when calculating the total. See below:
from django.db.models import F
(Mainfee.objects.values('collected_by__username')
.annotate(a=F('collected_by__username'), b=Sum('amount'))
.values('a', 'b'))
Making some assumptions about your model, this would produce something like:
<QuerySet [{'a': 'item1', 'b': 1234}, {'a': 'item2', 'b': 4321}]>
I'm not sure how to rename the first key, it seems there is a solution here: How to rename items in values() in Django? but it returns error in your case since the generated SQL would be:
SELECT collected_by__username AS a WHERE ...
And of course collected_by__username doesn't exist.
Note: this feature is requested here https://code.djangoproject.com/ticket/16735
The 2nd key can be renamed by using: annotate(b=Sum('amount')
So try this first:
m = (Mainfee.objects
.extra(select={'a': 'collected_by.username'}) # check your DB for the exact table and field name
.values('a')
.distinct()
.annotate(b=Sum('amount')))
I guess it would spit out an exception but it worths a try. Otherwise don't try to rename the first key for now since it would be very hacky anway.
Hope it helps.
Just in case someone stumbles on this page.
m = Mainfee.objects.extra(select = 'a':'collected_by.username'}).values('collected_by__username','a').distinct().annotate(b=Sum('amount'))
Related
I have a filter which should return a queryset with 2 objects, and should have one different field. for example:
obj_1 = (name='John', age='23', is_fielder=True)
obj_2 = (name='John', age='23', is_fielder=False)
Both the objects are of same model, but different primary key. I tried usign the below filter:
qs = Model.objects.filter(name='John', age='23').annotate(is_fielder=F('plays__outdoor_game_role')=='Fielder')
I used annotate first time, but it gave me the below error:
TypeError: QuerySet.annotate() received non-expression(s): False.
I am new to Django, so what am I doing wrong, and what should be the annotate to get the required objects as shown above?
The solution by #ktowen works well, quite straightforward.
Here is another solution I am using, hope it is helpful too.
queryset = queryset.annotate(is_fielder=ExpressionWrapper(
Q(plays__outdoor_game_role='Fielder'),
output_field=BooleanField(),
),)
Here are some explanations for those who are not familiar with Django ORM:
Annotate make a new column/field on the fly, in this case, is_fielder. This means you do not have a field named is_fielder in your model while you can use it like plays.outdor_game_role.is_fielder after you add this 'annotation'. Annotate is extremely useful and flexible, can be combined with almost every other expression, should be a MUST-KNOWN method in Django ORM.
ExpressionWrapper basically gives you space to wrap a more complecated combination of conditions, use in a format like ExpressionWrapper(expression, output_field). It is useful when you are combining different types of fields or want to specify an output type since Django cannot tell automatically.
Q object is a frequently used expression to specify a condition, I think the most powerful part is that it is possible to chain the conditions:
AND (&): filter(Q(condition1) & Q(condition2))
OR (|): filter(Q(condition1) | Q(condition2))
Negative(~): filter(~Q(condition))
It is possible to use Q with normal conditions like below:
(Q(condition1)|id__in=[list])
The point is Q object must come to the first or it will not work.
Case When(then) can be simply explained as if con1 elif con2 elif con3 .... It is quite powerful and personally, I love to use this to customize an ordering object for a queryset.
For example, you need to return a queryset of watch history items, and those must be in an order of watching by the user. You can do it with for loop to keep the order but this will generate plenty of similar queries. A more elegant way with Case When would be:
item_ids = [list]
ordering = Case(*[When(pk=pk, then=pos)
for pos, pk in enumerate(item_ids)])
watch_history = Item.objects.filter(id__in=item_ids)\
.order_by(ordering)
As you can see, by using Case When(then) it is possible to bind those very concrete relations, which could be considered as 1) a pinpoint/precise condition expression and 2) especially useful in a sequential multiple conditions case.
You can use Case/When with annotate
from django.db.models import Case, BooleanField, Value, When
Model.objects.filter(name='John', age='23').annotate(
is_fielder=Case(
When(plays__outdoor_game_role='Fielder', then=Value(True)),
default=Value(False),
output_field=BooleanField(),
),
)
I have a django model with the fields name (string) and value (integer). Say I need to return a queryset of {name:..., value:...} objects, with each value doubled. What I am trying to do now is:
queryset.annotate(value=F('value') * 2)
However, django tells me that the field value already exists.
I also tried using extra:
queryset.annotate(value_double=F('value') * 2).extra(select={'value': 'value_double'})
but that also does not work since value_double is not a valid field.
Of course, in this case I could use something like queryset.extra(select={'value': 'value * 2'}), but I have some other, more complicated cases involving a lot of functions where I really don't want to write sql, but I'd rather find a solution in django. Again, in all of my cases, annotate works perfectly fine as long as I give the result a new name.
Say your model is XYZ with fields name and val. If you want val to contain val*2 use below queryset
x = XYZ.objects.values('name').annotate(val=F('val')*2)
print(x)
Result
<QuerySet [{'name': abc, 'val': 104},...............]
If you want queryset that return name,val and doubleval. You can use below query for same.
x = XYZ.objects.values('name','val',doubleval=F('val')*2)
print(x)
Result
<QuerySet [{'name': abc, 'val':52,'doubleval': 104},...............]
Hope this help.
I have the following query:
>>> z = Restaurant.objects.values_list('city',flat=True).order_by('city').distinct()
>>> z
[u'ELURU', u'Eluru', u'Hyderabad']
As you can see, it is not completely distinct because of the case sensitivity. How do i correct this issue?
You can use annotate in conjunction with Lower (or Upper, etc...) to normalize your values and return truly distinct values like this...
from django.db.models.functions import Lower
z = Restaurant.objects.annotate(
city_lower=Lower('city')).values_list(
'city_lower',flat=True).order_by('city_lower').distinct()
Note: Make sure order_by is set to 'city_lower' and not 'city' to avoid duplicates.
I'm not sure you're going to find a solution to this since django doesn't offer a case-insensitive distinct method (currently). But then maybe it would be better to fix the values in your database anyway since you don't really want your end users to see their city in capitals since it will look ugly.
I'd suggest thinking about making a simple method that you could run either once in a data migration and stopping the city field from ever getting in this state again - or just running this periodically.
something similar to
for restaurant in Restaurant.objects.all():
if restaurant.city != restaurant.city.title():
restaurant.city = restaurant.city.title()
restaurant.save()
Try this;
z = Restaurant.objects.extra(select = {'tmp_city': lower('city')}).values_list('city',flat=True).order_by('city').distinct('tmp_city')
This works, although it is a little messy. I ended up having to use values, since distinct only works on database tables, regardless of whether or not you use annotate, extra, or rawSQL.
You end up creating an extra field with annotate, and then use that field in your list of dictionaries created by values. Once you have that list of dictionaries, you can use groupby to group dictionaries based on the Lower values key in the values list of dicts. Then, depending on how you want to select the object (in this case, just taking the first object of the group), you can select the version of the distinct that you want.
from django.db.models.functions import Lower
from itertools import groupby
restaurant = [g.next() for k, g in groupby(
list(
Restaurant.objects.annotate(city_lower=Lower('message_text')).values_list('city', flat=True)
).order_by('city').values('city_lower', 'city')
), lambda x: x['city_lower'])]
I have made a loop to retrieve conditions:
for level in levels:
requete += ' and level_concat like %'+level+'%'
and i made in my query:
countries = Country.objects.extra(where=['continent_id = "'+continent_id+'"', requete])
I have tried to add condition to my where clause, but return error:
not enough arguments for format string
Respected results:
SELECT * FROM `Country` WHERE country_id = "US-51" AND level_concat LIKE %level_test%
Is there a way to add requete to my 'where' clause?
Firstly, it is not a good practice to keep data in a relational database as a "list of [values] concatenated by coma" - you should create a new table for those values.
Still, even now you can use filter(), instead of extra() (which should be always your last resort - I don't see the rest of your code, but if you don't properly escape levels values you may even be introducing an SQL Injection vulnerability here).
An example of a secure, extra()-less code, that does the exact same thing:
from django.db.models import Q
q = Q()
for level in levels:
q &= Q(level_concat__contains=level)
countries = Country.objects.filter(q)
or the same functionality, but in even less number of lines:
from django.db.models import Q
q = (Q(level_concat__contains=l) for l in levels)
countries = Country.objects.filter(*q)
You can read more about Q object in Django docs.
I think you need to escape the % sign in your query:
' and level_concat like %%'+level+'%%'
I want to do pretty much the same like in this ticket at djangoproject.com, but with some additonal formatting. From this query
>>> MyModel.objects.values('cryptic_value_name')
[{'cryptic_value_name': 1}, {'cryptic_value_name': 2}]
I want to get something like that:
>>> MyModel.objects.values(renamed_value='cryptic_value_name')
[{'renamed_value': 1}, {'renamed_value': 2}]
Is there another, more builtin way or do I have to do this manually?
From django>=1.8 you can use annotate and F object
from django.db.models import F
MyModel.objects.annotate(renamed_value=F('cryptic_value_name')).values('renamed_value')
Also extra() is going to be deprecated, from the django docs:
This is an old API that we aim to deprecate at some point in the future. Use it only if you cannot express your query using other queryset methods. If you do need to use it, please file a ticket using the QuerySet.extra keyword with your use case (please check the list of existing tickets first) so that we can enhance the QuerySet API to allow removing extra(). We are no longer improving or fixing bugs for this method.
Without using any other manager method (tested on v3.0.4):
from django.db.models import F
MyModel.objects.values(renamed_value=F('cryptic_value_name'))
Excerpt from Django docs:
An F() object represents the value of a model field or annotated
column. It makes it possible to refer to model field values and
perform database operations using them without actually having to pull
them out of the database into Python memory.
It's a bit hacky, but you could use the extra method:
MyModel.objects.extra(
select={
'renamed_value': 'cryptic_value_name'
}
).values(
'renamed_value'
)
This basically does SELECT cryptic_value_name AS renamed_value in the SQL.
Another option, if you always want the renamed version but the db has the cryptic name, is to name your field with the new name but use db_column to refer to the original name in the db.
I am working with django 1.11.6
( And the key:value pair is opposite to that of accepted answer )
This is how i am making it work for my project
def json(university):
address = UniversityAddress.objects.filter(university=university)
address = address.extra(select={'city__state__country__name': 'country', 'city__state__name': 'state', 'city__name': 'city'})
address = address.values('country', 'state', "city", 'street', "postal_code").get()
return address
Note that adding simultanous objects.filter().extra().values() is same as above.
Try passing as kwargs:
MyModel.objects.annotate(**{'A B C':F('profile_id')}).values('A B C')
In my case, there were spaces and other special characters included in the key of each value in the result set so this did the trick.
Its more than simple if you want to rename few fields of the mode.
Try
projects = Project.objects.filter()
projects = [{'id': p.id, 'name': '%s (ID:%s)' % (p.department, p.id)} for p in projects]
Here i do not have a name field in the table, but i can get that after tweaking a little bit.