I have made a loop to retrieve conditions:
for level in levels:
requete += ' and level_concat like %'+level+'%'
and i made in my query:
countries = Country.objects.extra(where=['continent_id = "'+continent_id+'"', requete])
I have tried to add condition to my where clause, but return error:
not enough arguments for format string
Respected results:
SELECT * FROM `Country` WHERE country_id = "US-51" AND level_concat LIKE %level_test%
Is there a way to add requete to my 'where' clause?
Firstly, it is not a good practice to keep data in a relational database as a "list of [values] concatenated by coma" - you should create a new table for those values.
Still, even now you can use filter(), instead of extra() (which should be always your last resort - I don't see the rest of your code, but if you don't properly escape levels values you may even be introducing an SQL Injection vulnerability here).
An example of a secure, extra()-less code, that does the exact same thing:
from django.db.models import Q
q = Q()
for level in levels:
q &= Q(level_concat__contains=level)
countries = Country.objects.filter(q)
or the same functionality, but in even less number of lines:
from django.db.models import Q
q = (Q(level_concat__contains=l) for l in levels)
countries = Country.objects.filter(*q)
You can read more about Q object in Django docs.
I think you need to escape the % sign in your query:
' and level_concat like %%'+level+'%%'
Related
I have the following queryset:
photos = Photo.objects.all()
I filter out two queries:
a = photos.filter(gallery__name='NGL')
b = photos.filter(gallery__name='NGA')
I add them together, and they form one new, bigger queryset:
c = a | b
Indeed, the length of a + b equals c:
a.count() + b.count() == c.count()
>>> True
So far so good. Yet, if I introduce a .annotate(), the | no longer seems to work:
a = photos.annotate(c=Count('label').exclude(c__lte=4)
b = photos.filter(painting=True)
c = a | b
a.count() + b.count() == c.count()
>>> False
How do I combine querysets, even when .annotate() is being used? Note that query one and two both work as intended in isolation, only when combining them using | does it seem to go wrong.
the pipe | or ampersand & to combine querysets actually puts OR or AND to SQL query so it looks like combined.
one = Photo.objects.filter(id=1)
two = Photo.objects.filter(id=2)
combined = one | two
print(combined.query)
>>> ... WHERE ("photo_photo"."id" = 1 OR "photo_photo"."id" = 2)...
But when you combine more filters and excludes you may notice it will give you strange results due to this. So that is why it doesn't match when you compare counts.
If you use .union() you have to have same columns with same data type, so you have to annotate both querysets. Info about .union()
SELECT statement within .UNION() must have the same number of columns
The columns must also have similar data types
The columns in each SELECT statement must also be in the same order
You have to keep in mind, that pythons argument kwargs for indefinite number of arguments are dictionary, so if you want to use annotate with multiple annotations, you can't ensure correct order of columns. Fortunatelly you can solve this with chaining annotate commands.
# printed query of this won't be consistent
photo_queryset.annotate(label_count=Count('labels'), tag_count=Count('tags'))
# this will always have same order of columns
photo_queryset.annotate(label_count=Count('labels')).annotate(tag_count=Count('tags'))
Then you can use .union() and it won't mess up results of annotation. Also .union() should be last method, because after .union() you can't use filter like methods. If you want to preserve duplicates, you use .union(qs, all=True) since .union() has default all=False and calls DISTINCT on queryset
photos = Photo.objects.annotate(c=Count('labels'))
one = photos.exclude(c__lte=4)
two = photos.filter(painting=True)
all = one.union(two, all=True)
one.count() + two.count() == all.count()
>>> True
then it should work like you described in question
i have 2 queries which i have to join using '|' and apply the final one to get
the result.First i have list of countries i.e eu countries.and second i have a country from which the user is logged in.I want to join both ones and get the result using Q.First one is q= Q(country=auth.country)and second one is
q2 = Q(country = settings.euCountries), in which settings.enCountries is a django list.Any help in this matter is highly appreciated
If you need an OR for all countries
q = [Q(country=auth.country)] + [Q(country=i) for i in settings.euContries]
Then
import operator
Model.objects.filter(reduce(operator.or_, q))
I don't think you need multiple Q() objects here. You can use the __in lookup.
Q(country_in=[auth.country] + settings.euCountries)
Depending on your code, you might not need the Q object at all if you can do something like the following:
queryset = MyModel.objects.filter(country_in=[auth.country] + settings.euCountries)
I have the following query:
>>> z = Restaurant.objects.values_list('city',flat=True).order_by('city').distinct()
>>> z
[u'ELURU', u'Eluru', u'Hyderabad']
As you can see, it is not completely distinct because of the case sensitivity. How do i correct this issue?
You can use annotate in conjunction with Lower (or Upper, etc...) to normalize your values and return truly distinct values like this...
from django.db.models.functions import Lower
z = Restaurant.objects.annotate(
city_lower=Lower('city')).values_list(
'city_lower',flat=True).order_by('city_lower').distinct()
Note: Make sure order_by is set to 'city_lower' and not 'city' to avoid duplicates.
I'm not sure you're going to find a solution to this since django doesn't offer a case-insensitive distinct method (currently). But then maybe it would be better to fix the values in your database anyway since you don't really want your end users to see their city in capitals since it will look ugly.
I'd suggest thinking about making a simple method that you could run either once in a data migration and stopping the city field from ever getting in this state again - or just running this periodically.
something similar to
for restaurant in Restaurant.objects.all():
if restaurant.city != restaurant.city.title():
restaurant.city = restaurant.city.title()
restaurant.save()
Try this;
z = Restaurant.objects.extra(select = {'tmp_city': lower('city')}).values_list('city',flat=True).order_by('city').distinct('tmp_city')
This works, although it is a little messy. I ended up having to use values, since distinct only works on database tables, regardless of whether or not you use annotate, extra, or rawSQL.
You end up creating an extra field with annotate, and then use that field in your list of dictionaries created by values. Once you have that list of dictionaries, you can use groupby to group dictionaries based on the Lower values key in the values list of dicts. Then, depending on how you want to select the object (in this case, just taking the first object of the group), you can select the version of the distinct that you want.
from django.db.models.functions import Lower
from itertools import groupby
restaurant = [g.next() for k, g in groupby(
list(
Restaurant.objects.annotate(city_lower=Lower('message_text')).values_list('city', flat=True)
).order_by('city').values('city_lower', 'city')
), lambda x: x['city_lower'])]
I want to retrieve a sum of two fields (which are aggregations themselves) for each object in a table.
The following may describe a bit better what I'm after but results in an Unknown column in field list-Error:
items = MyModel.objects.annotate(
field1=Sum("relatedModel__someField"),
field2=Sum("relatedModel__someField")).extra(
select={"sum_field1_field2": "field1 + field2"})
I also tried using F() for the field lookups but that gives me an invalid sql statement.
Any ideas on how to solve this are much appreciated.
it this what you want?
items = MyModel.objects.extra(
select = {'sum_field1_field2': 'SUM(relatedModel__someField) + SUM(relatedModel__someField)'},
)
To make it work for many to many or for many to one (reverse) relations, you may use the following:
items = MyModel.objects.extra(
select = {'sum_field1_field2': 'SUM("relatedModel"."someField") + SUM("relatedModel"."someField")'},
)
But this will break also if you need another annotate, like for a count, because extra will add the statement to the GROUP BY clause, whereas aggregate functions are not allowed in there.
I have first_name, last_name & alias (optional) which I need to search for. So, I need a query to give me all the names that have an alias set.
Only if I could do:
Name.objects.filter(alias!="")
So, what is the equivalent to the above?
You could do this:
Name.objects.exclude(alias__isnull=True)
If you need to exclude null values and empty strings, the preferred way to do so is to chain together the conditions like so:
Name.objects.exclude(alias__isnull=True).exclude(alias__exact='')
Chaining these methods together basically checks each condition independently: in the above example, we exclude rows where alias is either null or an empty string, so you get all Name objects that have a not-null, not-empty alias field. The generated SQL would look something like:
SELECT * FROM Name WHERE alias IS NOT NULL AND alias != ""
You can also pass multiple arguments to a single call to exclude, which would ensure that only objects that meet every condition get excluded:
Name.objects.exclude(some_field=True, other_field=True)
Here, rows in which some_field and other_field are true get excluded, so we get all rows where both fields are not true. The generated SQL code would look a little like this:
SELECT * FROM Name WHERE NOT (some_field = TRUE AND other_field = TRUE)
Alternatively, if your logic is more complex than that, you could use Django's Q objects:
from django.db.models import Q
Name.objects.exclude(Q(alias__isnull=True) | Q(alias__exact=''))
For more info see this page and this page in the Django docs.
As an aside: My SQL examples are just an analogy--the actual generated SQL code will probably look different. You'll get a deeper understanding of how Django queries work by actually looking at the SQL they generate.
Name.objects.filter(alias__gt='',alias__isnull=False)
Firstly, the Django docs strongly recommend not using NULL values for string-based fields such as CharField or TextField. Read the documentation for the explanation:
https://docs.djangoproject.com/en/dev/ref/models/fields/#null
Solution:
You can also chain together methods on QuerySets, I think. Try this:
Name.objects.exclude(alias__isnull=True).exclude(alias="")
That should give you the set you're looking for.
1. When using exclude, keep the following in mind to avoid common mistakes:
Should not add multiple conditions into an exclude() block like filter(). To exclude multiple conditions, you should use multiple exclude().
Example: (NOT a AND NOT b)
Entry.objects.exclude(title='').exclude(headline='')
equal to
SELECT... WHERE NOT title = '' AND NOT headline = ''
======================================================
2. Only use multiple when you really know about it:
Example: NOT (a AND b)
Entry.objects.exclude(title='', headline='')
equal to
SELECT.. WHERE NOT (title = '' AND headline = '')
If you want to exclude null (None), empty string (""), as well as a string containing white spaces (" "), you can use the __regex along with __isnull filter option
Name.objects.filter(
alias__isnull = False,
alias__regex = r"\S+"
)
alias__isnull=False excludes all the columns null columns
aliax__regex = r"\S+" makes sure that the column value contains at least one or more non whitespace characters.
From Django 1.8,
from django.db.models.functions import Length
Name.objects.annotate(alias_length=Length('alias')).filter(alias_length__gt=0)
You can simply do this:
Name.objects.exclude(alias="").exclude(alias=None)
It's really just that simple. filter is used to match and exclude is to match everything but what it specifies. This would evaluate into SQL as NOT alias='' AND alias IS NOT NULL.
Another approach using a generic isempty lookup, that can be used with any field.
It can also be used by django rest_framework or other apps that use django lookups:
from distutils.util import strtobool
from django.db.models import Field
from django.db.models.lookups import BuiltinLookup
#Field.register_lookup
class IsEmpty(BuiltinLookup):
lookup_name = 'isempty'
prepare_rhs = False
def as_sql(self, compiler, connection):
sql, params = compiler.compile(self.lhs)
condition = self.rhs if isinstance(self.rhs, bool) else bool(strtobool(self.rhs))
if condition:
return "%s IS NULL or %s = ''" % (sql, sql), params
else:
return "%s <> ''" % sql, params
You can then use it like this:
Name.objects.filter(alias__isempty=False)
this is another simple way to do it .
Name.objects.exclude(alias=None)