Is there a negation filter available by default. The idea is that you can do the following in the django ORM:
model.objects.filter(field!=value)
How can I do that in tastypie if that is even possible. I tried:
someapi.com/resource/pk/?field__not=value
someapi.com/resource/pk/?field__!=value
someapi.com/resource/pk/?field!=value
And all of them given me errors.
Unfortunately there's not.
The problem is that Tastypie's ModelResource class uses the filter() method of the QuerySet only, i.e. it does not use exclude() which should be used for negative filters. There is no filter() field lookup that would mean negation though. The valid lookups are (after this SO post):
exact
iexact
contains
icontains
in
gt
gte
lt
lte
startswith
istartswith
endswith
iendswith
range
year
month
day
week_day
isnull
search
regex
iregex
However it shouldn't be so hard to implement the support for something like "__not_eq". All you need to do is to modify the apply_filters() method and separate filters with "__not_eq" from the rest. Then you should pass the first group to exclude() and the rest to filter().
Something like:
def apply_filters(self, request, applicable_filters):
"""
An ORM-specific implementation of ``apply_filters``.
The default simply applies the ``applicable_filters`` as ``**kwargs``,
but should make it possible to do more advanced things.
"""
positive_filters = {}
negative_filters = {}
for lookup in applicable_filters.keys():
if lookup.endswith( '__not_eq' ):
negative_filters[ lookup ] = applicable_filters[ lookup ]
else:
positive_filters[ lookup ] = applicable_filters[ lookup ]
return self.get_object_list(request).filter(**positive_filters).exclude(**negative_filters)
instead of the default:
def apply_filters(self, request, applicable_filters):
"""
An ORM-specific implementation of ``apply_filters``.
The default simply applies the ``applicable_filters`` as ``**kwargs``,
but should make it possible to do more advanced things.
"""
return self.get_object_list(request).filter(**applicable_filters)
should allow for the following syntax:
someapi.com/resource/pk/?field__not_eq=value
I haven't tested it. It could probably be written in more elegant way too, but should get you going :)
Another way to do this without code changes is to use a iregex with inverse matching
http://HOST/api/v1/resource/?format=json&thing__iregex=^((?!notThis).)*$
I've opened a bug for this and provided a simple solution here: https://github.com/toastdriven/django-tastypie/issues/524
It would probably be nicer to add the '!' character at the end of the field name as you've done in your question ...
Note on Gorneau's above answer: seems like this only works if you're not using a MySQL backend. See:
#1139 - Got error 'repetition-operator operand invalid' from regexp
I use exclude() to avoid some values. For example:
Person.filter(name="Tim").exclude(state="Down");
Related
I have a filter which should return a queryset with 2 objects, and should have one different field. for example:
obj_1 = (name='John', age='23', is_fielder=True)
obj_2 = (name='John', age='23', is_fielder=False)
Both the objects are of same model, but different primary key. I tried usign the below filter:
qs = Model.objects.filter(name='John', age='23').annotate(is_fielder=F('plays__outdoor_game_role')=='Fielder')
I used annotate first time, but it gave me the below error:
TypeError: QuerySet.annotate() received non-expression(s): False.
I am new to Django, so what am I doing wrong, and what should be the annotate to get the required objects as shown above?
The solution by #ktowen works well, quite straightforward.
Here is another solution I am using, hope it is helpful too.
queryset = queryset.annotate(is_fielder=ExpressionWrapper(
Q(plays__outdoor_game_role='Fielder'),
output_field=BooleanField(),
),)
Here are some explanations for those who are not familiar with Django ORM:
Annotate make a new column/field on the fly, in this case, is_fielder. This means you do not have a field named is_fielder in your model while you can use it like plays.outdor_game_role.is_fielder after you add this 'annotation'. Annotate is extremely useful and flexible, can be combined with almost every other expression, should be a MUST-KNOWN method in Django ORM.
ExpressionWrapper basically gives you space to wrap a more complecated combination of conditions, use in a format like ExpressionWrapper(expression, output_field). It is useful when you are combining different types of fields or want to specify an output type since Django cannot tell automatically.
Q object is a frequently used expression to specify a condition, I think the most powerful part is that it is possible to chain the conditions:
AND (&): filter(Q(condition1) & Q(condition2))
OR (|): filter(Q(condition1) | Q(condition2))
Negative(~): filter(~Q(condition))
It is possible to use Q with normal conditions like below:
(Q(condition1)|id__in=[list])
The point is Q object must come to the first or it will not work.
Case When(then) can be simply explained as if con1 elif con2 elif con3 .... It is quite powerful and personally, I love to use this to customize an ordering object for a queryset.
For example, you need to return a queryset of watch history items, and those must be in an order of watching by the user. You can do it with for loop to keep the order but this will generate plenty of similar queries. A more elegant way with Case When would be:
item_ids = [list]
ordering = Case(*[When(pk=pk, then=pos)
for pos, pk in enumerate(item_ids)])
watch_history = Item.objects.filter(id__in=item_ids)\
.order_by(ordering)
As you can see, by using Case When(then) it is possible to bind those very concrete relations, which could be considered as 1) a pinpoint/precise condition expression and 2) especially useful in a sequential multiple conditions case.
You can use Case/When with annotate
from django.db.models import Case, BooleanField, Value, When
Model.objects.filter(name='John', age='23').annotate(
is_fielder=Case(
When(plays__outdoor_game_role='Fielder', then=Value(True)),
default=Value(False),
output_field=BooleanField(),
),
)
I am using Django, with mongoengine. I have a model Classes with an inscriptions list, And I want to get the docs that have an id in that list.
classes = Classes.objects.filter(inscriptions__contains=request.data['inscription'])
Here's a general explanation of querying ArrayField membership:
Per the Django ArrayField docs, the __contains operator checks if a provided array is a subset of the values in the ArrayField.
So, to filter on whether an ArrayField contains the value "foo", you pass in a length 1 array containing the value you're looking for, like this:
# matches rows where myarrayfield is something like ['foo','bar']
Customer.objects.filter(myarrayfield__contains=['foo'])
The Django ORM produces the #> postgres operator, as you can see by printing the query:
print Customer.objects.filter(myarrayfield__contains=['foo']).only('pk').query
>>> SELECT "website_customer"."id" FROM "website_customer" WHERE "website_customer"."myarrayfield_" #> ['foo']::varchar(100)[]
If you provide something other than an array, you'll get a cryptic error like DataError: malformed array literal: "foo" DETAIL: Array value must start with "{" or dimension information.
Perhaps I'm missing something...but it seems that you should be using .filter():
classes = Classes.objects.filter(inscriptions__contains=request.data['inscription'])
This answer is in reference to your comment for rnevius answer
In Django ORM whenever you make a Database call using ORM, it will generally return either a QuerySet or an object of the model if using get() / number if you are using count() ect., depending on the functions that you are using which return other than a queryset.
The result from a Queryset function can be used to implement further more refinement, like if you like to perform a order() or collecting only distinct() etc. Queryset are lazy which means it only hits the database when they are actually used not when they are assigned. You can find more information about them here.
Where as the functions that doesn't return queryset cannot implement such things.
Take time and go through the Queryset Documentation more in depth explanation with examples are provided. It is useful to understand the behavior to make your application more efficient.
I've followed django tutorial and arrived at tutorial05.
I tried to not show empty poll as tutorial says, so I added filter condition like this:
class IndexView(generic.ListView):
...
def get_queryset(self):
return Question.objects.filter(
pub_date__lte=timezone.now(),
choice__isnull=False
).order_by('-pub_date')[:5]
But this returned two objects which are exactly same.
I think choice__isnull=False caused the problem, but not sure.
choice__isnull causes the problem. It leads to join with choice table (to weed out questions without choices), that is something like this:
SELECT question.*
FROM question
JOIN choice
ON question.id = choice.question_id
WHERE question.pub_date < NOW()
You can inspect query attribute of QuerySet to be sure. So if you have one question with two choices, you will get that question two times. You need to use distinct() method in this case: queryset.distinct().
Just use .distinct() at the end of your ORM.
A little late to the party, but I figured it could help others looking up the same issue.
Instead of using choice__isnull=False with the filter() method, use it with exclude() instead to exclude out any questions without any choices. So your code would look something like this:
...
def get_queryset(self):
return Question.objects.filter(pub_date__lte=timezone.now()).exclude(choice__isnull=True).order_by('-pub_date')[:5]
By doing it this way, it will return only one instance of the question. Be sure to use choice_isnull=True though.
Because you created two objects with same properties. If you want to ensure uniqueness, you should add validation in clean and add unique index on identifier field too.
Besides filter returns all the objects that match the criteria, if you are expecting only one item to be returned, you should use get instead. get would raise exception if less or more than 1 item is found.
let's say that I have an Address model with a postcode field. I can lookup addresses with postcode starting with "123" with this line:
Address.objects.filter(postcode__startswith="123")
Now, I need to do this search the "other way around". I have an Address model with a postcode_prefix field, and I need to retrieve all the addresses for which postcode_prefix is a prefix of a given code, like "12345". So if in my db I had 2 addresses with postcode_prefix = "123" and "234", only the first one would be returned.
Something like:
Address.objects.filter("12345".startswith(postcode_prefix))
The problem is that this doesn't work.
The only solution I can come up with is to perform a filter on the first char, like:
Address.objects.filter(postcode_prefix__startswith="12345"[0])
and then, when I get the results, make a list comprehension that filters them properly, like this:
results = [r for r in results if "12345".startswith(r.postcode_prefix)]
Is there a better way to do it in django?
Edit: This does not answer the original question but how to word a query the other way around.
I think what you are trying to do with your "something like" line is properly written as this:
Address.objects.filter(postcode__startswith=postcode_prefix)
In SQL terms, what you want to achieve reads like ('12345' is the postcode you are searching for):
SELECT *
FROM address
WHERE '12345' LIKE postcode_prefix||'%'
This is not really a standard query and I do not see any possibility to achieve this in Django using only get()/filter().
However, Django offers a way to provide additional SQL clauses with extra():
postcode = '12345'
Address.objects.extra(where=["%s LIKE postcode_prefix||'%%'"], params=[postcode])
Please see the Django documentation on extra() for further reference. Also note that the extra contains pure SQL, so you need to make sure that the clause is valid for your database.
Hope this works for you.
Bit of a mouthful but you can do this by annotating your search value and then filtering against it. All happens pretty quickly in-database.
from django.db.models import Value as V, F, CharField
Address.objects.exclude(
postcode_prefix=''
).annotate(
postcode=Value('12345', output_field=CharField())
).filter(
postcode__startswith=F('postcode_prefix')
)
The exclude is only necessary if postcode_prefix can be empty. This would result in an SQL like '%', which would match every postcode.
I'm sure you could do this via a nice templated function these days too... But this is clean enough for me.
A possible alternative. (Have no idea how it compares to the accepted solution with a column as the second param to like, in execution time)
q=reduce(lambda a,b:a|b, [Q(postcode__startswith=postcode[:i+1]) for i in range(len(postcode))])
Thus, you generate all prefixes, and or them together...
The raw SQL query that would do that you need looks something like this:
select * from postal_code_table where '1234567' like postal_code||'%'
This query will select any postal_code from your table that is a substring of '1234567' and also must start from begining, ie: '123', '1234', etc.
Now to implement this in Django, the preferred method is using a custom look up:
from django.db.models.fields import Field
from django.db.models import Lookup
#Field.register_lookup
class LowerStartswithContainedBy(Lookup):
'''Postgres LIKE query statement'''
lookup_name = 'istartswithcontainedby'
def as_sql(self, compiler, connection):
lhs, lhs_params = self.process_lhs(compiler, connection)
rhs, rhs_params = self.process_rhs(compiler, connection)
params = lhs_params + rhs_params
return f"LOWER({rhs}) LIKE LOWER({lhs}) || '%%'", params
Now you can write a django query such as the following:
PostCode.objects.filter(code__istartswithcontainedby='1234567')
Similarly, if you are just looking for substring and do not require the startswith condition, simply modify the return line of as_sql method to the following:
return f"LOWER({rhs}) LIKE '%%' || LOWER({lhs}) || '%%'", params
For more detailed explanation, see my git gist Django custom lookup
A. If not the issue https://code.djangoproject.com/ticket/13363,
you could do this:
queryset.extra(select={'myconst': "'this superstring is myconst value'"}).filter(myconst__contains=F('myfield'))
Maybe, they will fix an issue and it can work.
B. If not the issue 16731 (sorry not providing full url, not enough rep, see another ticket above) you could filter by fields that added with '.annotate', with creation of custom aggreation function, like here:
http://coder.cl/2011/09/custom-aggregates-on-django/
C. Last and successful. I have managed to do this using monkeypatching of the following:
django.db.models.sql.Query.query_terms
django.db.models.fields.Field.get_prep_lookup
django.db.models.fields.Field.get_db_prep_lookup
django.db.models.sql.where.WhereNode.make_atom
Just defined custom lookup '_starts', which has reverse logic of '_startswith'
I have first_name, last_name & alias (optional) which I need to search for. So, I need a query to give me all the names that have an alias set.
Only if I could do:
Name.objects.filter(alias!="")
So, what is the equivalent to the above?
You could do this:
Name.objects.exclude(alias__isnull=True)
If you need to exclude null values and empty strings, the preferred way to do so is to chain together the conditions like so:
Name.objects.exclude(alias__isnull=True).exclude(alias__exact='')
Chaining these methods together basically checks each condition independently: in the above example, we exclude rows where alias is either null or an empty string, so you get all Name objects that have a not-null, not-empty alias field. The generated SQL would look something like:
SELECT * FROM Name WHERE alias IS NOT NULL AND alias != ""
You can also pass multiple arguments to a single call to exclude, which would ensure that only objects that meet every condition get excluded:
Name.objects.exclude(some_field=True, other_field=True)
Here, rows in which some_field and other_field are true get excluded, so we get all rows where both fields are not true. The generated SQL code would look a little like this:
SELECT * FROM Name WHERE NOT (some_field = TRUE AND other_field = TRUE)
Alternatively, if your logic is more complex than that, you could use Django's Q objects:
from django.db.models import Q
Name.objects.exclude(Q(alias__isnull=True) | Q(alias__exact=''))
For more info see this page and this page in the Django docs.
As an aside: My SQL examples are just an analogy--the actual generated SQL code will probably look different. You'll get a deeper understanding of how Django queries work by actually looking at the SQL they generate.
Name.objects.filter(alias__gt='',alias__isnull=False)
Firstly, the Django docs strongly recommend not using NULL values for string-based fields such as CharField or TextField. Read the documentation for the explanation:
https://docs.djangoproject.com/en/dev/ref/models/fields/#null
Solution:
You can also chain together methods on QuerySets, I think. Try this:
Name.objects.exclude(alias__isnull=True).exclude(alias="")
That should give you the set you're looking for.
1. When using exclude, keep the following in mind to avoid common mistakes:
Should not add multiple conditions into an exclude() block like filter(). To exclude multiple conditions, you should use multiple exclude().
Example: (NOT a AND NOT b)
Entry.objects.exclude(title='').exclude(headline='')
equal to
SELECT... WHERE NOT title = '' AND NOT headline = ''
======================================================
2. Only use multiple when you really know about it:
Example: NOT (a AND b)
Entry.objects.exclude(title='', headline='')
equal to
SELECT.. WHERE NOT (title = '' AND headline = '')
If you want to exclude null (None), empty string (""), as well as a string containing white spaces (" "), you can use the __regex along with __isnull filter option
Name.objects.filter(
alias__isnull = False,
alias__regex = r"\S+"
)
alias__isnull=False excludes all the columns null columns
aliax__regex = r"\S+" makes sure that the column value contains at least one or more non whitespace characters.
From Django 1.8,
from django.db.models.functions import Length
Name.objects.annotate(alias_length=Length('alias')).filter(alias_length__gt=0)
You can simply do this:
Name.objects.exclude(alias="").exclude(alias=None)
It's really just that simple. filter is used to match and exclude is to match everything but what it specifies. This would evaluate into SQL as NOT alias='' AND alias IS NOT NULL.
Another approach using a generic isempty lookup, that can be used with any field.
It can also be used by django rest_framework or other apps that use django lookups:
from distutils.util import strtobool
from django.db.models import Field
from django.db.models.lookups import BuiltinLookup
#Field.register_lookup
class IsEmpty(BuiltinLookup):
lookup_name = 'isempty'
prepare_rhs = False
def as_sql(self, compiler, connection):
sql, params = compiler.compile(self.lhs)
condition = self.rhs if isinstance(self.rhs, bool) else bool(strtobool(self.rhs))
if condition:
return "%s IS NULL or %s = ''" % (sql, sql), params
else:
return "%s <> ''" % sql, params
You can then use it like this:
Name.objects.filter(alias__isempty=False)
this is another simple way to do it .
Name.objects.exclude(alias=None)