This question already has answers here:
Django __exact queryset filter - case sensitivity not working as expected
(2 answers)
Closed 24 days ago.
I have been testing user searching with sorted results and I found this strange behavior
>>> User.objects.filter(username__istartswith="AbC")
<QuerySet [<User: AbC>, <User: AbCuuu>, <User: abc>, <User: abcuuu>]>
>>> User.objects.filter(username__startswith="AbC")
<QuerySet [<User: AbC>, <User: AbCuuu>, <User: abc>, <User: abcuuu>]>
Shouldn't __startswith only have 2 of those results?
I need to actually search with case sensitivity, how do I do that?
I expect __startswith to be case sensitive and __istartswith to be case insensitive, but both return the same, case insensitive QuerySet
You are most likely using SQLite, where startswith will return the same result as istartswith, due to lack of case sensive LIKE (see text in bold at the bottom):
startswith
Case-sensitive starts-with.
Example:
Entry.objects.filter(headline__startswith='Lennon')
SQL equivalent:
SELECT ... WHERE headline LIKE 'Lennon%';
SQLite doesn’t support case-sensitive LIKE statements; startswith acts like istartswith for SQLite.
https://docs.djangoproject.com/en/4.1/ref/models/querysets/#startswith
Related
node_status is JOSNField, how to filter node_status has {'2': True} of queryset.
>>> instance.node_status
{'cat': '1', '2': True, 'dog': True}
>>> qs.filter(node_status__cat='1')
Yeah got result
>>> qs.filter(node_status__has_key='dog')
Yeah got result
>>> qs.filter(node_status__2=True)
<QuerySet []>
node_status__2=True got empty queryset.
#models.py
from django.db import models
class Foo(models.Model):
node_status = models.JSONField(null=True, blank=True)
instance = Foo.objects.create(node_status={'cat': '1', '2': True, 'dog': True})
qs = Foo.objects.all()
qs.filter(node_status__cat='1')
qs.filter(node_status__has_key='dog')
qs.filter(node_status__2=True)
Envrioment:
MariaDB 10.2
Django 4.0.1
Django automatically converting number-like values to numbers and array indexes:
Let's compare raw SQLs
print(Foo.objects.filter(node_status__has_key='dog').query)
print(Foo.objects.filter(node_status__has_key='2').query)
SELECT `django4_foo`.`id`, `django4_foo`.`node_status` FROM `django4_foo` WHERE JSON_CONTAINS_PATH(`django4_foo`.`node_status`, 'one', $."dog")
SELECT `django4_foo`.`id`, `django4_foo`.`node_status` FROM `django4_foo` WHERE JSON_CONTAINS_PATH(`django4_foo`.`node_status`, 'one', $[2])
As you can see 2 is converted to $[2] but dog is converted to $."dog"
One possible solution is to get raw SQL, reformat it manually and init new query set using raw SQL:
raw_sql = str(Foo.objects.filter(node_status__has_key='2').query).replace(f'$[2]', f'\'$.2\'')
print(raw_sql)
rqs = Foo.objects.raw(raw_sql)
for o in rqs:
print(o.node_status)
SELECT `django4_foo`.`id`, `django4_foo`.`node_status` FROM `django4_foo` WHERE JSON_CONTAINS_PATH(`django4_foo`.`node_status`, 'one', '$.2')
{'cat': '1', '2': True, 'dog': True}
However this hack don't work with node_status__2=True filter.
Let's check this query for dog key:
print(Foo.objects.filter(node_status__dog=True).query)
Output is:
SELECT `django4_foo`.`id`, `django4_foo`.`node_status` FROM `django4_foo` WHERE JSON_UNQUOTE(JSON_EXTRACT(`django4_foo`.`node_status`, $."dog")) = JSON_EXTRACT(true, '$')
But actually working raw SQL is:
SELECT `django4_foo`.`id`, `django4_foo`.`node_status` FROM `django4_foo` WHERE JSON_UNQUOTE(JSON_EXTRACT(`django4_foo`.`node_status`, "$.dog")) = 'True'
So here is many problems with filter → raw conversions. (Django community also don't recommend to use str(query) for raw SQL query generations)
Django is positioned like universal screwdriver for all types of screws. Internal conflicts and overcomplicated "universal and all in one" logic is a common problem for Django.
Yes it is possible to override query class and change key transformation logic but it is hard to read and understand, pretty complicated and another problems may occur with customized logic (because Django don't expect that logic was changed).
So easier ways will be:
just use Foo.objects.raw(sql_query) where sql_query is predefined manually string
use cursor.execute(sql_query, \[params\]) with sql_query predefined same as ①
get Foo.objects.all() and use python logic to filter it ([e for e in Foo.objects.all() if '2' in e.node_status])
Other possible ways are:
Integrate sqlalchemy
Migrate to another framework (FastAPI, Flask, etc) ※
※ My choice ;)
When the docs discuss values() and values_list(), or any query for that matter, they always require that you know what you are looking for, i.e.,
>>> Entry.objects.values_list('headline', flat=True).get(pk=1)
'First entry'
What about the situation where you need a value from a specific field, whether on this model or a foreign key, but you don't know the pk or the value in the specified field, and you don't care, you just need whatever is there. How do you query for it?
Alternatively, if I use this example from the docs:
>>> Entry.objects.values_list('id', flat=True).order_by('id')
<QuerySet [1, 2, 3, ...]>
Could I add slice notation to the end of the query? But even then, I might not know in advance which slice I need. In other words, how to dynamically get a value from a specified field without knowing in advance what it or its pk is? Thx.
Depending your scenario (this time a simple query) you have many of options to do it. One is to use a variable as field name. Then, feed that variable dynamically:
>>> field='headline'
>>> Entry.objects.values_list(field, flat=True).get(pk=1)
'First entry'
>>> field='body'
>>> Entry.objects.values_list(field, flat=True).get(pk=1)
'First entry body'
In order to slice results use offset/limit as follows:
Entry.objects.all()[offset:limit]
>>> field='headline'
>>> Entry.objects.values_list(field, flat=True)[5:10]
I'm not really sure what's happening here. I can see that I have a couple of emails in my database, but I can't seem to filter them.
for example, when I run
qs1 = EmailActivation.objects.all()
>>> print(qs1)
<EmailActivationQuerySet [<EmailActivation: a#yahoo.com>, <EmailActivation: b#gmail.com>]>
however, when I run the following I get nothing
qs2 = EmailActivation.objects.all().filter(email='a#yahoo.com')
>>> print(qs2)
<EmailActivationQuerySet []>
my model looks like this:
class EmailActivation(models.Model):
user = models.ForeignKey(User)
email = models.EmailField()
I'd expect my qs2 to return 'a#yahoo.com' since it is in the database as seen by qs1. Does someone see what I'm doing wrong?
Thanks,
Edit: looking closer I see the following:
qs2 =
EmailActivation.objects.all().filter(email__icontains='a#yahoo.com')
>>> print(qs2)
<EmailActivationQuerySet [<EmailActivation: a#yahoo.com>]>
Does this mean that there is some whitespace or hidden character in my 'email'? I imagine filtering with icontains would be bad as someone could be 'aa#yahoo.com'. What could I do to strip whatever hidden character is there?
If you explicitly specify exact match on the filter then you should get a result what you are after. I don't think so there are any hidden characters here...
>>> EmailActivation.objects.all().filter(email__exact='a#yahoo.com')
<QuerySet [<EmailActivation: EmailActivation object (1)>]>
this is a simple question. I'd like to know if it is the same to write:
queryset = Model.objects.filter(field=None)
than:
queryset = Model.objects.filter(field__isnull=True)
I'm using django 1.8
They are equal:
>>> str(Person.objects.filter(age__isnull=True).query) == str(Person.objects.filter(age=None).query)
True
>>> print(Person.objects.filter(age=None).query)
SELECT "person_person"."id", "person_person"."name", "person_person"."yes", "person_person"."age" FROM "person_person" WHERE "person_person"."age" IS NULL
>>> print(Person.objects.filter(age__isnull=True).query)
SELECT "person_person"."id", "person_person"."name", "person_person"."yes", "person_person"."age" FROM "person_person" WHERE "person_person"."age" IS NULL
Exclusion: the Postgres JSON field (see the answer of #cameron-lee)
It depends on the type of field. As mentioned in other answers, they are usually equivalent but in general, this isn't guaranteed.
For example, the Postgres JSON field uses =None to specify that the json has the value null while __isnull=True means there is no json:
https://docs.djangoproject.com/en/3.0/ref/contrib/postgres/fields/#jsonfield
Just to keep in mind that you cannot reverse the condition with your first solution:
# YOU CANNOT DO THIS
queryset = Model.objects.filter(field!=None)
However you can do this:
queryset = Model.objects.filter(field__isnull=False)
[Update: software versions Python 2.7.2, Django 1.3.1]
Can anyone explain this console code?
FinishingStep has a ForeignKey to a quote object, but that's not really relevant.
>>> fins = FinishingStep.objects.filter(quote=jq)
>>> fins
[<FinishingStep: Tabbing>, <FinishingStep: Collator>]
So far so good, we have returned a QuerySet with two objects.
But now the confusion. Both objects now appear to be the same:
>>> fins[0]
<FinishingStep: Collator>
>>> fins[1]
<FinishingStep: Collator>
Convert it to a list, and that fixes it.
>>> fins = list(fins)
>>> fins
[<FinishingStep: Tabbing>, <FinishingStep: Collator>]
>>> fins[0]
<FinishingStep: Tabbing>
>>> fins[1]
<FinishingStep: Collator>
[Update: Adding .distinct() to the query also fixes it. This especially odd since, at the moment, there are only those two items in the database.]
Is this a bug? Am I doing something wrong?
This ticket discusses this behavior: https://code.djangoproject.com/ticket/9006
Just use order_by query. This happens because the database engine is free to return any suitable row if you do not specify explicit ordering. So I guess it just picks the one from its cache.