So I want to find any kind of matching given some fields, so for example, this is what I would like to do:
possible_merchants = ["amazon", "web", "services"]
# Possible name --> "Amazon Service"
Companies.objects.filter(name__icontains__in=possible_merchants)
sadly it is not possible to mix icontains and the __in lookup.
It seems to be a pretty complex query so if at least I could ignore case the name that would be enough, for example:
Companies.objects.filter(name__ignorecase__in=possible_merchants)
Any ideas?
P.D.: The queries I posted don't work, it's just a way to express what I need (just in case heh)
You can create querysets with the Q constructor and combine them with the | operator to get their union:
from django.db.models import Q
def companies_matching(merchants):
"""
Return a queryset for companies whose names contain case-insensitive
matches for any of the `merchants`.
"""
q = Q()
for merchant in merchants:
q |= Q(name__icontains = merchant)
return Companies.objects.filter(q)
(And similarly with iexact instead of icontains.)
I find it a cleaner approach using reduce and or_ operator:
from django.db.models import Q
from functools import reduce
from operator import or_
def get_companies_from_merchants(merchant_list):
q_object = reduce(or_, (Q(name__icontains=merchant) for merchant in merchant_list))
return Companies.objects.filter(q_object)
This would create a list of Q objects querying the name to contain a single element in merchant list. This would happpen for all the elements in merchant_list and all these Q objects would be reduced to a single Q object having mutliple ORs which can be directly applied to the filter query.
This is the approach that I adopted:
class MyManager(models.Manager):
def exclusive_in(self, lookup, value_list):
return self.filter(reduce(or_, (Q(**{lookup:_}) for _ in value_list)))
Here is now to use it:
Companies.objects.exclusive_in('name__icontains', possible_merchants])
It was inspired by other answers in this thread, as well as Django filter queryset __in for *every* item in list.
Another approach would be to simulate the actions that Django normally does for iexact queries (it converts both parts of the comparison statement to the upper case via SQL Upper function.
This way, the query will look like this:
Companies.objects.annotate(
upper_name=models.Upper("name")
).filter(
upper_name__in=[rchant.upper() for merchant in possible_merchants]
)
Related
I am simplifying this for clarity. Let's say I had this function that finds Document records based on a requestedColor.
def find_docs(requestedColor):
docs = Document.objects.filter(Q(color=requestedColor) | Q(other_color=requestedColor))
I'd like to order the results so that Document found using color will appear before objects found with other_color and I want to do it within the query - without any external sorting.
Is there a way to do this within the ORM query? I could not find a way to do that.
Pointers will be appreciated.
You can use Conditional Expressions to annotate a value which would indicate which field matched and order by this value:
from django.db.models import Case, IntegerField, Value, When
docs = Document.objects.annotate(
color_order=Case(
When(color=requestedColor, then=Value(1, output_field=IntegerField())),
When(other_color=requestedColor, then=Value(2, output_field=IntegerField())),
default=Value(0, output_field=IntegerField()),
)
).filter(
Q(color=requestedColor) | Q(other_color=requestedColor)
).order_by('color_order')
Consider I have a model called Subject, and I want to filter the values based on the list of values.
Consider the below list.
["John", "Brad", None]
When I try to filter out the result using __in lookup it doesn't look for None values. For example
Subject.objects.filter(user__in=["John", "Brad", None])
This will provide the queryset for John and Brad but not None. What I'm missing here? Can anyone please help me
NULLs in DB (None in django) are not reqular values, so they need different approach:
Subject.objects.filter(Q(user__in=["John", "Brad", ]) | Q(user__isnull=True,))
Try filtering like this
from django.db.models import Q
Subject.objects.filter(Q(user__in=["John", "Brad"]) | Q(user__isnull=True))
Using Q objects is better suited for this:
Subject.objects.filter(Q(user__in=["John", "Brad"]) | Q(user__isnull=True))
You can use Q objects for OR, AND and NOT operations. Q objects provide you complete control over the where clause of the query.
Your queryset would be something like:
from django.db.models import Q
Subject.objects.filter(Q(user=None) | Q(user__in=["John", "Brad"]))
You can combine the Q objects in more complex ways to generate complex queries.
i'm try create or on filter on django
This is my little example :
products=Products.objects.values('name', 'price').all().filter(status=1|0)
The problem is that don't validate the two options (1|0)
don't get a error on the print(products.query) only validate one option don't the 2 options..!!
Please thanks !!
To filter using OR in django you need a special class called Q.
Documentation about Complex lookups with Q objects
from django.db.models import Q
products = Products.objects.values('name', 'price').filter(Q(status=1) | Q(status=0))
It's good to use Q object
manager.filter(Q(status=1) | Q(status=0))
You need to know that the method all() on a manager just delegates to get_queryset().
To use filter(), you would already have the QuerySet
Rather than all() whose calls the queryset, and then filter whose already call the queryset,
just do manager.filter()
all().filter() becomes just filter() because it's redundant
There it is:
from django.db.models import Q
products = Product.objects.values('name','price').filter(
Q(status=1) | Q(status=0),
)
In sqlalchemy, I define a function whose argument is a dictionary may contain multiple key-values.I want to query according to the key-values. Here is my program:
def get_contact_conditions(kwds):
for fieldName, fieldValue in kwds.items():
condition = fieldName + "=:fieldName"
contact = session.query(Contact).filter(condition).params(fieldName = fieldValue).all()
session.commit()
return contact
this situation above is when the dictionary has only one key-values, but maybe there are two or three in it, then how to code condition and the values in params(). In another word how to code query clause.
Thank you!
If the only comparison you need is "=" then this should work:
def get_contact_conditions(kwds):
contact = session.query(Contact).filter_by(**kwds).all()
return contact
You can read about it here:
http://docs.sqlalchemy.org/en/rel_0_7/orm/query.html#sqlalchemy.orm.query.Query.filter_by
If you need arbitrary operators(<,>,in_,like,etc.) you can pass in a list of sqlalchemy clauses as arguments to the and_ function and pass the result to filter(). These expressions are generated when you compare a model class attribute to something, like:
Contact.id < 5
So you could do something like this:
from sqlalchemy.sql import and_
def get_contacts(*exps):
return session.query(Contact).filter(and_(*exps)).all()
# All contacts with an id less than 5 and a name that starts with user.
contacts = get_contacts(Contact.id < 5, Contact.name.like('user%'))
But you should consider that at this point the function is getting closer to just calling session.query directly.
You should read the sqlalchemy docs more to learn about the operators/comparisons that are available on model columns/relations. There are some really useful ones.
I have a list of names that I want to match case insensitive, is there a way to do it without using a loop like below?
a = ['name1', 'name2', 'name3']
result = any([Name.objects.filter(name__iexact=name) for name in a])
Unfortunatley, there are no __iin field lookup. But there is a iregex that might be useful, like so:
result = Name.objects.filter(name__iregex=r'(name1|name2|name3)')
or even:
a = ['name1', 'name2', 'name3']
result = Name.objects.filter(name__iregex=r'(' + '|'.join(a) + ')')
Note that if a can contain characters that are special in a regex, you need to escape them properly.
NEWS: In Django 1.7+ it is possible to create your own lookups, so you can actually use filter(name__iin=['name1', 'name2', 'name3']) after proper initialization. See documentation reference for details.
In Postgresql you could try creating a case insensitive index as described here:
https://stackoverflow.com/a/4124225/110274
Then run a query:
from django.db.models import Q
name_filter = Q()
for name in names:
name_filter |= Q(name__iexact=name)
result = Name.objects.filter(name_filter)
Index search will run faster than the regex matching query.
Another way to this using django query functions and annotation
from django.db.models.functions import Lower
Record.objects.annotate(name_lower=Lower('name')).filter(name_lower__in=['two', 'one']
Adding onto what Rasmuj said, escape any user-input like so
import re
result = Name.objects.filter(name__iregex=r'(' + '|'.join([re.escape(n) for n in a]) + ')')
Keep in mind that at least in MySQL you have to set utf8_bin collation in your tables to actually make them case sensitive. Otherwise they are case preserving but case insensitive. E.g.
>>> models.Person.objects.filter(first__in=['John', 'Ringo'])
[<Person: John Lennon>, <Person: Ringo Starr>]
>>> models.Person.objects.filter(first__in=['joHn', 'RiNgO'])
[<Person: John Lennon>, <Person: Ringo Starr>]
So, if portability is not crucial and you use MySQL you may choose to ignore the issue altogether.
I am expanding Exgeny idea into an two liner.
import functools
Name.objects.filter(functools.reduce(lambda acc,x: acc | Q(name_iexact=x)), names, Q()))
After trying many methods, including annotate, which resulted in duplicate objects, I discovered transformers (https://docs.djangoproject.com/en/4.1/howto/custom-lookups/#a-transformer-example) which allow for a simple solution.
Add the following to models.py before model declarations:
class LowerCase(models.Transform):
lookup_name = "lower"
function = "LOWER"
models.CharField.register_lookup(LowerCase)
models.TextField.register_lookup(LowerCase)
You can now use the __lower transformer alongside any lookup, in this case: field__lower__in. You can also add bilateral = True to the transformer class for it to apply to both the field and the list items, which should be functionally equivalent to __iin.
Here is an example of custom User model classmethod to filter users by email case-insensitive
from django.db.models import Q
#classmethod
def get_users_by_email_query(cls, emails):
q = Q()
for email in [email.strip() for email in emails]:
q = q | Q(email__iexact=email)
return cls.objects.filter(q)
If this is a common use case for anyone, you can implement this by adapting the code from Django's In and IExact transformers.
Make sure the following code is imported before all model declarations:
from django.db.models import Field
from django.db.models.lookups import In
#Field.register_lookup
class IIn(In):
lookup_name = 'iin'
def process_lhs(self, *args, **kwargs):
sql, params = super().process_lhs(*args, **kwargs)
# Convert LHS to lowercase
sql = f'LOWER({sql})'
return sql, params
def process_rhs(self, qn, connection):
rhs, params = super().process_rhs(qn, connection)
# Convert RHS to lowercase
params = tuple(p.lower() for p in params)
return rhs, params
Example usage:
result = Name.objects.filter(name__iin=['name1', 'name2', 'name3'])