I have these classes:
class Keyword(models.Model):
keyword = models.CharField(max_length=100, unique=True)
class Snippet(models.Model):
keywords = models.ManyToManyField(Keyword)
and a list of searched keywords:
searchlist = ['Photo','Cat']
I would like to dynamically query the database for snippets that contain both (AND operation) of the keywords in searchlist.
From that list of snippets, return the list of unique keywords from each of the snippets, excluding the original search terms.
So far I have this:
# Turn list of values into list of Q objects
queries = [Q(keywords__keyword__exact=tag) for tag in searchlist]
# Take one Q object from the list
query = queries.pop()
# Or the Q object with the ones remaining in the list
for item in queries:
query &= item
# Query the model
snippet_list = Snippet.objects.filter(query)
which returns the desired list of snippets but I'm unsure of how to retrieve the combined list of keywords, or if there is a more efficient way to achieve the end result.
EDIT:
I would like to achieve the following pseudo code on snippet_list:
for snippet in snippet_list:
combined_keywords += snippet.keywords
ordered_keyword_list = getOrderedKeywords() # <- I have this function already
final_list = intersection(ordered_keyword_list, combined_keywords)
where combined_keywords is all the keywords in the snippet_list. And final_list is a ordered unique version of combined_keywords.
The final solution looks like this thanks to #Soravux and #Dave-Webb. I'm not sure if this is the most efficient method, but it works.
# Turn list of values into list of Q objects
queries = [Q(keywords__keyword__exact=tag) for tag in valueslist]
query = Q()
for item in queries:
query &= item
# Query the model
snippet_list = Snippet.objects.filter(query)
combined_keywords = []
for snippet in snippet_list:
combined_keywords += snippet.keywords.all()
ordered_keyword_list = Keyword.objects.all().annotate(count=Count('snippet')).order_by('-count')
final_list = [x for x in ordered_keyword_list if x in combined_keywords]
i am trying to filter the uniques from a list with this form:
class SpecForm(ModelForm):
a = Doctors_list.objects.values_list('specialty', flat=True)
unique = {z: i for i, z in a}
qs = Doctors_list.objects.filter(id__in=unique.values())
specialty = forms.ModelChoiceField(queryset=qs)
class Meta:
model = Doctors_list
everything seems correct for me, but i get this error: too many values to unpack
any hints?
I think the correct statement should be this:
unique = {z: i for i in a}
Are you specifically trying to put those values into a dictionary? This will yield a list:
unique = [ i for i in a ]
If you go with this, you will have to remove the .values() in qs = Doctors_list.objects.filter(id__in=unique.values()) leaving it like this:
qs = Doctors_list.objects.filter(id__in=unique)
What's going on here is that with brackets in the first approach you're creating a dictionary with just one key and a list as a the value of that key. When you issue .values() you get a list with the dictionary's values. So it's pointless to use a dictionary.
With the second approach you get a list directly.
Hope it helps.
I have a form that has a group of 13 checkboxes that together make up my search criteria... except that I also added a pair of radio buttons for ALL or ANY.
I was hoping to get away with something elegant like:
priority_ids = request.GET.getlist("priority") # checkboxes
collection = request.GET.get("collection") # radio buttons
priorities = []
for priority_id in priority_ids:
priorities.append(Q(focus__priority=priority_id))
if (collection == "any"): qset = any(priorities)
elif (collection == "all"): qset = all(priorities)
However, any() and all() return a boolean, not a queryset that I can use in a filter. I want an "any" or "all" that does the equivalent of "Q(...) | Q(...) | Q(...)" or "Q(...) & Q(...) & Q(...)" for anywhere from 1 to 13 criteria.
There's nothing that Django needs to do about that. You just need to combine your Q-s with & and respectively |, in a simple loop or in a more compact way with reduce.
And regarding terminology it seems to me that you are calling Q a queryset, but it's not. It's a filter on a queryset.
Something like the below should work:
priority_ids = request.GET.getlist("priority")
collection = request.GET.get("collection")
priority_filters = []
for priority_id in priority_ids:
priority_filters.append(Q(focus__priority=priority_id))
base_qs = SomeModel.objects.all()
if collection == "any":
filtered_qset = base_qs.filter(reduce(operator.or_, priority_filters))
elif collection == "all":
filtered_qset = base_qs.filter(reduce(operator.and_, priority_filters))
You can query dynamically with Q() as below...
from django.db.models import Q
from priority_app.models import Priority # get your model identified
#(whatever API place you had this):
priority_id = request.GET.get("priority") #getlist is isn't a proper list
field_id = request.query_params.get('collection')
if priority_id:
priority_id_list = list(map(str.strip, priority_id.split(",")))
dynamic_query_filters = Priority.objects.none()
for a_item in priority_id_list:
if collection == "any":
dynamic_query_filters |= (Q(id=a_item)) # OR query
else:
dynamic_query_filters &= (Q(id=a_item)) # AND query
queryset = Priority.objects.filter(dynamic_query_filters)
# you will notice, it executes in order of "query", maybe take them in different variables and chain them accordingly.
I have a list of names that I want to match case insensitive, is there a way to do it without using a loop like below?
a = ['name1', 'name2', 'name3']
result = any([Name.objects.filter(name__iexact=name) for name in a])
Unfortunatley, there are no __iin field lookup. But there is a iregex that might be useful, like so:
result = Name.objects.filter(name__iregex=r'(name1|name2|name3)')
or even:
a = ['name1', 'name2', 'name3']
result = Name.objects.filter(name__iregex=r'(' + '|'.join(a) + ')')
Note that if a can contain characters that are special in a regex, you need to escape them properly.
NEWS: In Django 1.7+ it is possible to create your own lookups, so you can actually use filter(name__iin=['name1', 'name2', 'name3']) after proper initialization. See documentation reference for details.
In Postgresql you could try creating a case insensitive index as described here:
https://stackoverflow.com/a/4124225/110274
Then run a query:
from django.db.models import Q
name_filter = Q()
for name in names:
name_filter |= Q(name__iexact=name)
result = Name.objects.filter(name_filter)
Index search will run faster than the regex matching query.
Another way to this using django query functions and annotation
from django.db.models.functions import Lower
Record.objects.annotate(name_lower=Lower('name')).filter(name_lower__in=['two', 'one']
Adding onto what Rasmuj said, escape any user-input like so
import re
result = Name.objects.filter(name__iregex=r'(' + '|'.join([re.escape(n) for n in a]) + ')')
Keep in mind that at least in MySQL you have to set utf8_bin collation in your tables to actually make them case sensitive. Otherwise they are case preserving but case insensitive. E.g.
>>> models.Person.objects.filter(first__in=['John', 'Ringo'])
[<Person: John Lennon>, <Person: Ringo Starr>]
>>> models.Person.objects.filter(first__in=['joHn', 'RiNgO'])
[<Person: John Lennon>, <Person: Ringo Starr>]
So, if portability is not crucial and you use MySQL you may choose to ignore the issue altogether.
I am expanding Exgeny idea into an two liner.
import functools
Name.objects.filter(functools.reduce(lambda acc,x: acc | Q(name_iexact=x)), names, Q()))
After trying many methods, including annotate, which resulted in duplicate objects, I discovered transformers (https://docs.djangoproject.com/en/4.1/howto/custom-lookups/#a-transformer-example) which allow for a simple solution.
Add the following to models.py before model declarations:
class LowerCase(models.Transform):
lookup_name = "lower"
function = "LOWER"
models.CharField.register_lookup(LowerCase)
models.TextField.register_lookup(LowerCase)
You can now use the __lower transformer alongside any lookup, in this case: field__lower__in. You can also add bilateral = True to the transformer class for it to apply to both the field and the list items, which should be functionally equivalent to __iin.
Here is an example of custom User model classmethod to filter users by email case-insensitive
from django.db.models import Q
#classmethod
def get_users_by_email_query(cls, emails):
q = Q()
for email in [email.strip() for email in emails]:
q = q | Q(email__iexact=email)
return cls.objects.filter(q)
If this is a common use case for anyone, you can implement this by adapting the code from Django's In and IExact transformers.
Make sure the following code is imported before all model declarations:
from django.db.models import Field
from django.db.models.lookups import In
#Field.register_lookup
class IIn(In):
lookup_name = 'iin'
def process_lhs(self, *args, **kwargs):
sql, params = super().process_lhs(*args, **kwargs)
# Convert LHS to lowercase
sql = f'LOWER({sql})'
return sql, params
def process_rhs(self, qn, connection):
rhs, params = super().process_rhs(qn, connection)
# Convert RHS to lowercase
params = tuple(p.lower() for p in params)
return rhs, params
Example usage:
result = Name.objects.filter(name__iin=['name1', 'name2', 'name3'])
Is there a way to do a query and exclude a list of things, instead of calling exclude multiple times?
Based on your reply to Ned, it sounds like you just want to exclude a list of tags. So you could just use the in filter:
names_to_exclude = [o.name for o in objects_to_exclude]
Foo.objects.exclude(name__in=names_to_exclude)
Does that do what you want?
What's wrong with calling exclude multiple times? Queries are lazy, nothing happens until you try to pull data from it, so there's no downside to using .exclude() more than once.
You can try this also.
exclude_list = ['A', 'B', 'C']
qs = Foo.objects.exclude(items__in=exclude_list)
You can do it pretty easily with the Q object:
from django.db.models import Q
excludes = None
for tag in ignored_tags:
q = Q(tag=tag)
excludes = (excludes and (excludes | q)) or q # makes sure excludes is set properly
set_minus_excluded = Foo.objects.exclude(excludes)
You should also be able to do it dynamically with exclude():
qs = Foo.objects.all()
for tag in ignored_tags:
qs = qs.exclude(tag=tag)
To improve on Daniel Roseman's answer I think it would be better to get the values you need directly from the queryset instead of the for loop that could be expensive on large data sets i.e.
names_to_exclude = objects_to_exclude.values_list('name')
Foo.objects.exclude(name__in=names_to_exclude)