Dynamically build queryset - django

I have these classes:
class Keyword(models.Model):
keyword = models.CharField(max_length=100, unique=True)
class Snippet(models.Model):
keywords = models.ManyToManyField(Keyword)
and a list of searched keywords:
searchlist = ['Photo','Cat']
I would like to dynamically query the database for snippets that contain both (AND operation) of the keywords in searchlist.
From that list of snippets, return the list of unique keywords from each of the snippets, excluding the original search terms.
So far I have this:
# Turn list of values into list of Q objects
queries = [Q(keywords__keyword__exact=tag) for tag in searchlist]
# Take one Q object from the list
query = queries.pop()
# Or the Q object with the ones remaining in the list
for item in queries:
query &= item
# Query the model
snippet_list = Snippet.objects.filter(query)
which returns the desired list of snippets but I'm unsure of how to retrieve the combined list of keywords, or if there is a more efficient way to achieve the end result.
EDIT:
I would like to achieve the following pseudo code on snippet_list:
for snippet in snippet_list:
combined_keywords += snippet.keywords
ordered_keyword_list = getOrderedKeywords() # <- I have this function already
final_list = intersection(ordered_keyword_list, combined_keywords)
where combined_keywords is all the keywords in the snippet_list. And final_list is a ordered unique version of combined_keywords.

The final solution looks like this thanks to #Soravux and #Dave-Webb. I'm not sure if this is the most efficient method, but it works.
# Turn list of values into list of Q objects
queries = [Q(keywords__keyword__exact=tag) for tag in valueslist]
query = Q()
for item in queries:
query &= item
# Query the model
snippet_list = Snippet.objects.filter(query)
combined_keywords = []
for snippet in snippet_list:
combined_keywords += snippet.keywords.all()
ordered_keyword_list = Keyword.objects.all().annotate(count=Count('snippet')).order_by('-count')
final_list = [x for x in ordered_keyword_list if x in combined_keywords]

Related

Optimize request in the FOR loop

How can I optimize the following request to eliminate loop? Codes count is several hundred, so I get several hundreds database queries, which is unacceptable.
n = 3
result = []
codes = Target.objects.filter(code__in=['ABC', 'CDE', ...])
for code in codes:
result.append(Data.objects.select_related('target')
.filter(target__code=code)
.values('spec', 'spec_type')
.order_by('-spec')[:n])
Models:
class Data(models.Model):
target = models.ForeignKey(Target)
spec_type = models.CharField()
spec = models.FloatField()
class Target(models.Model):
code = models.TextField(db_index=True)
You do not have to retrieve the codes as a QuerySet to enumerate over. We can directly work with the list of codes.
If you want to construct a QuerySet that contains all the given elements, you can make a QuerySet with union that will fetch these objects. In that case this can be done with .union(…) [Django-doc]:
codes = ['ABC', 'CDE']
n = 3
result = Data.objects.none().union(
*[
Data.objects.filter(target__code=code).values('spec', 'spec_type').order_by('-spec')[:n]
for code in codes
],
all=True
)
Like #Willem Van Onsem said, you don't need to get get a queryset of your Target objects since you already seem to have the codes that you want. Just store the codes in a variable and then you can do a django query using that list.
codes = ['ABC', 'CDE', ...]
result = Data.objects.filter(target__code__in = codes)
This query should return all Data objects of which the related Target object's code is in the list codes.

To create a list with sum of values from database for given condition in Django

I have the following model.
models.py
class WoCpt(models.Model):
order = models.ForeignKey(Order, null=False)
comp = models.ForeignKey(Comp, null=False)
proc = models.ForeignKey(Proc, null=False)
points = models.IntegerField('Standard Time', null=False)
Sample Database:
[order, comp, proc, points]
[5,4,3,24]
[5,4,1,29]
[5,4,4,23]
[5,3,1,44]
[5,2,1,11]
[7,4,3,24]
[7,4,1,29]
[7,4,4,23]
[7,3,1,44]
[7,2,1,11]
Problem:
Need to calculate total points for order=5 and proc=1
After getting the sum, I need to create a list as below:
(expectation - need to group the similar proc within a given order and prepare a list to pass it to view)
[order, proc, points]
[5,1,84]
[5,3,24]
[5,4,23]
Kindly help.
You need to write some logic your self. I am giving skeleton code for you. In your case you need to use django filters.
order=5
proc=1
new_list = [order, proc, None]
wocpt = WoCpt.objects.filter(order=order, proc=proc)
new_list[2]=sum(row.points for row in wocpt)
print new_list
[5,1,84]
https://docs.djangoproject.com/en/1.6/ref/models/querysets/#filter
No need to get all fancy in Python code. It can just as easily be done in the Django ORM with a single query.
>>> WoCpt.objects.values_list('order', 'proc').annotate(Sum('points'))
[(5,1,84), (5,3,24), (5,4,23)]
This will first group rows by their order and proc columns, and then annotate each row with the sum of points for all contained results. values_list will return a list of tuples, containing the values of the fields as ordered in the values_list parameters, followed by any annotated fields (so ('order', 'proc', 'points_sum') in this case).
If you have these results, and you need the total points for a specific order and proc, you can use a function like this:
def total_points(l, order, proc):
return [x for x in l if x[0] == order and x[1] == proc][2]

Newbie Django filter uniques 2 many values

i am trying to filter the uniques from a list with this form:
class SpecForm(ModelForm):
a = Doctors_list.objects.values_list('specialty', flat=True)
unique = {z: i for i, z in a}
qs = Doctors_list.objects.filter(id__in=unique.values())
specialty = forms.ModelChoiceField(queryset=qs)
class Meta:
model = Doctors_list
everything seems correct for me, but i get this error: too many values to unpack
any hints?
I think the correct statement should be this:
unique = {z: i for i in a}
Are you specifically trying to put those values into a dictionary? This will yield a list:
unique = [ i for i in a ]
If you go with this, you will have to remove the .values() in qs = Doctors_list.objects.filter(id__in=unique.values()) leaving it like this:
qs = Doctors_list.objects.filter(id__in=unique)
What's going on here is that with brackets in the first approach you're creating a dictionary with just one key and a list as a the value of that key. When you issue .values() you get a list with the dictionary's values. So it's pointless to use a dictionary.
With the second approach you get a list directly.
Hope it helps.

django dynamically filtering with q objects

I'm trying to query a database based on user input tags. The number of tags can be from 0-5, so I need to create the query dynamically.
So I have a tag list, tag_list, and I want to query the database:
design_list = Design.objects.filter(Q(tags__tag__contains = "tag1") and Q(tags__tag__contains = "tag2") and etc. etc. )
How can I create this feature?
You'll want to loop through the tag_list and apply a filter for each one.
tag_list = ['tag1', 'tag2', 'tag3']
base_qs = Design.objects.all()
for t in tag_list:
base_qs = base_qs.filter(tags__tag__contains=t)
This will give you results matching all tags, as your example indicated with and. If in fact you needed or instead, you will probably need Q objects.
Edit: I think I have what you're looking for now.
tags = ['tag1', 'tag2', 'tag3']
q_objects = Q() # Create an empty Q object to start with
for t in tags:
q_objects |= Q(tags__tag__contains=t) # 'or' the Q objects together
designs = Design.objects.filter(q_objects)
I tested this and it seems to work really well.
Edit 2: Credit to kezabelle in #django on Freenode for the initial idea.
You can use this way:
my_dict = {'field_1': 1, 'field_2': 2, 'field_3': 3, ...} # Your dict with fields
or_condition = Q()
for key, value in my_dict.items():
or_condition.add(Q(**{key: value}), Q.OR)
query_set = MyModel.objects.filter(or_condition)
By this way you can use dynamically generated field names.
Also you can use Q.AND for AND condition.
Just prepare a tag list first then, query like this:
tags = ['tag1', 'tag2',...]
design_list = Design.objects.filter(tags__tag__contains__in = tags)
You may need to add AND and OR conditions
query = (Q(fild1='ENABLE'))
# Filter by list
query.add(Q(fild2__in=[p.code for p in Objects.field.all()]), Q.AND)
# filter OR
q_objects = Q(field3='9999999')
for x in myList:
q_objects.add(Q(field3=x.code), Q.OR)
query.add(q_objects, Q.AND)
Use reduce :
from functools import reduce
design_list = Design.objects.filter(reduce(lambda q1,q2: q1 & q2,
[Q(tags__tag__contains=t)
for t in tag_list]))

Looking for Django any() and all() on querysets

I have a form that has a group of 13 checkboxes that together make up my search criteria... except that I also added a pair of radio buttons for ALL or ANY.
I was hoping to get away with something elegant like:
priority_ids = request.GET.getlist("priority") # checkboxes
collection = request.GET.get("collection") # radio buttons
priorities = []
for priority_id in priority_ids:
priorities.append(Q(focus__priority=priority_id))
if (collection == "any"): qset = any(priorities)
elif (collection == "all"): qset = all(priorities)
However, any() and all() return a boolean, not a queryset that I can use in a filter. I want an "any" or "all" that does the equivalent of "Q(...) | Q(...) | Q(...)" or "Q(...) & Q(...) & Q(...)" for anywhere from 1 to 13 criteria.
There's nothing that Django needs to do about that. You just need to combine your Q-s with & and respectively |, in a simple loop or in a more compact way with reduce.
And regarding terminology it seems to me that you are calling Q a queryset, but it's not. It's a filter on a queryset.
Something like the below should work:
priority_ids = request.GET.getlist("priority")
collection = request.GET.get("collection")
priority_filters = []
for priority_id in priority_ids:
priority_filters.append(Q(focus__priority=priority_id))
base_qs = SomeModel.objects.all()
if collection == "any":
filtered_qset = base_qs.filter(reduce(operator.or_, priority_filters))
elif collection == "all":
filtered_qset = base_qs.filter(reduce(operator.and_, priority_filters))
You can query dynamically with Q() as below...
from django.db.models import Q
from priority_app.models import Priority # get your model identified
#(whatever API place you had this):
priority_id = request.GET.get("priority") #getlist is isn't a proper list
field_id = request.query_params.get('collection')
if priority_id:
priority_id_list = list(map(str.strip, priority_id.split(",")))
dynamic_query_filters = Priority.objects.none()
for a_item in priority_id_list:
if collection == "any":
dynamic_query_filters |= (Q(id=a_item)) # OR query
else:
dynamic_query_filters &= (Q(id=a_item)) # AND query
queryset = Priority.objects.filter(dynamic_query_filters)
# you will notice, it executes in order of "query", maybe take them in different variables and chain them accordingly.