django conditionally filtering objects - django

I would like to retrieve a bunch of rows from my database using a set of filters.
I was wondering if conditional filter is applicable in django. That is, "filter if variable is not None, or not apply filtering otherwise".
Something like this:
user = User.objects.get(pk=1)
category = Category.objects.get(pk=1)
todays_items = Item.objects.filter(user=user, date=now()).conditional_filter(category=category))
What I would like to do is apply category filter only if category is not None.
If category is None (means it is not given in the request object) then this filter would not be applied at all. This would save me a bunch of 'if-elif-else' situations.
Is there a way to do this?

You can chain queries:
user = User.objects.get(pk=1)
category = Category.objects.get(pk=1)
qs = Item.objects.filter(user=user, date=now())
if category:
qs = qs.filter(category=category)
As queryset are executed lazily, DB hit will occur only when you display items.

They are several approach to your issue. One approach is to play with Complex lookups with Q objects
from django.db.models import Q
user = User.objects.get(pk=1)
category = Category.objects.get(pk=1)
f1 = Q( user=user, date=now() )
f_cat_is_none = Q( category__isnull = True )
f_cat_is_not_none = Q( category=category )
todays_items = Item.objects.filter( f1 & ( f_cat_is_none | f_cat_is_not_none ) )
I don't right understand in your answer if this is the query you are looking for, but, with this example you can compose easily your own query.
Edited due OP comment
category__isnull == True means that, in database, the item has not an associated category.
Perhaps the query you are looking for is:
from django.db.models import Q
user_pk = 1
category_pk = 1 #some times None
f = Q( user__pk = user_pk, date=now() )
if category_pk is not None:
f &= Q( category__pk = category_pk )
todays_items = Item.objects.filter( f )
This is only a code sample, fit it to your requirements. Be careful with single _ and double __.

Well, this is rather old question but for those who would like to do the conditional filtering on one line, here is my approach (Btw, the following code can probably be written in a more generic way):
from django.db.models import Q
def conditional_category_filter(category):
if category != None:
return Q(category=category)
else:
return Q() #Dummy filter
user = User.objects.get(pk=1)
category = Category.objects.get(pk=1)
todays_items = Item.objects.filter(conditional_category_filter(category), user=user, date=now())
The only thing you need to watch is to use the conditional_category_filter(category) call before the keyword arguments like user=user. For example the following code would throw an error:
todays_items = Item.objects.filter(user=user, date=now(), conditional_category_filter(category))

To continue on #iuysal answer:
To make it generic you need to pass the key too as parameter, to do that you need to pass a dictionary, here's how I did it:
Create your dictionary like this:
filters = {'filter1': 'value1', 'filter2__startswith': 'valu', ...}
Then pass it to your Item filters like this:
Item.objects.filter(*[Q(**{k: v}) for k, v in filters.items() if v], filter3='value3')
The first version less cryptic version I had:
def _filter(filters):
filters = []
for k, v in n.items():
if v:
filters.append(Q(**{k: v}))
return filters
filters = _filter({'name': name})
return Item.objects.filter(*filters)
Unpacking explanation: We want to give Q (queries) as args to objects.filter as args while we want to give kwargs to Q()
I have this on production now (I will just modify the filters names because it's sensitive):
def get_queryset(self):
filter1 = self.request.GET.get('filter1 ', '')
filter2__startswith = self.request.GET.get('filter2_prefix ', '')
def filters_to_Qs(filters):
return [Q(**{k: v}) for k, v in filters.items() if v]
filters = {'filter1': filter1 ,
'filter2__startswith': filter2__startswith }
return Order.objects.filter(*filters_to_Qs(filters))

from django.db.models import Q
qs = Users.objects.filter(
p_id=parent_id,
status=True
).all()
if user_id>0:
qs = qs.filter( ~Q(id=user_id) )
in qs we will get the filtered results

Related

What is the command to search a content in all the model fields at once?

I am developing an app with Django.
I have developed a search bar to filter the database contents.
I want that, when the user clicks on the search button, the indicated content is searched in all the model fields.
My model has 16 fields, so my filter command will be a very long line, like:
selected_entries = glossary_entry.objects.filter(Q(field1__icontains=query) | Q(field2__icontains=query)) | ...ETC ETC... | Q(field16__icontains=query))
Here you can see this line in my view function, in views.py:
def glossario(request):
query = request.GET.get('q')
template = "glossario.html"
# query applied
if query:
query = request.GET.get('q')
selected_entries = glossary_entry.objects.filter(Q(field1__icontains=query) | Q(field2__icontains=query)) | ...ETC ETC... | Q(field16__icontains=query))
return render(request, template, {'all_entries':selected_entries})
# no query
else:
all_entries = glossary_entry.objects.all
return render(request, template, {'all_entries':all_entries})
Is there a shorter command to do the same?
Like:
selected_entries = glossary_entry.objects.filter(Q(ALL_MODEL_FIELDS_ICONTAINS(model=MyModel)=query))
Note: ALL_MODEL_FIELDS_ICONTAINS is my invention
You will have to specify all the fields anyway. There isn't a shortcut. But you could concatenate all fields and just do one lookup:
from django.db.models import CharField, Value as V
from django.db.models.functions import Concat
glossary_entry.objects.annotate(
search_string=Concat(
'field1', V(' '), 'field2', V(' '), 'field3', ...,
output_field=CharField()
)
).filter(search_string__icontains=q)
Note: I've added spaces between each field using Value although that's not strictly necessary with icontains, it just helps with debugging if you're looking at the value of search_string.

Django2.2 Context Manager | {{template_tags}} | Variable limit?

I have a template view in Django2.2, where I added two models & queries to look up metrics on user data (how many articles they read each month).
I ran into a strange issue where if I place multiple context variables on the page, every variable on the page but the First Variable will return '0'. If I simply change the order of the variables in the markup, each date function appears to be calculating correctly (as long as it is the first to appear). I couldn't find anything about this in the docs...and I'm guessing that this isn't a great approach to display this information either and I should instead use a DjangoTemplateTag and perform the operations there.
*Define the Object & Query
def get_context_data(self, **kwargs):
context = super(userDashBoardView, self).get_context_data(**kwargs)
context['readArticleList'] = Articles.objects.filter(Unread = False,userId = self.request.user.SFPK )
*To avoid making further queries, I mutated the query into a set to perform further functions
article_list = set(context['readArticleList'])
article_read_date = (article_list.read_date for article_list in article_list)
context['articles_last30'] = len(set(x for x in article_read_date if x > timezone.now() - timedelta(days=30)))
context['articles_last60'] = len(set(x for x in article_read_date if x > timezone.now() - timedelta(days=60)))
context['articles_last90'] = len(set(x for x in article_read_date if x > timezone.now() - timedelta(days=90)))
return context
{% block content %}
{{articles_last30}}
{{articles_last60}}
{{articles_last90}}
{% endblock %}
<br/>
For context,context,context in the example above *using sample data
the output on the page is (4,0,0)
If the order is reversed, I get
(20,0,0)
NOTE: I am NOT receiving any errors in the console, and the page(s) are loading fine. Can anyone point me in the right direction?
-Thank you for your time =)
You use a generator for article_read_date, indeed:
article_read_date = (article_list.read_date for article_list in article_list)
This means that after you have iterated over it, the generator is "exhausted". Another loop over the iterator will not yield any values anymore. An illustrative example is the following:
>>> l = [1,4,2,5]
>>> g = (x for x in l)
>>> list(g)
[1, 4, 2, 5]
>>> list(g)
[]
as you can see, the second list(g) does not produces any values anymore.
You can create a set with:
article_read_date = [a.read_date for a in context['readArticleList']]
def count_since(iterable, _timed):
timestamp = timezone.now() - timed
return sum(x > timestamp for x in iterable)
context['articles_last30'] = count_since(article_read_date, timedelta(days=30))
context['articles_last60'] = count_since(article_read_date, timedelta(days=60))
context['articles_last90'] = count_since(article_read_date, timedelta(days=90))
That being said, since django-2.0, Count [Django-doc] has a filter= attribute, so you can count the articles with one extra query like:
from django.db.models import Count, Q
nw = timezone.now()
context.update(
context['readArticleList'].aggregate(
articles_last30=Count('pk', filter=Q(read_date__gt=nw-timedelta(days=30))),
articles_last60=Count('pk', filter=Q(read_date__gt=nw-timedelta(days=60))),
articles_last90=Count('pk', filter=Q(read_date__gt=nw-timedelta(days=90)))
)
)
You can do what you are doing in a much more efficient way and cleaner way.
First though you should Note that the convention is models are not named in plural. So it is not Articles.objects.. but Article.objects... You should rename your model to Article instead of Articles.
If we assume that as should be the case that Article(s) is a model which has field read date.
class Article(models.Model):
read_date = models.DateTimeField()
... other_fields ..
Since you very much want efficiency. You can count the results straight from the DB.
def get_context_data(self, **kwargs):
thirty_days_ago = datetime.now() - timedelta(days=30)
sixty_days_ago = datetime.now() - timedelta(day=60)
ninty_days_ago = datetime.now() - timedelta(day=90)
ctx = super().get_context_data(**kwargs)
ctx['articles_last90'] = \
Article.objects.filter(read_date__gt=ninty_days_ago).count()
ctx['articles_last60'] = \
Article.objects.filter(read_date__gt=sixty_days_ago).count()
ctx['last_last30'] = \
Article.objects.filter(read_date__gt=thirty_days_ago).count()
return ctx
This way you never ever load anything into python memory except the number of articles. It is much better than iterating or even using len(list_of_items).
You just have to be conscious of that read_time__gt=ninty_days_ago means articles that were last read more than ninty days ago.
See more from Django Docs.

Multiple querystring parameters

I've created this simple search function:
def search(request):
if "q" in request.GET:
querystring = request.GET.get("q")
print(querystring)
if len(querystring) == 0:
return redirect("/search/")
posts = Blog.objects.filter(title__icontains=querystring | tagline__icontains=querystring | contents__icontains=querystring)
context= {"posts": posts}
return render(request, "kernel/search.html", context)
else:
return render(request, "kernel/search.html")
When I use only one condition, for example:
posts = Blog.objects.filter(title__icontains=querystring)
it's shown me the correct results. But when I use multiple parameters I have SyntaxError: invalid syntax.
I was sure the query corresponded to:
SELECT * FROM Post WHERE "title" is "key_search" or "tagline" is
"key_search" or "contents" is "key_search"
How I can resolve?
The above is incorrect Python syntax, you can not put operators between named parameters.
Django however has Q objects [Django-doc] to express "conditions", so you can wrap the conditions in Q objects, and use the | operator to express a logical or, like:
from django.db.models import Q
posts = Blog.objects.filter(
Q(title__icontains=querystring) |
Q(tagline__icontains=querystring) |
Q(contents__icontains=querystring)
)
This will result in a query that looks, more or less, like:
SELECT *
FROM Post
WHERE "title" LIKE "%key_search%"
OR "tagline" LIKE "%key_search%"
OR "contents" LIKE "%key_search%"

How to optimize django query filter?

Basically I have this url dispatcher that capture a search term with each word separated by + to be search in the query. I have done this this is works but I think this will hit the performance due to repeated search to the database. Is there a better way to do this?
def search(request, **kwargs):
context = RequestContext(request)
test = {}
result = BlogPage.objects.select_related('ImageMedia')
if 'search_content' in kwargs:
test['search_content'] = kwargs['search_content']
if kwargs['search_content'] != '0':
search_words = kwargs['search_content'].split('+')
for words in search_words:
result = result.filter(content__icontains=words)
context_dict = {'blog_list': result}
return render_to_response('blog/blog_search.html', context_dict, context)
You could pre-build your filter, like (untested):
from django.db import Q
search_content = kwargs.get('search_content', '')
myfilter = Q()
for term in search_content.split('+'):
myfilter |= Q(content__icontains=term)
result = BlogPage.objects.filter(myfilter).select_related('ImageMedia')
I think Python list to bitwise operations is simpler when you are using python 2.7
PS: reduce() function has been removed in Python 3, so you can't use it and it is no compatibility.reduce() has been move into module functools

How to capture multiple arguments using a single RegEx in my Django urls.py?

I've got an application that allows you to filter data via 3 fields. I'm trying to write a RegEx in my urls.py that can capture multiple combinations without having to write-out each possible combination it it's own URL.
Here's my urls.py:
#urls.py
urlpatterns = patterns('',
# Uncomment the next line to enable the admin:
(r'^admin/', include(admin.site.urls)),
(r'(?P<key>\w*?)=(?P<value>\w*?)&|$', views.scriptFilter),
I tested the above RegEx at pythonregex.com and it appears to capture as many key/value pairs as I can throw at it. However, when I test try it in my app, Django only returns a queryset based on the first pair, and ignores the other pairs.
For example, if I enter this:
http://MYSITE/feature=1&session=1&
Django returns the data based on feature=1 only and ignores session=1.
Here's my views.py:
#views.py
def scriptFilter(request, key, value):
if key == 'session':
sessionID = value
qs = models.Script.objects.filter(session=sessionID)
elif key == 'product':
productID = value
qs = models.Script.objects.filter(product=productID)
elif key == 'feature':
featureID = value
scriptFeature = models.Feature.objects.get(pk=featureID)
qs = models.Script.objects.filter(feature=scriptFeature)
else:
qs = models.Script.objects.all()
caseTotal = qs.aggregate(Sum('caseCount'))
scriptTotal = qs.aggregate(Count('subScriptName'))
featureTotal = qs.aggregate(Count('feature'))
return render_to_response('scripts.html', locals())
I'm new to Python & Django so please be gentle :) Any help would be really appreciated.
These may be valid URLs (not entirely certain) but they're certainly not recommended.
If you want to allow parameters sent into your application using key-value pairs (as you are doing here), I'd suggest just using query parameters. Here's a way to implement that in your view:
def scriptFilter(request, session=None, product=None, feature=None):
session = request.REQUEST.get('session',session)
product = request.REQUEST.get('product',session)
feature = request.REQUEST.get('feature',session)
if session
qs = models.Script.objects.filter(session=session)
elif product:
qs = models.Script.objects.filter(product=product)
elif feature:
qs = models.Script.objects.filter(feature=feature)
else:
qs = models.Script.objects.all()
caseTotal = qs.aggregate(Sum('caseCount'))
scriptTotal = qs.aggregate(Count('subScriptName'))
featureTotal = qs.aggregate(Count('feature'))
return render_to_response('scripts.html', locals())
You'd then call the URLs as
http://example.com/myapp?session=X
http://example.com/myapp?product=X
http://example.com/myapp?session=X&feature=Y
etc.
Note that I assume when you say that pythonregex captured all the groups, that's probably because you were looking at the .findall() response. I'm not sure of the exact mechanics of Django's url dispatcher, but even if you just think about it logically, how could it assign more than one value to key and value? Your scriptFilter function does not even handle multiple values being sent in. You really want it to read:
def scriptFilter(request, session=None, product=None, feature=None):
session = request.REQUEST.get('session',session)
product = request.REQUEST.get('product',session)
feature = request.REQUEST.get('feature',session)
qs = models.Script.objects.all()
if session
qs = qs.filter(session=session)
if product:
qs = qs.filter(product=product)
if feature:
qs = qs.filter(feature=feature)
caseTotal = qs.aggregate(Sum('caseCount'))
scriptTotal = qs.aggregate(Count('subScriptName'))
featureTotal = qs.aggregate(Count('feature'))
return render_to_response('scripts.html', locals())
Finally, I'm guessing you should rewrite these lines as well:
caseTotal = qs.aggregate(Sum('caseCount'))
scriptTotal = qs.aggregate(Count('subScriptName'))
featureTotal = qs.aggregate(Count('feature'))
The aggregate function creates a QuerySet with the aggregated values. You may as well group these into one queryset:
script_totals = qs.aggregate(Sum('casecount'), Count('subscriptname'), Count('feature'))
In the template, you'd access these values as:
{{ script_totals.casecount__sum }}
{{ script_totals.subscriptname__count }}
{{ script_totals.feature__count }}
Which is a bit cleaner than
{{ caseTotal.casecount__sum }}
{{ scriptTotal.subscriptname__count }}
{{ featureTotal.feature__count }}