How to structure views for database pulling and manipulation - django

I am having difficulties understanding how best to structure my views.
I am pulling data on various users and creating variables that summarise some of these variables (such as occurrences per week etc). This is so I can graph these summary variables in my templates. I am doing quite a lot of different manipulations which is getting quite messy , and i shall need these manipulations for other templates. Can somebody recommend how best to structure views in this case. I think using classes is the solution to use the same functions for other templates but I cannot quite understand how. I also feel there must be a better way to structure each manipulation of database data.
def dashboard(request):
posts= Post.objects.filter(user=request.user)
posts_count = posts.count()
post_early = Post.objects.filter(user=request.user).earliest('date') #need to extract the date value from this so I can take the difference
total_days = (datetime.datetime.now().date()- post_early.date).days
average_30days= round((posts_count/total_days)*30,2)
list4=[]
list5=[]
i=1
time3=datetime.datetime.now() + datetime.timedelta(-30)
while i<32:
list4.append(days2(time3,request,Post))
list5.append(time3.strftime('%b %d, %Y'))
i+=1
time3=time3 + datetime.timedelta(+1)

def dashboardView(request):
posts = Post.objects.filter(user=request.user)
posts_count = posts.count()
#need to extract the date value from post_early so I can take the difference
post_early = Post.objects.filter(user=request.user).earliest('date')
total_days = (datetime.datetime.now().date() - post_early.date).days
average_30days = round((posts_count/total_days)*30,2)
list_4 = []
list_5 = []
i = 1
time_3=datetime.datetime.now() + datetime.timedelta(-30)
while i<32:
list_4.append(days2(time_3, request, Post))
list_5.append(time_3.strftime('%b %d, %Y'))
i += 1
time_3 = time_3 + datetime.timedelta(1)
I'd do something like this. There were a few inconsistency:
-keep a space before and a space after operators (=, *, -, +, ...).
-I'd consider a good practice to always suffix -View to your views, but it's just personal preference
-Use empty lines to separate blocks of code, not groups of variables. If you have a long list of variables declarations (not this case) you can use comments to separate and categorize them.
-Use list_3 instead of list3 (and similar cases), it's more readable.
For more you can always check the official python style guide: https://www.python.org/dev/peps/pep-0008/
Anyway, if you're consistent and attain to the coding style used in the Django documentation while learning, you'll be fine.
##### EDIT:
Note: my answer is based on the code you provided, which seems cut (no return statement?) and without the other modules.
You're using a function based view which is not wrong nor correct, just one of the possible choices. If you don't like it, or want to try something else, a ListView may work for you: https://docs.djangoproject.com/en/2.1/topics/class-based-views/generic-display/
Example:
from django.views import ListView
class DashboardView(ListView):
model = Post
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
context['posts'] = Post.objects.filter(user=request.user)
# add all the data you need to the context dictionary
return context

Related

How can I filter objects with the model method - Django [duplicate]

Is it possible to filter a Django queryset by model property?
i have a method in my model:
#property
def myproperty(self):
[..]
and now i want to filter by this property like:
MyModel.objects.filter(myproperty=[..])
is this somehow possible?
Nope. Django filters operate at the database level, generating SQL. To filter based on Python properties, you have to load the object into Python to evaluate the property--and at that point, you've already done all the work to load it.
I might be misunderstanding your original question, but there is a filter builtin in python.
filtered = filter(myproperty, MyModel.objects)
But it's better to use a list comprehension:
filtered = [x for x in MyModel.objects if x.myproperty()]
or even better, a generator expression:
filtered = (x for x in MyModel.objects if x.myproperty())
Riffing off #TheGrimmScientist's suggested workaround, you can make these "sql properties" by defining them on the Manager or the QuerySet, and reuse/chain/compose them:
With a Manager:
class CompanyManager(models.Manager):
def with_chairs_needed(self):
return self.annotate(chairs_needed=F('num_employees') - F('num_chairs'))
class Company(models.Model):
# ...
objects = CompanyManager()
Company.objects.with_chairs_needed().filter(chairs_needed__lt=4)
With a QuerySet:
class CompanyQuerySet(models.QuerySet):
def many_employees(self, n=50):
return self.filter(num_employees__gte=n)
def needs_fewer_chairs_than(self, n=5):
return self.with_chairs_needed().filter(chairs_needed__lt=n)
def with_chairs_needed(self):
return self.annotate(chairs_needed=F('num_employees') - F('num_chairs'))
class Company(models.Model):
# ...
objects = CompanyQuerySet.as_manager()
Company.objects.needs_fewer_chairs_than(4).many_employees()
See https://docs.djangoproject.com/en/1.9/topics/db/managers/ for more.
Note that I am going off the documentation and have not tested the above.
Looks like using F() with annotations will be my solution to this.
It's not going to filter by #property, since F talks to the databse before objects are brought into python. But still putting it here as an answer since my reason for wanting filter by property was really wanting to filter objects by the result of simple arithmetic on two different fields.
so, something along the lines of:
companies = Company.objects\
.annotate(chairs_needed=F('num_employees') - F('num_chairs'))\
.filter(chairs_needed__lt=4)
rather than defining the property to be:
#property
def chairs_needed(self):
return self.num_employees - self.num_chairs
then doing a list comprehension across all objects.
I had the same problem, and I developed this simple solution:
objects = [
my_object
for my_object in MyModel.objects.all()
if my_object.myProperty == [...]
]
This is not a performatic solution, it shouldn't be done in tables that contains a large amount of data. This is great for a simple solution or for a personal small project.
PLEASE someone correct me, but I guess I have found a solution, at least for my own case.
I want to work on all those elements whose properties are exactly equal to ... whatever.
But I have several models, and this routine should work for all models. And it does:
def selectByProperties(modelType, specify):
clause = "SELECT * from %s" % modelType._meta.db_table
if len(specify) > 0:
clause += " WHERE "
for field, eqvalue in specify.items():
clause += "%s = '%s' AND " % (field, eqvalue)
clause = clause [:-5] # remove last AND
print clause
return modelType.objects.raw(clause)
With this universal subroutine, I can select all those elements which exactly equal my dictionary of 'specify' (propertyname,propertyvalue) combinations.
The first parameter takes a (models.Model),
the second a dictionary like:
{"property1" : "77" , "property2" : "12"}
And it creates an SQL statement like
SELECT * from appname_modelname WHERE property1 = '77' AND property2 = '12'
and returns a QuerySet on those elements.
This is a test function:
from myApp.models import myModel
def testSelectByProperties ():
specify = {"property1" : "77" , "property2" : "12"}
subset = selectByProperties(myModel, specify)
nameField = "property0"
## checking if that is what I expected:
for i in subset:
print i.__dict__[nameField],
for j in specify.keys():
print i.__dict__[j],
print
And? What do you think?
i know it is an old question, but for the sake of those jumping here i think it is useful to read the question below and the relative answer:
How to customize admin filter in Django 1.4
It may also be possible to use queryset annotations that duplicate the property get/set-logic, as suggested e.g. by #rattray and #thegrimmscientist, in conjunction with the property. This could yield something that works both on the Python level and on the database level.
Not sure about the drawbacks, however: see this SO question for an example.

Queryset in Django if empty field returns all elements

I want to do a filter in Django that uses form method.
If the user type de var it should query in the dataset that var, if it is left in blank to should bring all the elements.
How can I do that?
I am new in Django
if request.GET.get('Var'):
Var = request.GET.get('Var')
else:
Var = WHAT SHOULD I PUT HERE TO FILTER ALL THE ELEMNTS IN THE CODE BELLOW
models.objects.filter(Var=Var)
It's not a great idea from a security standpoint to allow users to input data directly into search terms (and should DEFINITELY not be done for raw SQL queries if you're using any of those.)
With that note in mind, you can take advantage of more dynamic filter creation using a dictionary syntax, or revise the queryset as it goes along:
Option 1: Dictionary Syntax
def my_view(request):
query = {}
if request.GET.get('Var'):
query['Var'] = request.GET.get('Var')
if request.GET.get('OtherVar'):
query['OtherVar'] = request.GET.get('OtherVar')
if request.GET.get('thirdVar'):
# Say you wanted to add in some further processing
thirdVar = request.GET.get('thirdVar')
if int(thirdVar) > 10:
query['thirdVar'] = 10
else:
query['thirdVar'] = int(thirdVar)
if request.GET.get('lessthan'):
lessthan = request.GET.get('lessthan')
query['fieldname__lte'] = int(lessthan)
results = MyModel.objects.filter(**query)
If nothing has been added to the query dictionary and it's empty, that'll be the equivalent of doing MyModel.objects.all()
My security note from above applies if you wanted to try to do something like this (which would be a bad idea):
MyModel.objects.filter(**request.GET)
Django has a good security track record, but this is less safe than anticipating the types of queries that your users will have. This could also be a huge issue if your schema is known to a malicious site user who could adapt their query syntax to make a heavy query along non-indexed fields.
Option 2: Revising the Queryset
Alternatively, you can start off with a queryset for everything and then filter accordingly
def my_view(request):
results = MyModel.objects.all()
if request.GET.get('Var'):
results = results.filter(Var=request.GET.get('Var'))
if request.GET.get('OtherVar'):
results = results.filter(OtherVar=request.GET.get('OtherVar'))
return results
A simpler and more explicit way of doing this would be:
if request.GET.get('Var'):
data = models.objects.filter(Var=request.GET.get('Var'))
else:
data = models.objects.all()

Convert Boolean expression to django queryset

I have tagging in place on my django site, and I'd like to allow urls of the form:
http://example.com/search/tags/(foo+dog)|(goat+cat)
Which in English would mean:
Find the items tagged with (foo AND dog) OR (goat AND cat).
So essentially what I need is a way to pare this down into queries using the Django API. At present, I just want to support AND, OR and parentheses.
I imagine there are libraries for interpreting Booleans of this sort, but I haven't been able to find any outside of a full-blown search engine. Are there any tricks to doing this or good starting points using the Django API?
Right now, my code is pretty basic, but it supports either OR queries or AND queries, but not them combined (thus no parentheses either).
EDIT:
I'm fairly convinced that if I could sort this out into a series of AND and OR queries, I'd be all set...but I can't think through how to go from the randomly parenthesised boolean query to a logically useful understanding of the query.
Here's the code I have so far, in case it's useful. I'm not using the tagging module (though maybe I should), and the code is still drafty, but...
#login_required
def view_opinions_by_tag(request, tagValues):
'''Displays opinions tagged by a user with certain tags.
Given a set of tags separated by pluses, pipes, and parentheses, unpack
the set of tags and display the correct opinions. Currently only supports
pluses (AND filters), and pipes (OR filters).
'''
if '|' in tagValues:
# it's an or query.
tagList = tagValues.split('|')
tags = Tag.objects.filter(tag__in = tagList, user = request.user)\
.values_list('pk', flat=True)
faves = Favorite.objects.filter(tags__in = list(tags), user = request.user).distinct()
elif '+' in tagValues:
# it's an and query - not very efficient.
tagList = tagValues.split('+')
tagObject = Tag.objects.get(tag = tagList[0], user = request.user)
faves = Favorite.objects.filter(tags = tagObject, user = request.user)
for tag in tagList[1:]:
tagObject = Tag.objects.filter(tag = tag, user = request.user)
faves = faves.filter(tags = tagObject, user = request.user).distinct()
else:
# it's a single tag
tag = Tag.objects.get(tag = tagValues, user = request.user)
faves = Favorite.objects.filter(tags = tag, user = request.user)
From here, I essentially take the faves queryset, and render it. Perhaps not the most efficient, but seems to work so far.
Use Q objects.
For the parsing, just set a regexp in your urls.py that will get you the stuff that you need:
url(r'^/search/tags/(?p<query>[a-z\-\+\|])$')
In your views:
def your_view(request, query):
for and_exp in query.split('|'):
for tag in and_exp.split('+'):
# do your stuff with q object
EDIT:
Actually this is a generic answer, but I just realized that you are talking about tags, and therefor a many to many relation.
It will depends very much of the implementation so you must give us more details.
If you use django-tagging, you can make several queries using Object.tagged.with_all('tag1', 'tag2')
It's different for django-taggit, and different if you are using your own implementation.

Django most efficient way to do this?

I have developed a few Django apps, all pretty straight-forward in terms of how I am interacting with the models.
I am building one now that has several different views which, for lack of a better term, are "canned" search result pages. These pages all return results from the same model, but they are filtered on different columns. One page we might be filtering on type, another we might be filtering on type and size, and on yet another we may be filtering on size only, etc...
I have written a function in views.py which is used by each of these pages, it takes a kwargs and in that are the criteria upon which to search. The minimum is one filter but one of the views has up to 4.
I am simply seeing if the kwargs dict contains one of the filter types, if so I filter the result on that value (I just wrote this code now, I apologize if any errors, but you should get the point):
def get_search_object(**kwargs):
q = Entry.objects.all()
if kwargs.__contains__('the_key1'):
q = q.filter(column1=kwargs['the_key1'])
if kwargs.__contains__('the_key2'):
q = q.filter(column2=kwargs['the_key2'])
return q.distinct()
Now, according to the django docs (http://docs.djangoproject.com/en/dev/topics/db/queries/#id3), these is fine, in that the DB will not be hit until the set is evaluated, lately though I have heard that this is not the most efficient way to do it and one should probably use Q objects instead.
I guess I am looking for an answer from other developers out there. My way currently works fine, if my way is totally wrong from a resources POV, then I will change ASAP.
Thanks in advance
Resource-wise, you're fine, but there are a lot of ways it can be stylistically improved to avoid using the double-underscore methods and to make it more flexible and easier to maintain.
If the kwargs being used are the actual column names then you should be able to pretty easily simplify it since what you're kind of doing is deconstructing the kwargs and rebuilding it manually but for only specific keywords.
def get_search_object(**kwargs):
entries = Entry.objects.filter(**kwargs)
return entries.distinct()
The main difference there is that it doesn't enforce that the keys be actual columns and pretty badly needs some exception handling in there. If you want to restrict it to a specific set of fields, you can specify that list and then build up a dict with the valid entries.
def get_search_object(**kwargs):
valid_fields = ['the_key1', 'the_key2']
filter_dict = {}
for key in kwargs:
if key in valid_fields:
filter_dict[key] = kwargs[key]
entries = Entry.objects.filter(**filter_dict)
return entries.distinct()
If you want a fancier solution that just checks that it's a valid field on that model, you can (ab)use _meta:
def get_search_object(**kwargs):
valid_fields = [field.name for field in Entry._meta.fields]
filter_dict = {}
for key in kwargs:
if key in valid_fields:
filter_dict[key] = kwargs[key]
entries = Entry.objects.filter(**filter_dict)
return entries.distinct()
In this case, your usage is fine from an efficiency standpoint. You would only need to use Q objects if you needed to OR your filters instead of AND.

Django Newbie here... Is there a good way of handling empty MultipleChoiceField results (like a List-Comp format?)

I came across this blog entry which describes an elegant way of handling results returned from a ChoiceField to a view using a list comprehension technique - one that eliminates empty keys/values without all the intermediate data structures. This particular approach doesn't seem to work for MultipeChoiceFields, though. Is there a similar way one might approach those? (If, for example, the bedroom and bathrooms fields in the following example returned multiple values).
The code is as follows:
if search_form.is_valid():
searchdict = search_form.cleaned_data
# It's easier to store a dict of the possible lookups we want, where
# the values are the keyword arguments for the actual query.
qdict = { 'city': 'city__icontains',
'zip_code': 'zip_code',
'property_type': 'property_type_code',
'county': 'county__icontains',
'minimum_price': 'sale_price__gte',
'maximum_price': 'sale_price__lte',
'bedrooms': 'bedrooms__gte',
'bathrooms': 'baths_total__gte'}
# Then we can do this all in one step instead of needing to call
# 'filter' and deal with intermediate data structures.
q_objs = [Q(**{qdict[k]: searchdict[k]}) for k in qdict.keys() if searchdict.get(k, None)]
Thank you very much...this is an awesome community.
Hmm... tricky one, the problem is that you only want that specific case to work as multivalued so you can't use the "normal" approach of adding the filters.
Atleast... I'm assuming that if someone selects multiple bathrooms that it would be either instead of both.
This is too much for a one liner I think, but it can work ;)
import operator
create_qs = lambda k, vs: reduce(operator.or_, [Q(**{k: v}) for v in vs])
q_objs = [create_qs(k, searchdict.getlist(k)) for k in qdict.keys() if k in searchdict]
You might be looking for the IN field lookup like this:
'bathrooms': 'baths_total__in',