Django: optimizing queries - django

I want to list the number of items for each list. How can I find this number in a single query, rather than a query for each list?
Here is a simplified version of my current template code:
{% for list in lists %}
<li>
{{ listname }}:
{% with list.num_items as item_count %}
{{ item_count }} item{{ item_count|pluralize }}
{% endwith %}
</li>
{% endfor %}
lists is passed as: List.objects.filter(user=user)
and num_items is a property of the List model:
def _get_num_items(self):
return self.item_set.filter(archived=False).count()
num_items = property(_get_num_items)
This queries SELECT COUNT(*) FROM "my_app_item" WHERE... n times, where n is the number of lists. Is it possible to make a single query here?

You should do this in your view and send the dictionary, instead.
Model.objects.values_list('item').annotate(Count('num_items'))
This will produce the SQL same as, (or equivalent to) the one you have posted.

In the following, I tried to take into account all your constraints: filtering on fields of List, on fields of Item, counting items , and grouping by list.
The solution I see is that you could use values() (here is the django doc about this : http://docs.djangoproject.com/en/dev/topics/db/aggregation/#values)
from django.db.models import Count
lists = list(List.objects.filter(user=user))
items=Item.objects.values(list).filter(archived=False,list__in=lists).annotate(count=Count("id"))
#you will get a list of dicts of the form [{'count':2,'list':5},...] where 5 is the id of the list
#now, you can match you list with you item counts in python
list_items_count_dict={}
for item in items:
list_items_count_dict[item['list']]=item['count']
for list in lists :
list.item_count = list_items_count_dict.get(list.id)
That will make only 2 queries, one for getting the lists, the other for computing the item counts. Afterwards, you will have two loops (that could probably be replaced by list comprehension one-liners ), but only for the lists you are interested in.
afterwards, in your template, you can use
{{list.item_count}}
There might be a more elegant option, but that is what I have found right now. I am also certain that you could reduce the number of query to one by using custom sql.
Disclaimer: I have not tested this code, but I have tested similar code on similar models. You could have problems because list is one of the keyword of the Python language.

Related

How to join two tables into a dictionary in dbt jinja

I want to create a for loop in dbt that loops through two columns of a table in my database and created case when statements using the values in each row of the two columns. Just like this:
{% set locations = {'%United-States%':'United States', '%USA%':'United States'} %}
select
case
{% for feeder, correct in locations.items() %}
when lower(locationname) like {{feeder}} then {{correct}}
{% endfor %}
end as city
from table
I was able to create lists for both feeder and correct, but I am not able to merge them as key-value pairs of a dictionary to loop through it. Any ideas on how should I do this?
Sounds like you have two problems:
fetch data from another table, and
use that data to populate a case when statement.
Big thing to remember is that the main thing that dbt-jinja does is create a string of SQL. That said, there is clever functionality that lets you query a database before jinja starts to put the string together.
The idea being:
fetch the values from the database that you want to have included in your SELECT query
populate the previously fetched values into the query while rendering the statement
there's two macros that may be of use in the first step:
run_query() (docs), and
dbt-utils' get_query_results_as_dict() (docs)
Something like this might work (provided you already have dbt-utils installed:
{% set locations_query %}
select feeder, correct from my_other_table
{% endset %}
{% set locations = run_query(locations_query) %}
select
case
-- not sure how this part will work yet....
{% for feeder, correct in locations.items() %}
when lower(locationname) like {{feeder}} then {{correct}}
{% endfor %}
end as city
from table

How to test if it is a list or a value in a Django template?

In a template in Django I am importing a list of dicts and in one of the keys (tested) I can have either a single value or a list of values depending on the case.
My context dict in the template looks something like this:
context_dicts.append({'url': p.url,
'type_': p.type,
'error': error,
'tested': p.tested})
In the html template I want to if test the tested key to do something if it is a single value and something else if it's a list. So when looping through the dicts, if I use {% if value|length > 1% } it will give me the string size of the value when it's just a value and the length of the list when it's a list. How can I test the if to tell me if it's a "list of one value" or more?
Welcome to SO!
Storing possibly different kind of data in a single variables looks cumbersome to me. It makes your logic less easy to understand and may be prone to errors.
I think the best is to always store a list, but possibly a list with a single element. In the template, you could then do:
{% if tested.count == 1 %}
do stuff with {{ tested.0 }} value
{% else %}
do stuff with {{ tested }} list
{% endif %}

Length of list in jinja template, where the list is the result of a DBSession.query

How do I find the length of a list of objects in a jinja template when the list of objects has been created by querying a database?
I thought There are {{ items|length }} items in this category. would work, but items is a result of:
items = db_session.query(Item).filter_by(category_id=category.id)
and I get the error
TypeError: object of type 'Query' has no len()
Obviously, I could calculate the length separately and pass into render_template(), but I wondered if there was a better way?
Any help from the community would be much appreciated :)
items object is not a list yet, it is an unprocessed Query object as you see in the error. You can use Query.all() method to get a list of items:
items = db_session.query(Item).filter_by(category_id=category.id).all()
After that length filter can be applicable.
Use {{ items | count }} in jinja template
Try to add loopcontrol extension
app.jinja_env.add_extension('jinja2.ext.loopcontrols')
Then
{% for item in items %}
{{loop.length}}
{% break %}
{% endfor %}

Django - iterate through 2 objects synchronously in the template form

How do I iterate through 2 objects synchronously in the template form?
I have 2 objects in my views; one object is products, the other prices. Both are lists.
I want to cycle through both products and prices at the same time in the templates form.
Usually in python I would just make a simple while loop as such:
n = 0
while n < len(products):
print products[n], prices[n]
n+=1
I haven't found anything similar in the templates form yet.
Any help would be appreciated.
You could write a custom template filter to get the nth item in the list, but this is what I would do:
Zip the 2 lists together in the view:
products_prices = zip(products, prices)
Then loop through that looped list in the template:
{% for product, prices in products_prices %}
{{ product }}, {{ price }}
{% endfor %}
You might want to use forloop.counter, docs at https://docs.djangoproject.com/en/dev/ref/templates/builtins/
The built in template tags don't have an easy way to do this that I can recall. The simplest approach will be to zip the two lists together in your view, then provide the zipped data structure in the context. You could also write a custom zip template tag, if you're feeling adventurous.

Whats the difference between QuerySet , Tuple , Dictionary in Django template

I am having trouble in understanding how to iterate over QuerySet , Tuple and Dictionarty in django.
I am confused which djnago functions returns what like objects.all or objects.get
Suppose i have
a = model.objects.all()
b = model.object.get(pk=1)
c = Blog.objects.values('name', 'entry__headline')
d = Entry.objects.values_list('id', 'headline')
e = Person.objects.raw('SELECT * FROM myapp_person')
What is the retured in each scenario and the biggest problem is how can i iterate over.
All these confuse me very much . I studies the docs but they tell one thing and don't tell how to use in template. I know its related to python but then python don't have template to deal with
QuerySet: A Django class that processes SQL responses and returns a python construct representing the results. Although it functions like a list in many ways, it's actually what's called an "iterable". It simply mocks the behavior of a list to allow you to use things like for-loops on it.
Tuple: An immutable list. That means that once it's set, it can't be altered. In virtually every other way it behaves just like a list.
Dictionary: Also known as a hash in other languages. It can be considered a "keyed list". A "list" in the strictest of senses is a group of items stored serially in memory. In the old days of programming, you'd have to "pop" items off and "push" items onto a list, and they could only be retrieved in a FIFO, or first-in-first-out fashion. Dictionaries provide a way to "lookup" items in a list. It is composed of key-value pairs, so you can reference a key and get the attached value.
Now in terms of Django templates:
QuerySets: You iterate over these using the standard methods. Once you get a result from something like MyModel.objects.all(), you can use a {% for value in queryset %} tag.
Tuples: As far as iteration goes, these behave exactly as standard lists. You can also just simply use a {% for value in tuple %} tag. The only potential hangup is that sometimes you'll end up with tuples of tuples or a list of tuples. These are just like multi-level lists. You have to iterate over the outer list or tuple and then iterate over the inner ones.
Dictionaries: These are probably the most complicated, only because they require a method call to get an iterable object.
mydict.iteritems() # returns an iterable consisting of `(key, value)` tuples.
mydict.iterkeys() # returns an iterable consisting of the keys. You can then use mydict[key] to get the values
mydict.itervalues() # returns an iterable consisting of the values.
The last method is probably the best of simple iteration in a Django template:
{% for value in mydict.itervalues %}
Are you referring to these docs? https://docs.djangoproject.com/en/dev/ref/templates/builtins/#for
I think that's what you were looking for.
Basically you iterate over them like:
{% for item in a %}
{{item.field}}
{{item.field2}}
{% endfor %}
{{b.field}}
{% for item in c %}
{{item.name}}
{{item.entry__headline}}
{% endfor %}
{% for item in d %}
{{item}}
{% endfor %}
# Also you can do this if you want to access just a particular index:
{{d.0}}
{{d.1}}
{% for item in e %}
{{item.field}}
{{item.field2}}
{% endfor %}
As for your data types:
a would be a QuerySet or list of model objects
b would be a model object
c would be a ValuesQuerySet or a list of dictionaries
d would also be a ValuesQuerySet but it's actually a list of tuples
e would be a RawQuerySet, which acts like a normal QuerySet
Sources:
https://docs.djangoproject.com/en/dev/topics/db/sql/#django.db.models.Manager.raw
https://docs.djangoproject.com/en/dev/ref/models/querysets/#values
https://docs.djangoproject.com/en/dev/ref/models/querysets/#values-list