In my html file, how can I output the size of the queryset that I am using (for my debugging purposes)
I've tried
{{ len(some_queryset) }}
but that didn't work. What is the format?
Give {{ some_queryset.count }} a try.
This is better than using len (which could be invoked with {{ some_queryset.__len__ }}) because it optimizes the SQL generated in the background to only retrieve the number of records instead of the records themselves.
some_queryset.count() or {{some_queryset.count}} in your template.
dont use len, it is much less efficient. The database should be doing that work. See the documentation about count().
However, taking buffer's advice into account, if you are planning to iterate over the records anyway, you might as well use len which will involve resolving the queryset and making the resulting rows resident in main memory - this wont go to waste because you will visit these rows anyway. It might actually be faster, depending on db connection latency, but you should always measure.
Just to highlight #Yuji'Tomita'Tomita comment above as a separate answer:
There is a filter called length to call len() on anything.
So you could use:
{{ some_queryset|length }}
The accepted answer is not entirely correct. Whether you should use len() (or the length-filter in a template) vs count() depends on your use case.
If the QuerySet only exists to count the amount of rows, use count().
If the QuerySet is used elsewhere, i.e. in a loop, use len() or |length. Using count() here would issue another SELECT-query to count the rows, while len() simply counts the amount of cached results in the QuerySet.
From the docs:
Note that if you want the number of items in a QuerySet and are also retrieving model instances from it (for example, by iterating over it), it’s probably more efficient to use len(queryset) which won’t cause an extra database query like count() would.
Although it seems that with related objects that you have already eager-loaded using prefetch_related(), you can safely use count() and Django will be smart enough to use the cached data instead of doing another SELECT-query.
Related
I have multiple small key/value tables in Django, and there value never change
ie: 1->"Active", 2->"Down", 3->"Running"....
and multiple times, I do some get by id and other time by name.
So I'm asking, if it's not more optimize to move them all as Dict (global or in models) ?
thank you
Generally django querysets are slower than dicts, so if you want to write model with one field that has these statuses (active, down, running) it's generally better to use dict until there is need for editability.
Anyway I don't understand this kind of question, the performance benefits are not really high until you got ~10k+ records in single QS, and even by then you can cast the whole model to list by using .values_list syntax. Execution will take approximately part of second.
Also if I understand, these values should be anyway in models.CharField with choices field set, rather than set up by fixture in models.ForeignKey.
I am currently using taggableManager as my tags in django. I can see in admin what is currently using the tag but then is there a way to count them?
let's say I have the follow tag and as can see there are a 4 objects using this tag. how can I get the count of 4 for this tag?
Thanks in advance
You will want to do a typical query on your database for the count of rows for a particular tag. Instead of looking at the len of a queryset there is another count feature less commonly known in Django which gives you the count number from SQL as opposed to having to query the entire database just to get the length.
len(). A QuerySet is evaluated when you call len() on it. This, as you
might expect, returns the length of the result list.
Note: If you only need to determine the number of records in the set
(and don’t need the actual objects), it’s much more efficient to
handle a count at the database level using SQL’s SELECT COUNT(*).
Django provides a count() method for precisely this reason.
https://docs.djangoproject.com/en/1.11/ref/models/querysets/
On a Django website of mine, users contribute posts, which are then showed globally on the home page, sorted by most-recent first.
I'm introducing redis into this mix, via doing an lpush of all post_ids into a redis list (which is kept trimmed at 1000 entries). The code is:
def add_post(link_id):
my_server = redis.Redis(connection_pool=POOL)
my_server.lpush("posts:1000", link_id)
my_server.ltrim("posts:1000", 0, 9999)
Then, when a user requests the contents of the home page, I simply execute the following query in the get_queryset method of the relevant class-based view:
Post.objects.filter(id__in=all_posts())
Where all_posts() is simply:
def all_posts():
my_server = redis.Redis(connection_pool=POOL)
return my_server.lrange("posts:1000", 0, -1)
Next, I iterate over the context["object_list"] in a Django template (i.e. {% for post in object_list %}, and one by one populate the latest posts for my users to see.
My problem is that this arrangement does not show most-recent first. It always shows most-recent last. So I changed lpush to rpush instead, but the result didn't change at all. Why isn't changing redis' list insert method changing the ordering of the results Django's queryset is returning to me?
Perhaps I'm missing something rudimentary. Please advise me on what's going on, and how can I fix this (is {% for post in object_list reversed %} my sole option here). My reason for taking the redis route was, naturally, performance. Prior to redis, I would do: Post.objects.order_by('-id')[:1000] Thanks in advance.
Note: please ask for more information if required.
You're iterating through a queryset that doesn't have an order_by clause, which means that you can't have any expectations about the order or the results. The __in clause just controls which rows to return, not their order.
The fact that the returned results are in the id order is an implementation detail. If you want to rely on that, you can just iterate through the queryset in reverse order. A more robust solution would be to reorder (in Python) the instances based on the order of the ids returned from Redis.
All that said, though, I don't think there will be any performance advantage to using Redis here. I think that any relational database with an index on id will be able to execute Post.objects.order_by('-id')[:1000] very efficiently. (Note that slicing a queryset does a LIMIT on the database; you're not fetching all the rows into Python and then slicing a huge list.)
I am loving Django, and liking its implemented pagination functionality. However, I encounter issues when attempting to split a randomly ordered queryset across multiple pages.
For example, I have 100 elements in a queryset, and wish to display them 25 at a time. Providing the context object as a queryset ordered randomly (with the .order_by('?') specification), a completely new queryset is loaded into the context each time a new page is requested (page 2, 3, 4).
Explicitly stated: how do I (or can I) request a single queryset, randomly ordered, and display it across digestible pages?
I ran into the same problem recently where I didn't want to have to cache all the results.
What I did to resolve this was a combination of .extra() and raw().
This is what it looks like:
raw_sql = str(queryset.extra(select={'sort_key': 'random()'})
.order_by('sort_key').query)
set_seed = "SELECT setseed(%s);" % float(random_seed)
queryset = self.model.objects.raw(set_seed + raw_sql)
I believe this will only work for postgres. Doing a similar thing in MySQL is probably simpler since you can pass the seed directly to RAND(123).
The seed can be stored in the session/a cookie/your frontend in the case of ajax calls.
Warning - There is a better way
This is actually a very slow operation. I found this blog post describes a very good method both for retrieving a single result as well as sets of results.
In this case the seed will be used in your local random number generator.
i think this really good answer will be useful to you: How to have a "random" order on a set of objects with paging in Django?
basically he suggests to cache the list of objects and refer to it with a session variable, so it can be maintained between the pages (using django pagination).
or you could manually randomize the list and pass a seed to maintain the randomification for the same user!
The best way to achive this is to use some pagination APP like:
pure-pagination
django-pagination
django-infinite-pagination
Personally i use the first one, it integrates pretty well with Haystack.
""" EXAMPLE: (django-pagination) """
#paginate 10 results.
{% autopaginate my_query 10 %}
So, my problem is this. I have a legacy MySQL database that I'm building a shiny new Django application over. For whatever reason, some fairly daft design decisions were made—such as all fields, no matter what they contain, being stored as varchars—but because since other systems that I'm not rewriting depend on that same data I can't destructively change that schema at all.
I want to treat a certain field—the stock quantity on hand—as an integer, so that in my template I can check its amount and display a relevant value (basically, if there are more than 100 items available, I want to just display "100+ Available").
The existing value for stock quantity is stored as, oddly, a varchar holding a float (as if it's possible to have fractional amounts of an item in stock):
item.qty: u"72.0"
Now, I figure as a worst case I can use QuerySet.values(), and iterate over the results, replacing each stock quantity with an int() parsed version of itself. Something like ...
item_list = items.values()
for item in item_list:
item['qty'] = int(float(item['qty']))
... but won't that cause my QuerySet to evaluate itself completely? I confess to being fairly ignorant of the process by which Django handles lazy execution of queries, but it seems like working with actual values would mean evaluating the query before it needs to.
So, am I complaining about nothing (I mean, it's definitely evaluating these values in the template anyway), or is there a better way to do what I need to do?
Yes, iterating through in the view would evaluate the entire queryset. That may or not be what you want - for example, if you're paginating, the paginator limits the query automatically, so you don't want to evaluate the whole thing separately.
I would approach this in a different way, by calling a function in the template to format the data. You can either do this with a method on the model, if there's just one field you want to format:
class Item(models.Model):
...
def get_formatted_qty(self):
return int(float(self.qty))
and call it:
{{ item.get_formatted_qty }}
Or, to make it more general, you can define a custom filter in your templatetags:
#register.filter
def format_value(value):
return int(float(value))
{{ item.qty|format }}