I fetch the latest 5 rows from a Foo model which is ordered by a datetime field.
qs = Foo.objects.all()[:5]
In the following step, I want to reorder the queryset by some other criteria (actually, by the same datetime field in the opposite direction). But reordering after a slice is not permitted. reverse() undoes the first ordering, giving me a differet queryset. Is there a way to accomplish what I want without creating a list from the queryset and doing the ordering using it?
order_by gives you SQL in-database ordering. You're already using that, and then slicing on it. At that point, the results are retrieved into memory. If you want to change their order, you need to use Python in-memory sorting to do it, not the ORM in-database sorting.
In your case, Daniel has already given the best solution: since you simply want to sort by the same field, but in the other order, just reverse the list you have:
qs = Foo.objects.all()[:5]
objs = reversed(qs)
If you had wanted to sort by some other field, then you'd use the sorted() function with a custom key function:
qs = Foo.objects.all()[:5]
objs = sorted(qs, key=lambda o: o.some_other_field)
No, there's no way of doing that. order_by is an operation on the database, but when you slice a queryset it is evaluated and doesn't go back to the database after that.
Sounds like you already know the solution, though: run reversed() on the evaluated qs.
qs = reversed(Foo.objects.all()[:5])
Late answer, but this just worked for me:
import random
sorted(queryset[:10], key=lambda x: random.random())
Related
I am writing a web based music application and I want implement some feature that user can see most favor album in last week-month-year.
so this is my model :
class album(models.Model):
def get_weely_count():
...
def get_monthly_count():
...
def get_yearly_count():
...
class like(models.Model):
created_at = models.DateField()
albumID = models.ForeignKey(Album)
Now I want to receive albums that most liked in last week or last month or last year,I want done some thing like this(but I can not):
Album.objects.all().order_by('get_weekly_count')
can any one help me to fix it or give another approach to achieve that goal??
The order_by method translates into an SQL ORDER BY, therefore it works only with model fields, which correspond to table columns. It won't work if you intend to sort your elements by a model's method.
So, if you want to accomplish something like
Album.objects.all().order_by('get_weekly_count')
You'll have to do it the python way
sorted(Album.objects.all(), key=lambda x: x.get_weekly_count())
Performance-wise, this means you'll get your elements with a query and then you'll sort them with python (that's different from getting a sorted queryset in one shot).
Otherwise, if it's possible for you to turn get_weekly_count into raw SQL, you could use it with a Count() or an extra modifier, that would make order_by usable, i.e.:
Album.objects.all().extra(
select={'weekly_count': "<some SQL>"},
select_params=(<you params>,),
).order_by('weekly_count')
Have a look at https://docs.djangoproject.com/en/1.8/ref/models/querysets/#extra
According to the documentation, you should use:
from django.db.models import Count
like.objects.filter(created_at__gt=START_OF_MONTH, created_at__lt=Datetime.now()).values('albumID').annotate(count=Count('albumID')).order_by('count')
This will get results for you in single db query. For more details visit https://docs.djangoproject.com/en/dev/topics/db/aggregation/.
Which one would be better for performance?
We take a slice of products. which make us impossible to bulk update.
products = Product.objects.filter(featured=True).order_by("-modified_on")[3:]
for product in products:
product.featured = False
product.save()
or (invalid)
for product in products.iterator():
product.update(featured=False)
I have tried QuerySet's in statement too as following.
Product.objects.filter(pk__in=products).update(featured=False)
This line works fine on SQLite. But, it rises following exception on MySQL. So, I couldn't use that.
DatabaseError: (1235, "This version of MySQL doesn't yet support
'LIMIT & IN/ALL/ANY/SOME subquery'")
Edit: Also iterator() method causes re-evaluate the query. So, it is bad for performance.
As #Chris Pratt pointed out in comments, the second example is invalid because the objects don't have update methods. Your first example will require queries equal to results+1 since it has to update each object. That might really be costly if you have 1000 products. Ideally you do want to reduce this to a more fixed expense if possible.
This is a similar situation to another question:
Django: Cannot update a query once a slice has been taken
That being said, you would have to do it in at least 2 queries, but you have to be a bit sneaky on how to construct the LIMIT...
Using Q objects for complex queries:
# get the IDs we want to exclude
products = Product.objects.filter(featured=True).order_by("-modified_on")[:3]
# flatten them into just a list of ids
ids = products.values_list('id', flat=True)
# Now use the Q object to construct a complex query
from django.db.models import Q
# This builds a list of "AND id NOT EQUAL TO i"
limits = [~Q(id=i) for i in ids]
Product.objects.filter(featured=True, *limits).update(featured=False)
In some cases it's acceptable to cache QuerySet in array
products = list(products)
Product.objects.filter(pk__in=products).update(featured=False)
Small optimization with values_list
products_id = list(products.values_list('id', flat=True)
Product.objects.filter(pk__in=products_id).update(featured=False)
I don't want any ordering to be applied to a query. So, I have a QuerySet follow as:
question_obj = Question.objects.filter(pk__in=[100,50,27,35,10,42,68]).order_by()
However, when I retrieve the results, they are always ordered by questionID. I iterate the question_obj and this is the result:
for obj in question_obj:
obj.questionID
The result is displayed such as:
10L
27L
35L
42L
50L
68L
100L
If you want to display the objects in the same order as the list of primary keys, then you could use in_bulk to create a dictionary keyed by pk. You can then use a list comprehension to generate the list of questions.
>>> pks = [100,50,27,35,10,42,68]
>>> questions_dict = Question.objects.in_bulk(pks)
>>> questions = [questions_dict[pk] for pk in pks]
>>> for question in questions:
print question.pk
100
50
27
35
...
If you want an unordered collection, use Python's Set object, documented here: http://docs.python.org/tutorial/datastructures.html#sets
If you want the ordering to be the same as the list you're passing as the value for pk__in, you could try:
ids = [100,50,27,35,10,42,68]
questions = list(Question.objects.filter(pk__in=ids))
question_obj = sorted(questions, key=lambda x: ids.index(x.id))
EDIT: And because it's extremely unclear as to what you mean by 'unordered' in reference to a data structure that is by definition ordered: Random ordering can be accomplished through the following:
.order_by('?')
Luke, use the Source, er, the Docs! Yeah, that's it!
Django QuerySet API - order_by()
You could do some raw SQL (with FIELD()) a lá:
Ordering by the order of values in a SQL IN() clause
which should allow you to retrieve them in the order suggested in the list.
To run custom SQL with the ORM:
https://docs.djangoproject.com/en/dev/topics/db/sql/#executing-custom-sql-directly
When I do this,
>>> b = Blog.objects.all()
>>> b
I get this:
>>>[<Blog: Blog Title>,<Blog: Blog Tile>]
When I query what type b is,
>>> type(b)
I get this:
>>> <class 'django.db.models.query.QuerySet'>
What does this mean? Is it a data type like dict, list, etc?
An example of how I can build data structure like a QuerySet will be appreciated.
I would want to know how Django builds that QuerySet (the gory details).
A django queryset is like its name says, basically a collection of (sql) queries, in your example above print(b.query) will show you the sql query generated from your django filter calls.
Since querysets are lazy, the database query isn't done immediately, but only when needed - when the queryset is evaluated. This happens for example if you call its __str__ method when you print it, if you would call list() on it, or, what happens mostly, you iterate over it (for post in b..). This lazyness should save you from doing unnecessary queries and also allows you to chain querysets and filters for example (you can filter a queryset as often as you want to).
Yes, it's just another type, built like every other type.
A QuerySet represents a collection of objects from your database. It can have zero, one or many filters. Filters narrow down the query results based on the given parameters. In SQL terms, a QuerySet equates to a SELECT statement, and a filter is a limiting clause such as WHERE or LIMIT.
https://docs.djangoproject.com/en/1.8/topics/db/queries/
A QuerySet is a list of objects of a given model, QuerySet allow you to read data from database
How can I retrieve the last record in a certain queryset?
Django Doc:
latest(field_name=None) returns the latest object in the table, by date, using the field_name provided as the date field.
This example returns the latest Entry in the table, according to the
pub_date field:
Entry.objects.latest('pub_date')
EDIT : You now have to use Entry.objects.latest('pub_date')
You could simply do something like this, using reverse():
queryset.reverse()[0]
Also, beware this warning from the Django documentation:
... note that reverse() should
generally only be called on a QuerySet
which has a defined ordering (e.g.,
when querying against a model which
defines a default ordering, or when
using order_by()). If no such ordering
is defined for a given QuerySet,
calling reverse() on it has no real
effect (the ordering was undefined
prior to calling reverse(), and will
remain undefined afterward).
The simplest way to do it is:
books.objects.all().last()
You also use this to get the first entry like so:
books.objects.all().first()
To get First object:
ModelName.objects.first()
To get last objects:
ModelName.objects.last()
You can use filter
ModelName.objects.filter(name='simple').first()
This works for me.
Django >= 1.6
Added QuerySet methods first() and last() which are convenience methods returning the first or last object matching the filters. Returns None if there are no objects matching.
When the queryset is already exhausted, you may do this to avoid another db hint -
last = queryset[len(queryset) - 1] if queryset else None
Don't use try...except....
Django doesn't throw IndexError in this case.
It throws AssertionError or ProgrammingError(when you run python with -O option)
You can use Model.objects.last() or Model.objects.first().
If no ordering is defined then the queryset is ordered based on the primary key. If you want ordering behaviour queryset then you can refer to the last two points.
If you are thinking to do this, Model.objects.all().last() to retrieve last and Model.objects.all().first() to retrieve first element in a queryset or using filters without a second thought. Then see some caveats below.
The important part to note here is that if you haven't included any ordering in your model the data can be in any order and you will have a random last or first element which was not expected.
Eg. Let's say you have a model named Model1 which has 2 columns id and item_count with 10 rows having id 1 to 10.[There's no ordering defined]
If you fetch Model.objects.all().last() like this, You can get any element from the list of 10 elements. Yes, It is random as there is no default ordering.
So what can be done?
You can define ordering based on any field or fields on your model. It has performance issues as well, Please check that also. Ref: Here
OR you can use order_by while fetching.
Like this: Model.objects.order_by('item_count').last()
If using django 1.6 and up, its much easier now as the new api been introduced -
Model.object.earliest()
It will give latest() with reverse direction.
p.s. - I know its old question, I posting as if going forward someone land on this question, they get to know this new feature and not end up using old method.
In a Django template I had to do something like this to get it to work with a reverse queryset:
thread.forumpost_set.all.last
Hope this helps someone looking around on this topic.
MyModel.objects.order_by('-id')[:1]
If you use ids with your models, this is the way to go to get the latest one from a qs.
obj = Foo.objects.latest('id')
You can try this:
MyModel.objects.order_by('-id')[:1]
The simplest way, without having to worry about the current ordering, is to convert the QuerySet to a list so that you can use Python's normal negative indexing. Like so:
list(User.objects.all())[-1]