view_set.get_queryset().query.order_by will get you the ordering tuple of a ViewSet class in Django REST Framework, but ModelName.objects.get_queryset().query.order_by is always an empty tuple. How come it's not populated? I can see from the database logs that the query is ordered when running ModelName.objects.all(). I've also tried to run the actual query, in case the ordering is populated lazily, but even that does not work:
>>> all = ModelName.objects.all()
>>> for instance in all:
... True
>>> all.query.order_by
()
This bit of the internals of the SQL compiler might help. It looks like the ordering information is stored in several different places depending on where it is defined, to allow for overriding at different levels:
if self.query.extra_order_by:
ordering = self.query.extra_order_by
elif not self.query.default_ordering:
ordering = self.query.order_by
elif self.query.order_by:
ordering = self.query.order_by
elif self.query.get_meta().ordering:
ordering = self.query.get_meta().ordering
self._meta_ordering = ordering
query.order_by is only set when the queryset is explicitly ordered. If you're relying on default ordering defined on the model class, this will be stored in query.get_meta().ordering.
Related
I have a model that has arbitrary key/value pairs (attributes) associated with it. I'd like to have the option of sorting by those dynamic attributes. Here's what I came up with:
class Item(models.Model):
pass
class Attribute(models.Model):
item = models.ForeignKey(Item, related_name='attributes')
key = models.CharField()
value = models.CharField()
def get_sorted_items():
return Item.objects.all().annotate(
first=models.select_attribute('first'),
second=models.select_attribute('second'),
).order_by('first', 'second')
def select_attribute(attribute):
return expressions.RawSQL("""
select app_attribute.value from app_attribute
where app_attribute.item_id = app_item.id
and app_attribute.key = %s""", (attribute,))
This works, but it has a bit of raw SQL in it, so it makes my co-workers wary. Is it possible to do this without raw SQL? Can I make use of Django's ORM to simplify this?
I would expect something like this to work, but it doesn't:
def get_sorted_items():
return Item.objects.all().annotate(
first=Attribute.objects.filter(key='first').values('value'),
second=Attribute.objects.filter(key='second').values('value'),
).order_by('first', 'second')
Approach 1
Using Djagno 1.8+ Conditional Expressions
(see also Query Expressions)
items = Item.objects.all().annotate(
first=models.Case(models.When(attribute__key='first', then=models.F('attribute__value')), default=models.Value('')),
second=models.Case(models.When(attribute__key='second', then=models.F('attribute__value')), default=models.Value(''))
).distinct()
for item in items:
print item.first, item.second
Approach 2
Using prefetch_related with custom models.Prefetch object
keys = ['first', 'second']
items = Item.objects.all().prefetch_related(
models.Prefetch('attributes',
queryset=Attribute.objects.filter(key__in=keys),
to_attr='prefetched_attrs'),
)
This way every item from the queryset will contain a list under the .prefetched_attrs attribute.
This list will contains all filtered-item-related attributes.
Now, because you want to get the attribute.value, you can implement something like this:
class Item(models.Model):
#...
def get_attribute(self, key, default=None):
try:
return next((attr.value for attr in self.prefetched_attrs if attr.key == key), default)
except AttributeError:
raise AttributeError('You didnt prefetch any attributes')
#and the usage will be:
for item in items:
print item.get_attribute('first'), item.get_attribute('second')
Some notes about the differences in using both approaches.
you have a one idea better control over the filtering process using the approach with the custom Prefetch object. The conditional-expressions approach is one idea harder to be optimized IMHO.
with prefetch_related you get the whole attribute object, not just the value you are interested in.
Django executes prefetch_related after the queryset is being evaluated, which means a second query is being executed for each clause in the prefetch_related call. On one way this can be good, because it this keeps the main queryset untouched from the filters and thus not additional clauses like .distinct() are needed.
prefetch_related always put the returned objects into a list, its not very convenient to use when you have prefetchs returning 1 element per object. So additional model methods are required in order to use with pleasure.
Consider the following model:
class Test(Model):
# Some fields...
class TestExecution(Model):
test = ForeignKey(Test)
execution_date = DateTimeField()
# more fields...
class Goal(Model):
tests = ManyToManyField(Test)
# more fields...
I want to get all the latest result of each test performed as part of a certain goal, so I perform the following query:
TestExecution.objects.filter(test__goal_id=goal_id).order_by("execution_date")
but the problem is that I get ALL the executions performed, and I want only the latest for each test.
I saw that the distinct(*fields) method can be used to eliminate duplicate execution of the same test, but it only works in PostgreSQL, so it is not suitable for me.
Is there any other way to filter a QuerySet so that it'll include only rows that are distinct on selected columns?
You can remove duplicates not by a query, but sth like list(set(list_of_objects)) (I recommend first check if it works), for removing list_of_objects duplicates you'll need to define a uniqueness of an object.
In order to do that, you'll need to make the object hashable. You need to define both hash and eq method:
def __eq__(self, other):
return self.execution_date==other.execution_date\
and self.title==other.title
def __hash__(self):
return hash(('title', self.title,
'execution_date', self.execution_date))
also you can do that more easily but not in clean way by getting values_list in query:
list(set(TestExecution.objects.filter(test__goal_id=goal_id)
.values_list("sth", flat = True)
.order_by("execution_date")))
if objects are not hashable remove in the dirty way:
seen_titles = set()
new_list = []
for obj in myList:
if obj.title not in seen_titles:
new_list.append(obj)
seen_titles.add(obj.title)
I have 2 models namely A and B. Both have a common field, say timestamp.
X = A.objects.all()
Y = A.objects.all()
results = chain(X,Y) # Union
Now I want to apply ORDERBY query on results. How can I do that?
While the python way of doing it is by using some kind of sorting, this is normally not the correct way of doing it in Django.
The reason is that in order to join two queries of two different models, you have to transform them to lists (as other answers suggest).
While this seems good, most of the time it is not, because you hit the database, i.e. the database is asked for all elements of both models for creating both lists, and them joining.
The most likely reason why you are joining two models in a single list is because both should be derived from a common model, e.g. an Abstract model. Doing this both models have a natural common field obtained by being subclasses of the abstract model.
The ordering of such a query can then be made using order_by. This is more efficient because queries are lazy.
EDIT:
Example:
class Parent(models.Model):
date_field = models.DateField()
def pretty_show():
# returns general behavior
class B(Parent):
# other specific fields
def pretty_show():
# returns specific behavior
class C(Parent):
# other specific fields
def pretty_show():
# returns specific behavior
X = Parent.objects.all().order_by("date_field")
for x in X:
x.pretty_show()
Notice that Parent class in this case is not Abstract and childs are not proxies, but this choice strongly depends on what you want to achieve. Hope this helps.
You mean:
X = A.objects.all()
Y = B.objects.all()
results = chain(X,Y) # Union
You can cast result to list(). List has order function:
ut.sort(key=lambda x: x.count, reverse=True)
more
You could try to create a list of the model objects and then sort by their time stamp like this (both models must have the same time_stamp attribute):
l= list(x)+list(y)
l.sort(key=lambda l: l.time_stamp)
Hope this is what you are looking for.
Suppose I have a Person model that has a first name field and a last name field. There will be many people who have the same first name. I want to write a TastyPie resource that allows me to get a list of the unique first names (without duplicates).
Using the Django model directly, you can do this easily by saying something like Person.objects.values("first_name").distinct(). How do I achieve the same thing with TastyPie?
Update
I've adapted the apply_filters method linked below to use the values before making the distinct call.
def apply_filters(self, request, applicable_filters):
qs = self.get_object_list(request).filter(**applicable_filters)
values = request.GET.get('values', '').split(',')
if values:
qs = qs.values(*values)
distinct = request.GET.get('distinct', False) == 'True'
if distinct:
qs = qs.distinct()
return qs
values returns dictionaries instead of model objects, so I don't think you need to override alter_list_data_to_serialize.
Original response
There is a nice solution to the distinct part of the problem here involving a light override of apply_filters.
I'm surprised I'm not seeing a slick way to filter which fields are returned, but you could implement that by overriding alter_list_data_to_serialize and deleting unwanted fields off the objects just before serialization.
def alter_list_data_to_serialize(self, request, data):
data = super(PersonResource, self).alter_list_data_to_serialize(request, data)
fields = request.GET.get('fields', None)
if fields is not None:
fields = fields.split(',')
# Data might be a bundle here. If so, operate on data.objects instead.
data = [
dict((k,v) for k,v in d.items() if k in fields)
for d in data
]
return data
Combine those two to use something like /api/v1/person/?distinct=True&values=first_name to get what you're after. That would work generally and would still work with additional filtering (&last_name=Jones).
I'm trying to order by a count of a manyToMany field is there a way to do this with TastyPie?
For example
class Person(models.Model):
friends = models.ManyToMany(User, ..)
I want PersonResource to spit out json that is ordered by the number of friends a person has...
is that possible?
I know this is an old question, but I recently encountered this problem and came up with a solution.
Tastypie doesn't easily allow custom ordering, but it is easy to modify the queryset it uses.
I actually just modified the default queryset for the model using a custom manager.
for instance:
class PersonManager(models.Manager):
def get_query_set(self):
return super(PersonManager self).get_query_set().\
annotate(friend_count=models.Count('friends'))
class Person(models.Model):
objects = PersonManager()
friends = ...
You could also add the annotation in Tastypie, wither in the queryset=... in the Meta class, or overriding the get_object_list(self,request) method.
I wasn't able to get the results ordering as per coaxmetal's solution, so I solved this a different way, by overriding the get_object_list on the Resource object as per http://django-tastypie.readthedocs.org/en/latest/cookbook.html. Basically if the 'top' querystring parameter exists, then the ordered result is returned.
class MyResource(ModelResource):
class Meta:
queryset = MyObject.objects.all()
def get_object_list(self, request):
try:
most_popular = request.GET['top']
result = super(MyResource, self).get_object_list(request).annotate(num_something=Count('something')).order_by('num_something')
except:
result = super(MyResource, self).get_object_list(request)
return result
I have not used TastyPie, but your problem seems to be more general. You can't have custom ordering in a Django ORM query. You're better off storing tuples of the form (Person, friend_count). This is pretty easy:
p_list = []
for person in Person.objects.all():
friendcount = len(person.friends.all())
p_list.append((person, friendcount))
Then, you can use the built in sorted function like so:
sorted_list = [person for (person, fc) in sorted(p_list, key=lambda x: x[1])]
The last line basically extracts the Persons from a sorted list of Persons, sorted on the no of friends one has.
`