I have a Django ModelForm which exposes a multiple choice field corresponding to a many-to-many relation through a model which holds order of selection (a list of documents) as an extra attribute. At the front-end, the field is displayed as two multiple select fields similar to that in admin, one to list available choices and the other holds the selected elements.
The form can be saved with the correct selection of elements but they are always in the order of the original order of choices, not the selection. The browser sends the selection in correct order, but order in form.cleaned_data['documents'] is always the order in original order of choices.
How can I make the MultipleChoiceField respect the order of elements selected?
Thanks.
There is no simple way. You either need to override the clean method of the MultipleChoiceField or, as you mentioned in your comment, use the getlist to re-order them manually. It probably depends how often in your code do you need to do it.
The clean method of MultipleChoiceField creates a QuerySet that you are receiving, by filtering an object list through the IN operator like this, so the order is given by the database:
qs = self.queryset.filter(**{'%s__in' % key: value})
You can inherit from ModelMultipleChoiceField:
class OrderedModelMultipleChoiceField(ModelMultipleChoiceField):
def clean(self, value):
qs = super(OrderedModelMultipleChoiceField, self).clean(value)
return sorted(qs, lambda a,b: sorted(qs, key=lambda x:value.index(x.pk)))
The drawback is that the returned value is no longer a QuerySet but an ordinary list.
To return an ordered QuerySet when overriding the clean method you could also do this:
class OrderedModelMultipleChoiceField(ModelMultipleChoiceField):
def clean(self, value):
qs = super(OrderedModelMultipleChoiceField, self).clean(value)
clauses = ' '.join(['WHEN id=%s THEN %s' % (pk, i) for i, pk in enumerate(value)])
return qs.filter(pk__in=value).extra(
select={'ordering': 'CASE %s END' % clauses},
order_by=('ordering',)
)
I did it via a Widget. The benefit of it is, it will sort properly in different languages:
class SortedSelectMultiple(SelectMultiple):
def render_options(self, selected_choices):
self.choices = sorted(self.choices)
self.choices.sort(key=lambda x: x[1])
return super(SortedSelectMultiple, self).render_options(selected_choices)
I am able to maintain the order of the selection using following way:
class OrderedModelMultipleChoiceField(models.ModelMultipleChoiceField):
def clean(self, value):
qs = super(OrderedModelMultipleChoiceField, self).clean(value)
preserved = Case(*[When(pk=pk, then=pos) for pos, pk in enumerate(value)])
return qs.filter(pk__in=value).order_by(preserved)
Note: I am using Django 2.2
Related
I have a serializer class that represents a user.
class UserSerializer(BaseSerializer):
uid = serializers.IntegerField(required=True)
class Meta:
model = User
fields = "all"
def validate(self, data):
super().validate(data)
validate_user_data(data=self.initial_data, user=self.instance)
return data
users should be unique on uid, so when getting a post request what I really want is to change the uid field to:
uid = serializers.IntegerField(required=True, validators=[validators.UniqueValidator(queryset=User.objects.all())])
and this will probably work, the problem is, this will trigger an sql query that will select all the users.
This could have a very high impact on the system since there could be tens of thousands of them.
What I would really want is to change the query to User.objects.get(uid=uid), which will not select every user from the DB.
However, since I'm in the serializer definition of uid, I can't use uid=uid because uid is just being defined.
(…) and this will probably work, the problem is, this will trigger an sql query the will select all the users.
This is incorrect. Django will filter the queryset, but the filtering itself happens at the database side.
This will not query for all the items in the User table. The queryset is not evaluated. It acts as a "root queryset" against which queries will be constructed.
We can look up the source code on GitHub:
class UniqueValidator(object):
# ...
def __call__(self, value):
queryset = self.queryset
queryset = self.filter_queryset(value, queryset)
queryset = self.exclude_current_instance(queryset)
if qs_exists(queryset):
raise ValidationError(self.message, code='unique')
Here the queryset is thus filtered. This filtering is not done at the Python/Django level, but it constructs a filtered variant. Indeed, if we look at the filter_queryset function, we see:
def filter_queryset(self, value, queryset):
"""
Filter the queryset to all instances matching the given attribute.
"""
filter_kwargs = {'%s__%s' % (self.field_name, self.lookup): value}
return qs_filter(queryset, **filter_kwargs)
with as qs_filter:
def qs_filter(queryset, **kwargs):
try:
return queryset.filter(**kwargs)
except (TypeError, ValueError, DataError):
return queryset.none()
As you can see, it will thus generate a query User.objects.filter(uid=the_uid).exclude(pk=item_that_is_updated)
It will thus check if there is a User object in that database with the same uid as the one you set, and exclude the one your are updating (given that is applicable). The query this will look like:
SELECT user.*
FROM user
WHERE uid = the_uid
AND id <> item_that_is_updated
It will thus filter at the database level, and therefore boost efficiency.
Update Two
So unfortunately, #Reza Torkaman Ahmadi's idea didn't work out in the end. This is because our program has a filtering function that relies on get_queryset, and overriding the get_queryset method in our views was messing up that function.
So, my partner and I discussed it, and here's what he came up with.
class OrderbyFilter(filters.OrderingFilter):
def get_ordering(self, request, queryset, view):
"""
Ordering is set by a comma delimited ?$orderby=... query parameter.
Extends the OrderingFilter of the django rest framework to redefine
ordering parameters to "asc" and "desc".
"""
params = request.query_params.get(self.ordering_param)
if params:
field_queries = [param.strip() for param in params.split(',')]
fields = []
for field_query in field_queries:
field_query = field_query.split()
if len(field_query) <= 2:
while "asc" in field_query:
field_query.remove("asc")
for i, field in enumerate(field_query):
if field == "desc":
field_query[i-1] = "-" + field_query[i-1]
while "desc" in field_query:
field_query.remove("desc")
fields.append(field_query[0])
else:
fields.append([param.strip() for param in params.split(',')])
ordering = self.remove_invalid_fields(queryset, fields, view, request)
if ordering:
return ordering
return self.get_default_ordering(view)
Basically, this function over-rides Django REST's source code, specifically the get_ordering function in the OrderingFilter. What it does is, if 'asc' is in the query after the field, it removes it and treats it like normal (for normal ascension ordering)
Otherwise if 'desc' is there, it removed the 'desc', and applies a hyphen.
Update Answered. Applied #Reza Torkaman Ahmadi idea and it works great after modifying it to fit my needs. Thanks mate!
Currently, in Django API rest framework, if a user wants to see a list of something in ascending or descending order, they have to do the following:
'Datastream&order_by=-name' shows names in descending order
'Datastream&order_by=name' shows names in ascending order.
I want to make a custom query where typing in 'Datastream&order_by=asc name' will order the names by ascending order, and 'desc name' will do so in descending order.
I've looked at some of the source code for the REST framework, but I may be looking in the wrong area. Getting stumped on what I should do. Any ideas?
You can do it in your own way. like this:
class DatastreamViewSet(ModelViewSet):
def get_queryset(self):
queryset = super(DatastreamViewSet, self).get_queryset()
order_by = self.request.query_params.get('order_by', '')
if order_by:
order_by_name = order_by.split(' ')[1]
order_by_sign = order_by.split(' ')[0]
order_by_sign = '' if order_by_sign == 'asc' else '-'
queryset = queryset.order_by(order_by_sign + order_by_name)
return queryset
this will look for query parameter order_by if is supplied then will split it by space, the first one will indicate to use + or - sign on order_by filter, and the second will be the name of it. so put it all together and create a text, pass it to order_by and your good to go.
for example:
?order_by=asc name
will be like this in django =>
return queryset.order_by('name')
Django Rest Framework's standard way:
from rest_framework.filters import OrderingFilter
Then on your APIView or ViewSet
filter_backends = (OrderingFilter,)
ordering_fields = ['field_name']
query parameter is ordering and supports reverse ordering as well.
GET https://example.com/?ordering=-field_name
I want to BookResource to perform join (book table with author table) with dedydrate(...) function. Final result should be sorted by table Author.
dehydrate(...) is called for each item in Book table.
class Author(Model)
author_name = models.CharField(max_length=64)
class Book(Model)
author = models.ForeignKey('Author')
book_name = models.CharField(max_length=64)
class BookResource(ModelResource):
class Meta(object):
# The point here is Book table can be sorted. But, final result
# should be sorted by author_name
queryset = Book.objects.all().order_by('book_name')
resource_name = 'api_test'
serializer = Serializer(formats=['xml', 'json'])
allowed_methods = ('get')
always_return_data = True
def dehydrate(self, bundle):
author_id = bundle.obj.author.id # author is foreign key of book
author_obj = Author.objects.get(id=bundle.obj.author.id)
# Construct queryset with author_name. Same as join 2 tables.
# But, I want to sort by author.
bundle.data['author_name'] = author_obj.author_name
return bundle
# This is called before dehydrate(...) is called. Not sure how to use it.
def apply_sorting(self, obj_list, options=None):
return obj_list
Questions:
1) How to sort result by author if using above code?
2) Could not figure out how to do join. Can you provide alternative?
Thank you.
Very late answer, but I ran into this lately. If you look at the Tastypie resource on the dehydrate method
there is a series of methods called. Shortly, this is the order of calls:
build_bundle
obj_get_list
apply_sorting
paginators
full_dehydrate (field by field)
alter_list_data_to_serialize
return self.create_response
You want to focus on the second last method self.alter_list_data_to_serialize(request, to_be_serialized)
the base method returns the second parameter
def alter_list_data_to_serialize(self, request, data):
"""
A hook to alter list data just before it gets serialized & sent to the user.
Useful for restructuring/renaming aspects of the what's going to be
sent.
Should accommodate for a list of objects, generally also including
meta data.
"""
return data
Whatever you want to do you can do it in here. Consider that to be serialized format will be a dict with 2 fields: meta and objects. the object field has a list of bundle objects.
An additional sorting layer could be:
def alter_list_data_to_serialize(self, request, data):
try:
sord = True if request.GET.get("sord") == "asc" else False
data["objects"].sort(
key=lambda x: x.data[request.GET.get("sidx", default_value)],
reverse=sord)
except:
pass
return data
You can optimize it as is a bit dirt. But that's the main idea.
Again, it's a workaround and all the sorting should belong to apply_sorting
I have a boolean field on my model that represents whether someone has canceled their membership or not. I am trying to create a custom SimpleListFilter that allows this field to be filtered on.
However, I really want to show only those who are not canceled by default. Is there someway to select the "No" option by default? This is my filter so far:
class CanceledFilter(SimpleListFilter):
title = 'Canceled'
# Parameter for the filter that will be used in the URL query.
parameter_name = 'canceled'
def lookups(self, request, model_admin):
return (
(True, 'Yes'),
(False, 'No'),
)
def queryset(self, request, queryset):
if self.value() is True or self.value() is None:
return queryset.filter(canceled=True)
if self.value() is False:
return queryset.filter(canceled=False)
EDIT:
I should have been a bit clearer. I am specifically trying to do this in the Admin interface. When I add the above filter as a list_filter in admin. I get a filter on the side of the admin page with 3 choices: All, Yes and No.
I would like the "No" choice or none of the choices to be set by default. Instead the "All" choice is always set by default. Is there some none hacky way to set the default filter choice or something like that.
Basiclly in Admin when they view the Members, I only want to show the active (not canceled) by default. If they click "All" or "Yes" then I want to show the canceled ones.
Update:
Note this is the same as question Default filter in Django admin, but I that question is now 6 years old. The accepted answer is marked as requiring Django 1.4. I am not sure if that answer will still work with newer Django versions or is still the best answer.
Given the age of the answers on the other question, I am not sure how we should proceed. I don't think there is any way to merge the two.
Had to do the same and stumbled upon your question. This is how I fixed it in my code (adapted to your example):
class CanceledFilter(SimpleListFilter):
title = 'Canceled'
# Parameter for the filter that will be used in the URL query.
parameter_name = 'canceled'
def lookups(self, request, model_admin):
return (
(2, 'All'),
(1, 'Yes'),
(0, 'No'),
)
def queryset(self, request, queryset):
if self.value() is None:
self.used_parameters[self.parameter_name] = 0
else:
self.used_parameters[self.parameter_name] = int(self.value())
if self.value() == 2:
return queryset
return queryset.filter(cancelled=self.value())
Some explanation is required. The querystring is just part of the URL, and exactly what the name implies: a query string. Your values come in as strings, not as booleans or integers. So when you call self.value(), it returns a string.
If you examine the URL you get when you click on the Yes/No, when not using a custom list filter, you'll see it encodes it as 1/0, not True/False. I went with the same scheme.
For completeness and our future readers, I also added 2 for All. Without verifying, I assume that was None before. But None is also used when nothing is selected, which defaults to All. Except, in our case it needs to default to False, so I had to pick a different value. If you don't need the All option, just remove the final if-block in the queryset method, and the first tuple in the lookups method.
With that out of the way, how does it work? The trick is in realising that self.value() just returns:
self.used_parameters.get(self.parameter_name, None)
which is either a string, or None, depending on whether the key is found in the dictionary or not. So that's the central idea: we make sure it contains integers and not strings, so that self.value() can be used in the call to queryset.filter(). Special treatment for the value for All, which is 2: in this case, just return queryset rather than a filtered queryset. Another special value is None, which means there is no key parameter_name in the dictionary. In that case, we create one with value 0, so that False becomes the default value.
Note: your logic was incorrect there; you want the non-cancelled by default, but you treat None the same as True. My version corrects this.
ps: yes, you could check for 'True' and 'False' rather than True and False in your querystring method, but then you'd notice the correct selection would not be highlighted because the first elements in your tuple don't match up (you're comparing strings to booleans then). I tried making the first elements in the tuples strings too, but then I'd have to do string comparison or eval to match up 'True' to True, which is kind of ugly/unsafe. So best stick to integers, like in my example.
If anyone is still interested in a solution for this, I used a different and IMHO much cleaner approach. As I'm fine with a default choice and the handling of it, I decided I just want to rename the default display label. This is IMHO much cleaner and you don't need any "hacks" to handle the default value.
class CompleteFilter(admin.SimpleListFilter):
'''
Model admin filter to filter orders for their completion state.
'''
title = _('completion')
parameter_name = 'complete'
def choices(self, changelist):
'''
Return the available choices, while setting a new default.
:return: Available choices
:rtype: list
'''
choices = list(super().choices(changelist))
choices[0]['display'] = _('Only complete')
return choices
def lookups(self, request, model_admin):
'''
Return the optionally available lookup items.
:param django.http.HttpRequest request: The Django request instance
:param django.contrib.admin.ModelAdmin model_admin: The model admin instance
:return: Optional lookup states
:rtype: tuple
'''
return (
('incomplete', _('Only incomplete')),
('all', _('All')),
)
def queryset(self, request, queryset):
'''
Filter the retreived queryset.
:param django.http.HttpRequest request: The Django request instance
:param django.db.models.query.QuerySet: The Django database query set
:return: The filtered queryset
:rtype: django.db.models.query.QuerySet
'''
value = self.value()
if value is None:
return queryset.filter(state__complete=True)
elif value == 'incomplete':
return queryset.filter(state__complete=False)
return queryset
In the choices() method, I just rename the display label from All to Only complete. Thus, the new default (which has a value of None is now renamed).
Then I've added all additional lookups as usual in the lookups() method. Because I still want an All choice, I add it again. However, you can also skip that part if you don't need it.
That's basically it! However, if you want to display the All choice on top again, you might want to reorder the choices list in the choices() method before returning it. For example:
# Reorder choices so that our custom "All" choice is on top again.
return [choices[2], choices[0], choices[1]]
Look at the section called "Adding Extra Manager Methods" in the link below:
http://www.djangobook.com/en/2.0/chapter10.html
You can add an additional models.Manager to your model to only return people that have not cancelled their membership. A rough implementation of the additional models.Manager would look like this:
class MemberManager(models.Manager):
def get_query_set(self):
return super(MemberManager, self).get_query_set().filter(membership=True)
class Customer(models.Model):
# fields in your model
membership = BooleanField() # here you can set to default=True or default=False for when they sign up inside the brackets
objects = models.Manager # default Manager
members = MemberManager() # Manager to filter for members only
Anytime you need to get a list of you current members only, you would then just call:
Customer.members.all()
Suppose I have a Person model that has a first name field and a last name field. There will be many people who have the same first name. I want to write a TastyPie resource that allows me to get a list of the unique first names (without duplicates).
Using the Django model directly, you can do this easily by saying something like Person.objects.values("first_name").distinct(). How do I achieve the same thing with TastyPie?
Update
I've adapted the apply_filters method linked below to use the values before making the distinct call.
def apply_filters(self, request, applicable_filters):
qs = self.get_object_list(request).filter(**applicable_filters)
values = request.GET.get('values', '').split(',')
if values:
qs = qs.values(*values)
distinct = request.GET.get('distinct', False) == 'True'
if distinct:
qs = qs.distinct()
return qs
values returns dictionaries instead of model objects, so I don't think you need to override alter_list_data_to_serialize.
Original response
There is a nice solution to the distinct part of the problem here involving a light override of apply_filters.
I'm surprised I'm not seeing a slick way to filter which fields are returned, but you could implement that by overriding alter_list_data_to_serialize and deleting unwanted fields off the objects just before serialization.
def alter_list_data_to_serialize(self, request, data):
data = super(PersonResource, self).alter_list_data_to_serialize(request, data)
fields = request.GET.get('fields', None)
if fields is not None:
fields = fields.split(',')
# Data might be a bundle here. If so, operate on data.objects instead.
data = [
dict((k,v) for k,v in d.items() if k in fields)
for d in data
]
return data
Combine those two to use something like /api/v1/person/?distinct=True&values=first_name to get what you're after. That would work generally and would still work with additional filtering (&last_name=Jones).