I have a HyperlinkedModelSerializer set up with a few references to other models using the HyperlinkedRelatedField. This works just fine:
class LicenseSerializer(serializers.HyperlinkedModelSerializer):
style = serializers.HyperlinkedRelatedField(
view_name='styles-detail',
queryset=Style.objects.all()
)
order = serializers.HyperlinkedRelatedField(
view_name='orders-detail',
queryset=Order.objects.all()
)
But I need to do some data transformation on some of the other fields, so I override to_internal_value:
def to_internal_value(self, data):
return data
At which point the HyperlinkedRelatedField no long works. I get an error saying:
Cannot assign "'[hyperlinked identity url]'": "[Model.attribute]" must be a "[hyperlinked model]" instance.
Which, at least to me, suggests the HyperlinkedRelatedField no longer works?
Here's a dump of the Serializer when it crashes:
LicenseAppSerializer(context={'request': <rest_framework.request.Request object at 0x03E9D870>}, data={'price': 0, 'style': u'http://localhost:8000/api/1/styles/69/', 'years': 30, 'order': 'http://localhost:8000/api/1/orders/44/', 'qty': 80}):
url = HyperlinkedIdentityField(view_name='licenseapp-detail')
start = DateField(allow_null=True, required=False)
end = DateField(allow_null=True, required=False)
price = DecimalField(decimal_places=2, max_digits=10)
years = IntegerField(allow_null=True, required=False)
qty = IntegerField()
order = HyperlinkedRelatedField(queryset=Order.objects.all(), view_name='order-detail')
style = HyperlinkedRelatedField(queryset=Style.objects.all(), view_name='style-detail')
What am I doing wrong here?
Thanks for your time!
You cannot do this:
def to_internal_value(self, data):
return data
Serializer.to_internal_value validates the received data (data) and converts it into types that are useful on the ORM level.
For HyperLinkedRelatedFields it retrieves (via HyperLinkedRelatedField.to_internal_value) the object that is being linked to. In your case, you directly passed unvalidated data which contained the URLs instead of objects. This might have worked if data contained only primitive types, but would still be insecure.
If you are about to do any transformations, retrieve the objects first:
def to_internal_value(self, data):
validated_data = super().to_internal_value(data)
# Your transformations on validated_data
return validated_data
Related
I am having struggles with the alteration on the parameter name during serialization with DRF.
My input would be a JSON with some parameters:
{
"limit": 10,
"type": "group",
[...]
}
and my serializer looks like:
class RankSerializer(serializers.Serializer):
limit = serializers.IntegerField(default=100, min_value=1)
type = serializers.CharField()
def validate_type(self, t):
# validation
But this doesn't sound right. Type is a reserved keyword in Python so I don't want to use it as a parameter name. I'd like to somehow map it to i.e result_type or something like this.
I already tried using the source= parameters as follows:
result_type = serializers.CharField(source='type')
but this doesn't seem to work on non-model inputs.
I cannot rename the parameter on the frontend level.
I'd appreciate any tips regarding this issue. Cheers.
I searched https://www.django-rest-framework.org/api-guide/fields/ of DRF docs but was unable to find a viable solution. So I subclassed the Serializer class provided by DRF and created a CustomSerializer. In that, I subclassed the run_validation method. In that, before calling super().run_validation(), I access the initial_data passed and change it to the desired mapping as you mentioned. This information of field mappings I store in the Meta class nested inside RankSerializer. So for example, you have fields F1, F2, F3(in JSON data, for example) whose values you want to populate in fields named M1, M2, M3, you just have to write in the Meta class in the field_mappings dictionary the following :
field_mappings = {
'M1': 'F1',
'M2':'F2',
'M3': 'F3'
}
The other fields will function normally.
Here is the code
import rest_framework.serializers as serializers
from .models import Rank
from rest_framework.fields import empty
class CustomSerializer(serializers.Serializer):
def run_validation(self, data=empty):
for (field, mapping) in self.Meta.field_mappings.items():
data[field] = data[mapping]
del data[mapping]
return super().run_validation(data=data)
class RankSerializer(CustomSerializer):
limit = serializers.IntegerField(default=100, min_value=1)
result_type = serializers.CharField(required=True)
class Meta:
field_mappings = {
'result_type': 'type'
}
def create(self, validated_data):
print(validated_data)
def update(self, instance, validated_data):
print(validated_data)
'''
Please run the code below in the InteractiveShell to verify the result
from myapp.serializers import RankSerializer
serializer = RankSerializer(data={"limit":15, "type":"Hello"})
serializer.is_valid()
'''
After running the code in the multi line comments to verify the result here is the snapshot
The workaround I found was to modify the parameter name in the validate method as follows:
def validate(self, data):
data = super(RankSerializer, self).validate(data)
data['result_type'] = self.validate_result_type(self.initial_data.get('type'))
return data
Not sure if this is the cleanest way to do it, but I prefer it over #punter147 answer.
I have a data set like this.
{
'album_name': 'Dear John',
'artist': 'Loney Dear',
'tracks': [
'Airport Surroundings',
'Everything Turns to You',
'I Was Only Going Out',
]
}
when I serialize it, my json file does not look like the same every time. Because order of 'tracks' change every time.
I was looking at 'to_representation' but since this data doesn't have a Key i failed to implement it as i expected.
Can any one give a hint, to make sure that 'tracks' always in same order.
Edit:
This where i am so far,
class QaDetailSerializer(ModelSerializer):
"""Serializer to map the Model instance into JSON format."""
album_name = CharField(source='album_name')
artist = StringRelatedField()
tracks = TracksSerializer()
class Meta:
"""Meta class to map serializer's fields with the model fields."""
order_by = (('id',))
model = Qa
fields = (
'id',
'album_name',
'artist',
'tracks',
)
class TracksSerializer(ModelSerializer):
def to_representation(self, value):
representation = super().to_representation(value)
attributes_dict = representation['tracks']
attribute_keys_sorted = sorted(attributes_dict.keys())
sorted_attribute_dict = collections.OrderedDict()
for key in attribute_keys_sorted:
sorted_attribute_dict[key] = attributes_dict[key]
representation['paraphrases'] = sorted_attribute_dict
return representation
The problem you are experiencing is database problem not serializer problem.
You never told your database how to sort your tracks so each time database return tracks in different order.
In your Track model (not serializer) add this Meta class
class Track(models.Model):
...fields
class Meta:
ordering = ("pk",)
this will cause your tracks to be always ordered by primary key.
You can order by any other field and you can also order by multiple fields
i.e ordering = ("music_type", "name")
I want to BookResource to perform join (book table with author table) with dedydrate(...) function. Final result should be sorted by table Author.
dehydrate(...) is called for each item in Book table.
class Author(Model)
author_name = models.CharField(max_length=64)
class Book(Model)
author = models.ForeignKey('Author')
book_name = models.CharField(max_length=64)
class BookResource(ModelResource):
class Meta(object):
# The point here is Book table can be sorted. But, final result
# should be sorted by author_name
queryset = Book.objects.all().order_by('book_name')
resource_name = 'api_test'
serializer = Serializer(formats=['xml', 'json'])
allowed_methods = ('get')
always_return_data = True
def dehydrate(self, bundle):
author_id = bundle.obj.author.id # author is foreign key of book
author_obj = Author.objects.get(id=bundle.obj.author.id)
# Construct queryset with author_name. Same as join 2 tables.
# But, I want to sort by author.
bundle.data['author_name'] = author_obj.author_name
return bundle
# This is called before dehydrate(...) is called. Not sure how to use it.
def apply_sorting(self, obj_list, options=None):
return obj_list
Questions:
1) How to sort result by author if using above code?
2) Could not figure out how to do join. Can you provide alternative?
Thank you.
Very late answer, but I ran into this lately. If you look at the Tastypie resource on the dehydrate method
there is a series of methods called. Shortly, this is the order of calls:
build_bundle
obj_get_list
apply_sorting
paginators
full_dehydrate (field by field)
alter_list_data_to_serialize
return self.create_response
You want to focus on the second last method self.alter_list_data_to_serialize(request, to_be_serialized)
the base method returns the second parameter
def alter_list_data_to_serialize(self, request, data):
"""
A hook to alter list data just before it gets serialized & sent to the user.
Useful for restructuring/renaming aspects of the what's going to be
sent.
Should accommodate for a list of objects, generally also including
meta data.
"""
return data
Whatever you want to do you can do it in here. Consider that to be serialized format will be a dict with 2 fields: meta and objects. the object field has a list of bundle objects.
An additional sorting layer could be:
def alter_list_data_to_serialize(self, request, data):
try:
sord = True if request.GET.get("sord") == "asc" else False
data["objects"].sort(
key=lambda x: x.data[request.GET.get("sidx", default_value)],
reverse=sord)
except:
pass
return data
You can optimize it as is a bit dirt. But that's the main idea.
Again, it's a workaround and all the sorting should belong to apply_sorting
I have a Forum Topic model that I want to order on a computed SerializerMethodField, such as vote_count. Here are a very simplified Model, Serializer and ViewSet to show the issue:
# models.py
class Topic(models.Model):
"""
An individual discussion post in the forum
"""
title = models.CharField(max_length=60)
def vote_count(self):
"""
count the votes for the object
"""
return TopicVote.objects.filter(topic=self).count()
# serializers.py
class TopicSerializer(serializers.ModelSerializer):
vote_count = serializers.SerializerMethodField()
def get_vote_count(self, obj):
return obj.vote_count()
class Meta:
model = Topic
# views.py
class TopicViewSet(TopicMixin, viewsets.ModelViewSet):
queryset = Topic.objects.all()
serializer_class = TopicSerializer
Here is what works:
OrderingFilter is on by default and I can successfully order /topics?ordering=title
The vote_count function works perfectly
I'm trying to order by the MethodField on the TopicSerializer, vote_count like /topics?ordering=-vote_count but it seems that is not supported. Is there any way I can order by that field?
My simplified JSON response looks like this:
{
"id": 1,
"title": "first post",
"voteCount": 1
},
{
"id": 2,
"title": "second post",
"voteCount": 8
},
{
"id": 3,
"title": "third post",
"voteCount": 4
}
I'm using Ember to consume my API and the parser is turning it to camelCase. I've tried ordering=voteCount as well, but that doesn't work (and it shouldn't)
This is not possible using the default OrderingFilter, because the ordering is implemented on the database side. This is for efficiency reasons, as manually sorting the results can be incredibly slow and means breaking from a standard QuerySet. By keeping everything as a QuerySet, you benefit from the built-in filtering provided by Django REST framework (which generally expects a QuerySet) and the built-in pagination (which can be slow without one).
Now, you have two options in these cases: figure out how to retrieve your value on the database side, or try to minimize the performance hit you are going to have to take. Since the latter option is very implementation-specific, I'm going to skip it for now.
In this case, you can use the Count function provided by Django to do the count on the database side. This is provided as part of the aggregation API and works like the SQL COUNT function. You can do the equivalent Count call by modifying your queryset on the view to be
queryset = Topic.objects.annotate(vote_count=Count('topicvote_set'))
Replacing topicvote_set with your related_name for the field (you have one set, right?). This will allow you to order the results based on the number of votes, and even do filtering (if you want to) because it is available within the query itself.
This would require making a slight change to your serializer, so it pulls from the new vote_count property available on objects.
class TopicSerializer(serializers.ModelSerializer):
vote_count = serializers.IntegerField(read_only=True)
class Meta:
model = Topic
This will override your existing vote_count method, so you may want to rename the variable used when annotating (if you can't replace the old method).
Also, you can pass a method name as the source of a Django REST framework field and it will automatically call it. So technically your current serializer could just be
class TopicSerializer(serializers.ModelSerializer):
vote_count = serializers.IntegerField(read_only=True)
class Meta:
model = Topic
And it would work exactly like it currently does. Note that read_only is required in this case because a method is not the same as a property, so the value cannot be set.
Thanks #Kevin Brown for your great explanation and answer!
In my case I needed to sort a serializerMethodField called total_donation which is the sum of donations from the UserPayments table.
UserPayments has:
User as a foreignKey
sum which is an IntegerField
related_name='payments'
I needed to get the total donations per User but only donations that have a status of 'donated', not 'pending'. Also needed to filter out the payment_type coupon, which is related through two other foreign keys.
I was dumbfounded how to join and filter those donations and then be able to sort it via ordering_fields.
Thanks to your post I figured it out!
I realized it needed to be part of the original queryset in order to sort with ordering.
All I needed to do was annotate the queryset in my view, using Sum() with filters inside like so:
class DashboardUserListView(generics.ListAPIView):
donation_filter = Q(payments__status='donated') & ~Q(payments__payment_type__payment_type='coupon')
queryset = User.objects.annotate(total_donated=Sum('payments__sum', filter=donation_filter ))
serializer_class = DashboardUserListSerializer
pagination_class = DashboardUsersPagination
filter_backends = [filters.OrderingFilter]
ordering_fields = ['created', 'last_login', 'total_donated' ]
ordering = ['-created',]
I will put it here because the described case is not the only one.
The idea is to rewrite the list method of your Viewset to order by any of your SerializerMethodField(s) also without moving your logic from the Serializer to the ModelManager (especially when you work with several complex methods and/or related models)
def list(self, request, *args, **kwargs):
response = super().list(request, args, kwargs)
ordering = request.query_params.get('ordering')
if "-" in ordering:
response.data['results'] = sorted(response.data['results'], key=lambda k: (k[ordering.replace('-','')], ), reverse=True)
else:
response.data['results'] = sorted(response.data['results'], key=lambda k: (k[ordering], ))
return response
So I'm trying to return a JSON object for a project. I've spent a few hours trying to get Django just returning the JSON.
Heres the view that we've been working with:
def json(request, first_name):
user = User.objects.all()
#user = User.objects.all().values()
result = simplejson.dumps(user, default=json_util.default)
return HttpResponse(result)
Here's my model:
class User(Document):
gender = StringField( choices=['male', 'female', 'Unknown'])
age = IntField()
email = EmailField()
display_name = StringField(max_length=50)
first_name = StringField(max_length=50)
last_name = StringField(max_length=50)
location = StringField(max_length=50)
status = StringField(max_length=50)
hideStatus = BooleanField()
photos = ListField(EmbeddedDocumentField('Photo'))
profile =ListField(EmbeddedDocumentField('ProfileItem'))
allProfile = ListField(EmbeddedDocumentField('ProfileItem')) #only return for your own profile
This is what it's returning:
[<User: User object>, <User: User object>] is not JSON serializable
Any thoughts on how I can just return the JSON?
With MongoEngine 0.8 or greater, objects and querysets have a to_json() method.
>>> User.objects.to_json()
simplejson.dumps() doesn't know how to "reach into" your custom objects; the default function, json_util.default must just be calling str() or repr() on your documents. (Is json_util custom code you've written? If so, showing its source here could prove my claim.)
Ultimately, your default function will need to be able to make sense of the MongoEngine documents. I can think of at least two ways that this might be implemented:
Write a custom default function that works for all MongoEngine documents by introspecting their _fields attribute (though note that the leading underscore means that this is part of the private API/implementation detail of MongoEngine and may be subject to change in future versions)
Have each of your documents implement a as_dict method which returns a dictionary representation of the object. This would work similarly to the to_mongo method provided on documents by MongoEngine, but shouldn't return the _types or _cls fields (again, these are implementation details of MongoEngine).
I'd suggest you go with option #2: the code will be cleaner and easier to read, better encapsulated, and won't require using any private APIs.
As dcrosta suggested you can do something like this, hope that will help you.
Document definition
class MyDocument(Document):
# Your document definition
def to_dict(self):
return mongo_to_dict_helper(self)
helper.py:
from mongoengine import StringField, ListField, IntField, FloatField
def mongo_to_dict_helper(obj):
return_data = []
for field_name in obj._fields:
if field_name in ("id",):
continue
data = obj._data[field_name]
if isinstance(obj._fields[field_name], StringField):
return_data.append((field_name, str(data)))
elif isinstance(obj._fields[field_name], FloatField):
return_data.append((field_name, float(data)))
elif isinstance(obj._fields[field_name], IntField):
return_data.append((field_name, int(data)))
elif isinstance(obj._fields[field_name], ListField):
return_data.append((field_name, data))
else:
# You can define your logic for returning elements
return dict(return_data)