I have a setup where I need to write an API for an existing javascript datamodels which i do not want to touch (for now). The javascript data has a different architecture than I want to have on the server. So my goal is to transform the data that I get from javascript to fit my database model when data is sent to the API. When data is requested from the API, it should match the expected data model of javascript.
I wonder if I can do that with ModelSerializers, if yes, where is the right place to transform the data? In the view? In the serializer.
My setup is like so:
//javascript structure
{
scores: [
{
id: 12,
points: 2
maxpoints: 12
siteuxid: 'EXAMPLE'
},
{ ... }
]
}
//More models in django
{
scores: [
{
id: 12,
points: 2,
question: {
id: 12,
maxpoints: 12,
siteuxid: 'EXAMPLE'
}
},
]
}
Are there any examples anyone can point me to, that achive the same? Basically it is all about having different data structures in server and client and making them compatible. Googleing did not help.
EDIT:
My first problem is that I do not get all posted data in my Serializer. When I post
{
"scores": [{"id":"QFELD_1.1.3.QF2","siteuxid":"VBKM01_VariablenTerme","section":1,"maxpoints":4,"intest":false,"uxid":"ER2","points":0,"value":0,"rawinput":"363"}]
}
to
class UserDataSerializer(serializers.ModelSerializer):
scores = ScoreSerializer(many=True, required=False)
def create(self, validated_data):
print('userDataSerializer validated_data', validated_data)
...
class ScoreSerializer(serializers.ModelSerializer):
id = serializers.CharField(required=False, allow_blank=True, max_length=100)
question = QuestionSerializer(required=False)
class Meta:
model = Score
fields = ('id', 'question', 'points', 'value', 'rawinput', 'state')
I only get the output
userDataSerializer validated_data {'scores': [OrderedDict([('id', 'QFELD_1.1.3.QF2'), ('points', 0), ('value', 0), ('rawinput', '363')])]}
without the score.maxpoints and so on (as it is not in the serializer, but how can I add it? To validated data in order to create a proper question object from the posted data in UserDataSerializer)
The answer is yes, you will use the view to modify your input data, as you must do these tweaks in your data before the view sends the data to the serializers. This is due to the same reason that you only see the attributes of your serializer in your validated data - the serializer ignores all the attributes it does not recognize.
So, first of all, change request.data in your view's post method to make it structured as you need.
def post(self, request, *args, **kwargs):
request.data['question'] = {
'maxpoints': request.data.pop('maxpoints'),
'siteuxid': request.data.pop('siteuxid'),
}
This should be all you need to get started.
However, note that it's strange that question has id: 12 in your example. If you are trying to create a question object along with your score object, it should have no id. If the question is an existing object, though, you should not be sending a dict, but only the id instead.
For example, you should send question: 1 in your input. DRF's ModelSerializer is smart enough to know that the score you are trying to save is to be related with the question which has id = 1. While you're at it, inspect the serializer's validated_data and you'll see the instance of question with id = 1. Magic!
Related
I'm not very familiar with DRF and I haven't found a solution on google for this problem (most answers are about a model with a field as enum, my problem is different)
You see, we have an Enum in a Django application. Let's call it SomeValuesEnum.
class SomeValuesEnum(Enum):
ONE_VALUE = "One value"
ANOTHER_VALUE = "Another value"
What I need to do is to create a GET endpoint that returns the following
{
"count": 2,
"page_count": 1,
"next": null,
"previous": null,
"results": [
{
"value": "One value",
"name": "ONE_VALUE"
}, {
"value": "Another value",
"name": "ANOTHER_VALUE"
}
]
}
I know I need to create a serializer, but I haven't been able to create one and "feed it".
For example, I started with something like this:
class SomeValueSerializer(serializers.Serializer):
Meta:
model = SomeValuesEnum,
fields = '__all__'
and on the view:
class SomeValueListView(APIView):
serializer_class = SomeValueSerializer
def get(self, request):
choices = [{"value": target.value, "name": target.value.capitalize()} for target in SomeValuesEnum]
serializer = SomeValueSerializer(data=choices)
return Response(status=status.HTTP_200_OK, data=serializer.data)
I also tried this
class IncidentSerializer(serializers.Serializer):
name = serializers.CharField(required=False, allow_blank=True, max_length=100)
value = serializers.CharField(required=False, allow_blank=True, max_length=100)
I'm not sure if I'm failing on the creation of the serializer, or in how I invoke him on the view (or maybe both)
Any guidance on the right direction will be greatly appreciate it.
An enum is not a django model. You can have a DRF serializer for something that isn't a model, but you shouldn't give it a model field of something that isn't a model.
See here: Declaring Serializers
Not here: Model Serializers
Furthermore, you are creating a class-based view here:
SomeValueListView(APIView)
You don't necessarily need this, you could use a function based view, which may be easier for you to understand given you are new to DRF. I only say function based views are easier to understand since there isn't so much built in functionality. This can make it easier to debug for someone new to DRF. You can still use the serializer by calling it directly in the view.
See here: Function Based Views
And here: Serializing Objects
Finally...given this code:
choices = [{"value": target.value, "name": target.value.capitalize()} for target in SomeValuesEnum]
I am making the assumption that there could be multiple distinct objects going into this serializer, given that you are using a list comprehension. Either you need to call the serializer separately for each object in the array, or you call the serializer with (many=True). Pretty sure this is your main issue. Like this:
serializer = SomeValueSerializer(data=choices, many=True)
See here: Dealing with multiple objects
Also, in my experience it is better to parse incoming data to the serializer within the serializer itself, not in the view. To me it is a separation of concerns issue, but other people may feel differently. This would look something like this:
class SomeValueListView(APIView):
serializer_class = SomeValueSerializer
def get(self, request):
serializer = SomeValueSerializer(data=SomeValuesEnum)
return Response(status=status.HTTP_200_OK, data=serializer.data)
class SomeValueSerializer(serializer.Serializer):
def to_internal_value(self, data)
name = data.value.capitalize()
value = data.value
return {
'name': name,
'value': value,
}
I'm learning and new to Django Rest Framework and I'm having an issue in serializer validations and ListSerializer update method.
I have an APIView class which handles the put request. I just wanted to have a custom validation and so I've overridden validate method in the serializer class.
From postman I'm sending a JSON data to this APIView Class.
Sample JASON data:
[
{
"id": 1,
"ip": "10.1.1.1",
"host_name": "hostname 1"
},
{
"id": 2,
"ip": "10.1.1.2",
"host_name": "hostname 2"
}
]
When I receive the data and do serializer.is_valid() it passes the flow to the overridden validate function. But in there when I check for the attrs argument, I get all the fields except the id. The key and value for id are not present. It shows None.
The same issue occurred to me when I was trying to override the update method in ListSerializer.
when I tried the below code in the ListSerializer's update method,
data_mapping = {item['id']: item for item in validated_data}
I got an error saying KeyError 'id'.
It seems it's not accepting id field and I'm not sure why! Please, someone explain this to me if I'm wrong anywhere.
Serializer class
from rest_framework import serializers
from .models import NoAccessDetails
class NoAccessDetailsListSerializer(serializers.ListSerializer):
def update(self, instance, validated_data)
data_mapping = {data.id: data for data in instance}
#Here I'm getting KeyError ID
validated_data_mapping = {item['id']: item for item in validated_data}
return
class NoAccessDetailsSerializer(serializers.ModelSerializer):
class Meta:
model = NoAccessDetails
list_serializer_class = NoAccessDetailsListSerializer
fields = ("id", "ip", "host_name")
def validate(self, data):
id_val = data.get('id')
ip = data.get('ip')
host_name = data.get('host_name')
#here the id value is None
print('id val {} '.format(id_val))
return data
If I am understanding correctly, the issue is that you do not see the id field inside of validated_data. If so, I believe this is intentional in the framework:
https://github.com/encode/django-rest-framework/issues/2320
Basically, the id field is read_only by default. Let me know if you have questions that are not answered by Tom's response to that issue.
EDIT: Also feel free to share the higher level use case (what you are planning on doing with the ID inside of validation), and maybe we can offer alternative approaches.
Sorry if this has been asked before, I could not actually find a solution or similar question (maybe using the wrong words).
I'm updating an existing Flask API that receives data from a client we don't control (can't change the JSON data format), using marshmallow and peewee.
The data format comes this way:
{
"site_id": "0102931",
"update_date": "2018/02/11-09:33:23",
"updated_by": "chan1",
"crc": "a82131cf232ff120aaf00001293f",
"data": [{"num": 1,
"id": "09213/12312/1",
"chain": "chain2",
"operator": "0000122",
"op_name": "Fred",
"oid": "12092109300293"
},
{"num": 2,
"id": "09213/12312/2",
"chain": "chain1",
"operator": "0000021",
"op_name": "Melissa",
"oid": "8883390393"
}]
}
We are not interested about anything in the main block, but the site_id, which must be copied into each of the objects in the list when deserializing to create the models and store the data.
This is the model in peeewee:
class production_item(db.Model):
site_id = TextField(null=False)
id_prod = TextField(null=False)
num = SmallIntegerField(null=False)
chain = TextField(null=False)
operator = TextField(null=False)
operator_name = TextField(null=True)
order_id = TextField(null=False)
And this is the marshamallow schema:
class prodItemSchema(Schema):
num=String(required=True)
id=String(required=True)
chain=String(required=True)
operator=String(required=True)
op_name=String(required=False, allow_none=True)
oid=String(required=False, allow_none=True)
I can't find a way to pass the site-id from the main structure with load() method and pre-load / post-load decorators for the prodItemSchema, so the model can't be created. Also, I'd like for marshmallow to validate the whole structure for me, not doing in two parts between the resource and the schema, as they are doing in the code right now.
But can't find a way in the documentation to make something like this, is that possible?
In marshmallow it's possible to pass values from a parent scheme to its children before serialization by using the pre_dump decorator on the parent scheme to set the context. Once the context is set, a function field can be used to obtain the value from the parent.
class Parent(Schema):
id = fields.String(required=True)
data = fields.Nested('Child', many=True)
#pre_dump
def set_context(self, parent, **kwargs):
self.context['site_id'] = parent['id']
return data
class Child(Schema):
site_id = fields.Function(inherit_from_parent)
def inherit_from_parent(child, context):
child['site_id'] = context['site_id']
return child
I am a bit frustrated with this problem using the Django Rest Framework:
I am using a viewset, with a custom serializer. This serializer has its depth set to 1. When i query this viewset I get the correct representation of data for example:
data = {
id: 1,
issue_name: 'This is a problem',
status: {
id: 3,
name: 'todo'
}
}
The problem comes in when I need to update the status. For example if I want to select another status for this issue, for example:
status_new = {
id: 4,
name: 'done'
}
I send the following PATCH back to the server, this is the output:
data = {
id: 1,
issue_name: 'This is a problem',
status: {
id: 4,
name: 'done'
}
}
However, the status does not get updated. Infact, it is not even a part of the validated_data dictionary. I have read that nested relations are read-only. Could someone please tell me what I need to do this in a simple way?
Would really be obliged.
Thanks in advance
As stated in the documentation, you will need to write your own create() and update() methods in your serializer to support writable nested data.
You will also need to explicitly add the status field instead of using the depth argument otherwise I believe it won't be automatically added to validated_data.
EDIT: Maybe I was a bit short on the details: what you want to do is override update in ModelIssueSerializer. This will basically intercept the PATCH/PUT requests on the serializer level. Then get the new status and assign it to the instance like this:
class StatusSerializer(serializers.ModelSerializer):
class Meta:
model = Status
class ModelIssueSerializer(serializers.ModelSerializer):
status = StatusSerializer()
# ...
def update(self, instance, validated_data):
status = validated_data.pop('status')
instance.status_id = status.id
# ... plus any other fields you may want to update
return instance
The reason I mentioned in the comment that you might need to add a StatusSerializer field is for getting status into validated_data. If I remember correctly, if you only use depth then nested objects might not get serialized inside the update() / create() methods (although I might be mistaken on that). In any case, adding the StatusSerializer field is just the explicit form of using depth=1
I usually use custom field for such cases.
class StatusField(serializers.Field):
def to_representation(self, value):
return StatusSerializer(value).data
def to_internal_value(self, data):
try:
return Status.objects.filter(id=data['id']).first()
except (AttributeError, KeyError):
pass
And then in main serializer:
class IssueSerializer(serializers.ModelSerializer):
status = StatusField()
class Meta:
model = MyIssueModel
fields = (
'issue_name',
'status',
)
I would assume that your models mimic your serializer's data. Also, I would assume that you have a one to many relation with the status(es) but you don't need to create them via the issue serializer, you have a different endpoint for that. In such a case, you might get away with a SlugRelatedField.
from rest_framework import serializers
class StatusSerializer(serializers.ModelSerializer):
class Meta:
model = MyStatusModel
fields = (
'id',
'status',
)
class IssueSerializer(serializers.ModelSerializer):
status = serializers.SlugRelatedField(slug_field='status', queryset=MyStatusModel.objects.all())
class Meta:
model = MyIssueModel
fields = (
'issue_name',
'status',
)
Another valid solution would be to leave here the foreign key value and deal with the display name on the front-end, via a ui-select or select2 component - the RESTfull approach: you are handling Issue objects which have references to Status objects. In an Angular front-end app, you would query all the statuses from the back-end on a specific route and then you will display the proper descriptive name based on the foreign key value form Issue.
Let me know how is this working out for you.
I have a Forum Topic model that I want to order on a computed SerializerMethodField, such as vote_count. Here are a very simplified Model, Serializer and ViewSet to show the issue:
# models.py
class Topic(models.Model):
"""
An individual discussion post in the forum
"""
title = models.CharField(max_length=60)
def vote_count(self):
"""
count the votes for the object
"""
return TopicVote.objects.filter(topic=self).count()
# serializers.py
class TopicSerializer(serializers.ModelSerializer):
vote_count = serializers.SerializerMethodField()
def get_vote_count(self, obj):
return obj.vote_count()
class Meta:
model = Topic
# views.py
class TopicViewSet(TopicMixin, viewsets.ModelViewSet):
queryset = Topic.objects.all()
serializer_class = TopicSerializer
Here is what works:
OrderingFilter is on by default and I can successfully order /topics?ordering=title
The vote_count function works perfectly
I'm trying to order by the MethodField on the TopicSerializer, vote_count like /topics?ordering=-vote_count but it seems that is not supported. Is there any way I can order by that field?
My simplified JSON response looks like this:
{
"id": 1,
"title": "first post",
"voteCount": 1
},
{
"id": 2,
"title": "second post",
"voteCount": 8
},
{
"id": 3,
"title": "third post",
"voteCount": 4
}
I'm using Ember to consume my API and the parser is turning it to camelCase. I've tried ordering=voteCount as well, but that doesn't work (and it shouldn't)
This is not possible using the default OrderingFilter, because the ordering is implemented on the database side. This is for efficiency reasons, as manually sorting the results can be incredibly slow and means breaking from a standard QuerySet. By keeping everything as a QuerySet, you benefit from the built-in filtering provided by Django REST framework (which generally expects a QuerySet) and the built-in pagination (which can be slow without one).
Now, you have two options in these cases: figure out how to retrieve your value on the database side, or try to minimize the performance hit you are going to have to take. Since the latter option is very implementation-specific, I'm going to skip it for now.
In this case, you can use the Count function provided by Django to do the count on the database side. This is provided as part of the aggregation API and works like the SQL COUNT function. You can do the equivalent Count call by modifying your queryset on the view to be
queryset = Topic.objects.annotate(vote_count=Count('topicvote_set'))
Replacing topicvote_set with your related_name for the field (you have one set, right?). This will allow you to order the results based on the number of votes, and even do filtering (if you want to) because it is available within the query itself.
This would require making a slight change to your serializer, so it pulls from the new vote_count property available on objects.
class TopicSerializer(serializers.ModelSerializer):
vote_count = serializers.IntegerField(read_only=True)
class Meta:
model = Topic
This will override your existing vote_count method, so you may want to rename the variable used when annotating (if you can't replace the old method).
Also, you can pass a method name as the source of a Django REST framework field and it will automatically call it. So technically your current serializer could just be
class TopicSerializer(serializers.ModelSerializer):
vote_count = serializers.IntegerField(read_only=True)
class Meta:
model = Topic
And it would work exactly like it currently does. Note that read_only is required in this case because a method is not the same as a property, so the value cannot be set.
Thanks #Kevin Brown for your great explanation and answer!
In my case I needed to sort a serializerMethodField called total_donation which is the sum of donations from the UserPayments table.
UserPayments has:
User as a foreignKey
sum which is an IntegerField
related_name='payments'
I needed to get the total donations per User but only donations that have a status of 'donated', not 'pending'. Also needed to filter out the payment_type coupon, which is related through two other foreign keys.
I was dumbfounded how to join and filter those donations and then be able to sort it via ordering_fields.
Thanks to your post I figured it out!
I realized it needed to be part of the original queryset in order to sort with ordering.
All I needed to do was annotate the queryset in my view, using Sum() with filters inside like so:
class DashboardUserListView(generics.ListAPIView):
donation_filter = Q(payments__status='donated') & ~Q(payments__payment_type__payment_type='coupon')
queryset = User.objects.annotate(total_donated=Sum('payments__sum', filter=donation_filter ))
serializer_class = DashboardUserListSerializer
pagination_class = DashboardUsersPagination
filter_backends = [filters.OrderingFilter]
ordering_fields = ['created', 'last_login', 'total_donated' ]
ordering = ['-created',]
I will put it here because the described case is not the only one.
The idea is to rewrite the list method of your Viewset to order by any of your SerializerMethodField(s) also without moving your logic from the Serializer to the ModelManager (especially when you work with several complex methods and/or related models)
def list(self, request, *args, **kwargs):
response = super().list(request, args, kwargs)
ordering = request.query_params.get('ordering')
if "-" in ordering:
response.data['results'] = sorted(response.data['results'], key=lambda k: (k[ordering.replace('-','')], ), reverse=True)
else:
response.data['results'] = sorted(response.data['results'], key=lambda k: (k[ordering], ))
return response