Sorry if this has been asked before, I could not actually find a solution or similar question (maybe using the wrong words).
I'm updating an existing Flask API that receives data from a client we don't control (can't change the JSON data format), using marshmallow and peewee.
The data format comes this way:
{
"site_id": "0102931",
"update_date": "2018/02/11-09:33:23",
"updated_by": "chan1",
"crc": "a82131cf232ff120aaf00001293f",
"data": [{"num": 1,
"id": "09213/12312/1",
"chain": "chain2",
"operator": "0000122",
"op_name": "Fred",
"oid": "12092109300293"
},
{"num": 2,
"id": "09213/12312/2",
"chain": "chain1",
"operator": "0000021",
"op_name": "Melissa",
"oid": "8883390393"
}]
}
We are not interested about anything in the main block, but the site_id, which must be copied into each of the objects in the list when deserializing to create the models and store the data.
This is the model in peeewee:
class production_item(db.Model):
site_id = TextField(null=False)
id_prod = TextField(null=False)
num = SmallIntegerField(null=False)
chain = TextField(null=False)
operator = TextField(null=False)
operator_name = TextField(null=True)
order_id = TextField(null=False)
And this is the marshamallow schema:
class prodItemSchema(Schema):
num=String(required=True)
id=String(required=True)
chain=String(required=True)
operator=String(required=True)
op_name=String(required=False, allow_none=True)
oid=String(required=False, allow_none=True)
I can't find a way to pass the site-id from the main structure with load() method and pre-load / post-load decorators for the prodItemSchema, so the model can't be created. Also, I'd like for marshmallow to validate the whole structure for me, not doing in two parts between the resource and the schema, as they are doing in the code right now.
But can't find a way in the documentation to make something like this, is that possible?
In marshmallow it's possible to pass values from a parent scheme to its children before serialization by using the pre_dump decorator on the parent scheme to set the context. Once the context is set, a function field can be used to obtain the value from the parent.
class Parent(Schema):
id = fields.String(required=True)
data = fields.Nested('Child', many=True)
#pre_dump
def set_context(self, parent, **kwargs):
self.context['site_id'] = parent['id']
return data
class Child(Schema):
site_id = fields.Function(inherit_from_parent)
def inherit_from_parent(child, context):
child['site_id'] = context['site_id']
return child
Related
I'm not very familiar with DRF and I haven't found a solution on google for this problem (most answers are about a model with a field as enum, my problem is different)
You see, we have an Enum in a Django application. Let's call it SomeValuesEnum.
class SomeValuesEnum(Enum):
ONE_VALUE = "One value"
ANOTHER_VALUE = "Another value"
What I need to do is to create a GET endpoint that returns the following
{
"count": 2,
"page_count": 1,
"next": null,
"previous": null,
"results": [
{
"value": "One value",
"name": "ONE_VALUE"
}, {
"value": "Another value",
"name": "ANOTHER_VALUE"
}
]
}
I know I need to create a serializer, but I haven't been able to create one and "feed it".
For example, I started with something like this:
class SomeValueSerializer(serializers.Serializer):
Meta:
model = SomeValuesEnum,
fields = '__all__'
and on the view:
class SomeValueListView(APIView):
serializer_class = SomeValueSerializer
def get(self, request):
choices = [{"value": target.value, "name": target.value.capitalize()} for target in SomeValuesEnum]
serializer = SomeValueSerializer(data=choices)
return Response(status=status.HTTP_200_OK, data=serializer.data)
I also tried this
class IncidentSerializer(serializers.Serializer):
name = serializers.CharField(required=False, allow_blank=True, max_length=100)
value = serializers.CharField(required=False, allow_blank=True, max_length=100)
I'm not sure if I'm failing on the creation of the serializer, or in how I invoke him on the view (or maybe both)
Any guidance on the right direction will be greatly appreciate it.
An enum is not a django model. You can have a DRF serializer for something that isn't a model, but you shouldn't give it a model field of something that isn't a model.
See here: Declaring Serializers
Not here: Model Serializers
Furthermore, you are creating a class-based view here:
SomeValueListView(APIView)
You don't necessarily need this, you could use a function based view, which may be easier for you to understand given you are new to DRF. I only say function based views are easier to understand since there isn't so much built in functionality. This can make it easier to debug for someone new to DRF. You can still use the serializer by calling it directly in the view.
See here: Function Based Views
And here: Serializing Objects
Finally...given this code:
choices = [{"value": target.value, "name": target.value.capitalize()} for target in SomeValuesEnum]
I am making the assumption that there could be multiple distinct objects going into this serializer, given that you are using a list comprehension. Either you need to call the serializer separately for each object in the array, or you call the serializer with (many=True). Pretty sure this is your main issue. Like this:
serializer = SomeValueSerializer(data=choices, many=True)
See here: Dealing with multiple objects
Also, in my experience it is better to parse incoming data to the serializer within the serializer itself, not in the view. To me it is a separation of concerns issue, but other people may feel differently. This would look something like this:
class SomeValueListView(APIView):
serializer_class = SomeValueSerializer
def get(self, request):
serializer = SomeValueSerializer(data=SomeValuesEnum)
return Response(status=status.HTTP_200_OK, data=serializer.data)
class SomeValueSerializer(serializer.Serializer):
def to_internal_value(self, data)
name = data.value.capitalize()
value = data.value
return {
'name': name,
'value': value,
}
I'm learning and new to Django Rest Framework and I'm having an issue in serializer validations and ListSerializer update method.
I have an APIView class which handles the put request. I just wanted to have a custom validation and so I've overridden validate method in the serializer class.
From postman I'm sending a JSON data to this APIView Class.
Sample JASON data:
[
{
"id": 1,
"ip": "10.1.1.1",
"host_name": "hostname 1"
},
{
"id": 2,
"ip": "10.1.1.2",
"host_name": "hostname 2"
}
]
When I receive the data and do serializer.is_valid() it passes the flow to the overridden validate function. But in there when I check for the attrs argument, I get all the fields except the id. The key and value for id are not present. It shows None.
The same issue occurred to me when I was trying to override the update method in ListSerializer.
when I tried the below code in the ListSerializer's update method,
data_mapping = {item['id']: item for item in validated_data}
I got an error saying KeyError 'id'.
It seems it's not accepting id field and I'm not sure why! Please, someone explain this to me if I'm wrong anywhere.
Serializer class
from rest_framework import serializers
from .models import NoAccessDetails
class NoAccessDetailsListSerializer(serializers.ListSerializer):
def update(self, instance, validated_data)
data_mapping = {data.id: data for data in instance}
#Here I'm getting KeyError ID
validated_data_mapping = {item['id']: item for item in validated_data}
return
class NoAccessDetailsSerializer(serializers.ModelSerializer):
class Meta:
model = NoAccessDetails
list_serializer_class = NoAccessDetailsListSerializer
fields = ("id", "ip", "host_name")
def validate(self, data):
id_val = data.get('id')
ip = data.get('ip')
host_name = data.get('host_name')
#here the id value is None
print('id val {} '.format(id_val))
return data
If I am understanding correctly, the issue is that you do not see the id field inside of validated_data. If so, I believe this is intentional in the framework:
https://github.com/encode/django-rest-framework/issues/2320
Basically, the id field is read_only by default. Let me know if you have questions that are not answered by Tom's response to that issue.
EDIT: Also feel free to share the higher level use case (what you are planning on doing with the ID inside of validation), and maybe we can offer alternative approaches.
I have a setup where I need to write an API for an existing javascript datamodels which i do not want to touch (for now). The javascript data has a different architecture than I want to have on the server. So my goal is to transform the data that I get from javascript to fit my database model when data is sent to the API. When data is requested from the API, it should match the expected data model of javascript.
I wonder if I can do that with ModelSerializers, if yes, where is the right place to transform the data? In the view? In the serializer.
My setup is like so:
//javascript structure
{
scores: [
{
id: 12,
points: 2
maxpoints: 12
siteuxid: 'EXAMPLE'
},
{ ... }
]
}
//More models in django
{
scores: [
{
id: 12,
points: 2,
question: {
id: 12,
maxpoints: 12,
siteuxid: 'EXAMPLE'
}
},
]
}
Are there any examples anyone can point me to, that achive the same? Basically it is all about having different data structures in server and client and making them compatible. Googleing did not help.
EDIT:
My first problem is that I do not get all posted data in my Serializer. When I post
{
"scores": [{"id":"QFELD_1.1.3.QF2","siteuxid":"VBKM01_VariablenTerme","section":1,"maxpoints":4,"intest":false,"uxid":"ER2","points":0,"value":0,"rawinput":"363"}]
}
to
class UserDataSerializer(serializers.ModelSerializer):
scores = ScoreSerializer(many=True, required=False)
def create(self, validated_data):
print('userDataSerializer validated_data', validated_data)
...
class ScoreSerializer(serializers.ModelSerializer):
id = serializers.CharField(required=False, allow_blank=True, max_length=100)
question = QuestionSerializer(required=False)
class Meta:
model = Score
fields = ('id', 'question', 'points', 'value', 'rawinput', 'state')
I only get the output
userDataSerializer validated_data {'scores': [OrderedDict([('id', 'QFELD_1.1.3.QF2'), ('points', 0), ('value', 0), ('rawinput', '363')])]}
without the score.maxpoints and so on (as it is not in the serializer, but how can I add it? To validated data in order to create a proper question object from the posted data in UserDataSerializer)
The answer is yes, you will use the view to modify your input data, as you must do these tweaks in your data before the view sends the data to the serializers. This is due to the same reason that you only see the attributes of your serializer in your validated data - the serializer ignores all the attributes it does not recognize.
So, first of all, change request.data in your view's post method to make it structured as you need.
def post(self, request, *args, **kwargs):
request.data['question'] = {
'maxpoints': request.data.pop('maxpoints'),
'siteuxid': request.data.pop('siteuxid'),
}
This should be all you need to get started.
However, note that it's strange that question has id: 12 in your example. If you are trying to create a question object along with your score object, it should have no id. If the question is an existing object, though, you should not be sending a dict, but only the id instead.
For example, you should send question: 1 in your input. DRF's ModelSerializer is smart enough to know that the score you are trying to save is to be related with the question which has id = 1. While you're at it, inspect the serializer's validated_data and you'll see the instance of question with id = 1. Magic!
So we've got a Django/tastypie server going that has the following (simplified) model. If I remove our alter_detail_data_to_serialize:
{
"release": false,
"resource_uri": "/api/packages/1",
"id": 1,
"branch": {
"resource_uri": "/api/branches/1",
"id": 1,
# ... more bits the client doesn't need to know about
"version": "0.1"
},
"revision": "72"
}
With the alter, it becomes:
{
"release": false,
"branch": "0.1",
"revision": "72"
}
Which is what we want to work with through the API: It removes the foreign key traversal to simplify the JSON and do any CRUD without issues: supplying a version is sufficient to identify the branch. The issue is, to query this, requires
/api/packages?branch__version=1.0, where this not intuitive and exposes the structure of the underlying database. We'd prefer to be able to query:
/api/packages?branch=1.0 and handle the foreign key traversal on the backend.
alter_detail_data_to_serialize and alter_deserialized_detail_data allow me to interface with the simplified JSON and do any non-searching CRUD without issues, but is it possible to allow a /api/packages?branch=1.0 query and have the django/tastypie server correct that to /api/packages?branch__version=1.0, hiding the database structure?
Some additional code that might be relevant:
class PackageResource(ModelResource):
branch = fields.ForeignKey(BranchResource, 'branch', full=True)
class Meta:
queryset = Packages.objects.all()
resource_name = 'packages'
collection_name = 'packages'
def alter_detail_data_to_serialize(self, request, data):
data.data['branch'] = data.data['branch'].data['version']
return data
Branch Resource:
class BranchResource(ModelResource):
class Meta:
queryset = Branches.objects.all()
resource_name = 'branches'
collection_name = 'branches'
In the object resource, you can add something like this:
class PackageResourse(ModelResource):
version = fields.CharField( attribute = 'branch__version' )
class Meta:
resource_name='package'
What this is doing it making the PackageResource have a variable that is the same as its foreign key's variable. Now you can use api/packages?version=1.0 on the PackageResource.
Suppose I have a resource like below..
class PostResource(ModelResource):
children = fields.ToManyField('MyApp.api.resources.PostResource',
attribute='comments', full=True, null=True)
Basically, I want to return this children field only and flatten it.
It will look like
[ {child-1-data}, {child-2-data} ]
rather than
{ children: [ {child-1-data}, {child2-data} ] }
How can I do that?
Additionaly, if I want a different representation of the same model class, should I create a new resource class as bellow?
class PostNormalResource(ModelResource):
class Meta:
queryset= models.Post.objects.all()
fields = ['text', 'author']
Not really the answer you are looking for but some discoveries I made while digging.
Normally you would modify the bundle data in dehydrate. See the tastypie cookbook.
def dehydrate(self, bundle):
bundle.data['custom field'] = "This is some additional text on the resource"
return bundle
This would suggest you could manipulate your PostResource's bundle data along the lines of:
def dehydrate(self, bundle):
# Replace all data with a list of children
bundle.data = bundle.data['children']
return bundle
However this will error, AttributeError: 'list' object has no attribute 'items', as the tastypie serializer is looking to serialize a dictionary not a list.
# "site-packages/tastypie/serializers.py", line 239
return dict((key, self.to_simple(val, options)) for (key, val) in data.data.items())
# .items() being for dicts
So this suggests you need to look at different serializers. (Or just refer to post['children'] when processing your JSON :-)
Hope that helps get you in the right direction
And secondly yes, if you want a different representation of the same model then use a second ModelResource. Obviously you can subclass to try and avoid duplication.
You could try overriding the alter_detail_data_to_serialize method. It it called right after whole object has been dehydrated, so that you can do modifications on the resulting dictionary before it gets serialized.
class PostResource(ModelResource):
children = fields.ToManyField('MyApp.api.resources.PostResource',
attribute='comments', full=True, null=True)
def alter_detail_data_to_serialize(self, request, data):
return data.get('children', [])
As to different representation of the same model - yes. Basically, you shouldn't make a single Resource have many representations as that would lead to ambiguity and would be difficult to maintain.