I have a table into mysql that is the type TextField (django) by using the JSONField.
This is how my model looks
from django.db import models
from json_field import JSONField
class Model(models.Model):
obj = JSONField()
The value I send via tastypie is
json_string = '{"data":"value"}'
Into the database I can see
{"data":"value"}
But when retrive the data with curl I get something like this
"{u'data': u'value'}"
What I can do to not have the python u'field' representation into the tastypie's output ?
thanks!
I fixed this issue like so:
def dehydrate_user_inputs(self, bundle):
requirement = Requirement.objects.get(pk = bundle.obj.pk)
user_inputs = json.dumps(requirement.user_inputs)
return user_inputs
My JSONField is named user_inputs. Requirement is the model that it belongs to.
I feel weird doing a Query here when Tastypie has already done so for me, but, this works. I'd love if there are better solutions.
The error you're seeing is caused by Tastypie treating the JSONField like a TextArea and calling str() on the object JSONField returns before returning it to the caller.
Another approach is to use fields.ApiFields for the JSONField. This works because fields.ApiFields does not perform any conversions on either the way in (hydrate()) or way out (convert()). This is exactly what we want - the underlying JSONField will convert the JSON object to a string for persistence on the way in and recreate the object from the string on the way out. Thus, tastypie does not need to do anything. My code looks a bit like this (class/variable names based on OP's example) -
class JSONField(fields.apiField):
""" Wrapper over fields.apiField to make what we're doing here clear """
pass
class MyModelResource(ModelResource):
obj = JSONField('obj')
Use DictField:
obj = fields.DictField(attribute='obj')
I was running into similar problems where my unicode strings would be returned in a weird format in the API (I think the raw encoded strings were returned as opposed to the actual utf-8 characters).
Anyway instead of using the dehydrate method and re-doing the query, you are better off with a custom serializer in your resources.
This is what I used:
class JSONSerializer(Serializer):
'''using the standard json library for better unicode support,
also note django.utils.simplejson, used in the standard Tastypie serializer,
is set for depreciation'''
def to_json(self, data, options=None):
options = options or {}
data = self.to_simple(data, options)
return json.dumps(data)
then in your resources:
class PlaceResource(ModelResource):
class Meta:
queryset = Place.objects.all()
resource_name = 'place'
serializer = JSONSerializer()
Related
I want to save the python dictionary inside the Django model as JSON and I want to convert that JSON back into a python dictionary when that data is fetched.
I know I can do it inside view but I want to implement it in the model so it can return dictionary object when queried.
is there any signal or any post_fetch method that I can use to achieve it, I couldn't find anything googling it...
You might want to look into simply using the Python JSON package, unless you are using Postgres as your database - in which case JSONField is the way to go. The json package is explained here and you could use it in a model like so if I understand what you are saying:
import json
class MyModel(models.Model):
json_field = models.TextField() # or you can use JSONField if using Postgres
#property
def get_json_field_as_dictionary(self):
return json.loads(self.json_field)
def set_json_field(self, mydict):
self.json_field = json.dumps(mydict)
#classmethod
def get_json_from_dictionary(cls, mydict):
return json.dumps(mydict)
When you are saving to the database, you can use json.dumps(myDictionary) or convert the dictionary to JSON by calling MyModelObject.set_json_field(myDictionary) to convert a Python dictionary to JSON and then store it in the json_field of the model. To retrieve the JSON data as a dictionary, you simply call MyModel.objects.last().get_json_field_as_dictionary (or whatever you prefer to call it, json_dictionary perhaps so it would be MyModel.objects.last().json_dictionary) and it will return the value of that property as if it were an element in the model without having to do the conversion each time.
Or, if you are using Postgres as your backend, this is a lot easier with JSONField:
class MyModel(models.Model):
json_field = models.JSONField(null=True)
And to save:
myObject = myModel.objects.create(json_field=myDictionary)
If you clarify I can update my answer to explain better.
You may want to use JSONField. It allows storing data encoded as JSON and retrieve them as the corresponding Python data type, including dictionary.
JSONField only works on PostgreSQL with Django < 3.1, but works on any database with Django >= 3.1.
Using Django Rest Framework 3, Function Based Views, and the ModelSerializer (more specifically the HyperlinkedModelSerializer).
When a user submits a form from the client, I have a view that takes the request data, uses it to call to an external API, then uses the data from the external API to populate data for a model serializer.
I believe I have this part working properly, and from what I read, you are supposed to use context and validate()
In my model serializer, I have so far just this one overidden function:
from django.core.validators import URLValidator
def validate(self, data):
if 'foo_url' in self.context:
data['foo_url'] = self.context['foo_url']
URLValidator(data['foo_url'])
if 'bar_url' in self.context:
data['bar_url'] = self.context['bar_url']
URLValidator(data['bar_url'])
return super(SomeSerializer, self).validate(data)
Just in case, the relevant view code is like so:
context = {'request': request}
...
context['foo_url'] = foo_url
context['bar_url'] = bar_url
s = SomeSerializer(data=request.data, context=context)
if s.is_valid():
s.save(user=request.user)
return Response(s.data, status=status.HTTP_201_CREATED)
Now assuming I have the right idea going (my model does populate its foo_url and bar_url fields from the corresponding context data), where I get confused is how the validation is not working. If I give it bad data, the model serializer does not reject it.
I assumed that in validate(), by adding the context data to the data, the data would be checked for validity when is_valid() was called. Maybe not the case, especially when I print out s (after using the serializer but before calling is_valid()) there is no indication that the request object's data has been populated with the context data from validate() (I don't know if it should be).
So I tried calling the URLValidators directly in the validate() method, but still doesn't seem to be working. No errors despite giving it invalid data like 'asdf' or an empty python dict ({}). My test assertions show that the field indeed contains invalid data like '{}'.
What would be the proper way to do this?
You're not calling the validator.
By doing URLValidator(data['bar_url']) you're actually building an url validator with custom schemes (see the docs) and that's it. The proper code should be:
URLValidator()(data['bar_url'])
Where you build a default url validator and then validate the value.
But anyway I would not use this approach, what I would do instead is directly add the extra data (not using the context) and let DRF do the validation by declaring the right fields:
# Somewhere in your view
request.data['bar_url'] = 'some_url'
# In serializer:
class MySerializer(serializers.ModelSerializer):
bar_url = serializers.URLField()
class Meta:
fields = ('bar_url', ...)
To answer your comment
I also don't understand how this also manages to make it past the
Django's model validation
See this answer:
Why doesn't django's model.save() call full_clean()?
By default Django does not automatically call the .full_clean method so you can save a model instance with invalid values (unless the constraints are on the database level).
Complete DRF beginner here... I'm confused about the following concepts:
Let's say I POST some data, including a complex JSON blob for one of the fields, in order to create an object. Where should I actually create this object? Looking at the 3.1 docs, it seems like two places are equally valid for this: Serializer.create() and ViewSet.create(). How do I decide where to create my object and which way is considered "canonical"?
I understand that I need to run Serializer.is_valid() in order to validate the POSTed data. However, what is the difference between .data and .validated_data? They appear to be the same.
Finally, what is the "canonical" way to use a JSONField (e.g. django-jsonfield, but I'm not married to this package/implementation)? I have a model with several JSONFields and would like to use it "correctly" in DRF. I am aware of https://stackoverflow.com/a/28200902/585783, but it doesn't seem enough.
EDIT: My use case is an API POST that includes a complex JSON blob in one of the fields. I need to parse the JSON field, validate it, get/create several objects based on it, link new and existing objects, and finally store the JSON field in one of the new objects. So, I need to do custom validation for this JSON field by parsing it to python:
from django.utils.six import BytesIO
from rest_framework.parsers import JSONParser
class MySerializer(serializers.ModelSerializer):
my_json_field = JSONSerializerField()
def validate_my_json_field(self, value):
stream = BytesIO(value)
list_of_dicts = JSONParser().parse(stream)
# do lots of validation to list_of_dicts
# render list_of_dicts back to a JSON string
return validated_list_of_dicts_as_json
Now, depending on which way I choose in Concept 1, I have to parse the validated JSON again to create my objects in create(), which doesn't feel right.
Thanks in advance!
The contents of HTTP requests (POST, GET, PUT, DELETE) will always be processed by the views (View, APIView, generic views, viewsets). The serializers are just part of how these views process the requests. Serializers are the "means" to connect the View layer with the Model layer. For what serializers do specifically, please read the first paragraph of the this page of the official docs.
To answer #1: you almost always do not need to touch either unless you have a very specific use case. In those extraordinary cases:
You override Serializer.create() if you have to customize how model
instances are converted into native Python objects and vice versa. (e.g. create multiple objects)
You override ViewSet.create() if you need to customize how the actual request itself will be processed. (e.g. if there is an additional query parameter in the request, add some response headers)
To answer #2, you almost never need to use is_valid() when using generic views or ViewSets. They already do it under the hood for you. The serializer's .data and .validated_data are a bit tricky to explain. The former contains the Python datatype representation of the queryset/model instances you want to serialize, while the latter is the result of the validation process involved in checking if a Python object conforms to that particular Python datatype representation mentioned earlier, which in turn can be converted into a model instance. If that did not make sense, refer to Serializing objects and Deserializing objects.
As for #3, what do you mean by JSON field? As far as I know, Django does not have a model field called JSONField. Is this from a third party package or your own custom written model field? If so, then you will probably have to find or write a package that will let you integrate it with DRF smoothly and "correctly" whatever that means.
EDIT
Your use case is too complicated. I can only give you rough code for this one.
class MyModelSerializer(serializers.ModelSerializer):
my_json_field = JSONSerializerField()
class Meta:
model = MyModel
def validate(self, data):
# Get JSON blob
json_blob = data['my_json_field']
# Implement your own JSON blob cleanup method
# Return None if invalid
json_blob = clean_json_blob(json_blob)
# Raise HTTP 400 with your custom error message if not valid
if not json_blob:
raise serializers.ValidationError({'error': 'Your error message'})
# Reassign if you made changes and return
data['my_json_field'] = json_blob
return data
def create(self, validated_data):
json_blob = validated_data['my_json_field']
# Implement your object creation here
create_all_other_objects_from_json(json_blob)
# ...
# Then return a MyModel instance
return my_model
I am working on an application that uses Django and MongoDB (as a model). I am trying to initialize (by hand) a model object (in order to send it further to another server), using the data got from a form. The model looks like this:
class MyModel(DynamicDocument):
study_name = StringField(default="first study")
individual_name = StringField(default="individual")
file_list = ListField(StringField) # this is a list of paths to some files
In the form I am doing the following:
pilot_object = MyModel()
pilot_object.individual_name = self.data['individual_name']
pilot_object.study_name = self.data['study_name']
pilot_object.file_list = files_list #file paths (strings)
where self.data is the data received from the form.
Now I want to serialize to JSON this object (pilot_object) as described in the documentation:
data_serialized = serializers.serialize('json', [pilot_object, ])
but I get this error:
'MetaDict' object has no attribute 'concrete_model'
and the serialization fails.
Can anyone help?
Yes , for mongoengine documents use -- 'from rest_framework_mongoengine import serializers' rather from rest_framework import serializers. And the serializer class should inherit DocumentSerializer.
If you have meta {Inheritance:True} then remove it. It is looking for concrete class which i think you haven't designed so no need for inheritance to be True anyhow. It worked for me.
This is due to you're using an older version of django. Check the difference between 1.3 and 1.4, concrete_model is only available for 1.4 and higher.
I'm using Mongodb with mongoengine as a backend for a API in Django.
The framework I'm using to create the api is Django Rest Framework.
I need to store a dictionary in a field in Mongo and the best I've done when the method post is called is to use a charfield and parse the dictionary in the function restore_object.
There is a better way to achieve this goal?
It's better to create a dict field? I don't know how hard this could be.
Thank you.
edited to show some code, notice that I store the dictionary as a dict (DictField) and it's content could change from one object to other.
my mongoengine model is something like:
class MyDoc(mongoengine.Document):
name = mongoengine.StringField(max_length=200)
context = mongoengine.DictField()
and my serializer something like:
class MyDocSerializer(serializers.Serializer):
name = serializers.CharField(max_length=200)
context = serializers.CharField()
url = serializers.HyperlinkedIdentityField(
view_name="drf:mydoc-detail",)
def __init__(self,*args,**kwargs):
super(MyDocSerializer,self).__init__(*args,**kwargs)
def restore_object(self, attrs, instance=None):
# Parse string to dict
# this is so ugly, notice I had to repace ' for " to
# avoid an error parsing the json
context = JSONParser().parse(
StringIO.StringIO(
attrs['context'].replace("'","\"")
)
)
attrs['context'] = context
if instance is not None:
instance.name = attrs['name']
instance.context = context
return instance
return MyDoc(**attrs)
Rather than deal with the dictionary field in the Serializer's restore_object, you'll probably end up with something slightly cleaner, if instead you use a custom field for the dictionary field, that manages converting between the dictionary representation and internal char based storage.
You'll want to subclass serializers.WritableField and override the to_native() and from_native methods.
Relevant docs here.
Note: WritableField class that was present in version 2.x no longer exists. You should subclass Field and override to_internal_value() if the field supports data input.
Update: As of 3.0.4 you can now use serializers.DictField... http://www.django-rest-framework.org/api-guide/fields/#dictfield