I am starting to work with Django Rest Framework and I am a little confused with Serializers, sometimes its called to serialize and sometimes to deserialize, sometimes is called with the data param, and sometimes it doesn't. When and how is a good use for serializers to serialize and deserialize?
Serialization is the process of preparing your data to be sent ower the network in the case of REST. The result of the serialization is a json/xml in the case of Django REST framework. So you need to serialize your data when you get it, and you deserialize it when you save it to the model with a POST/PUT request.
For further reading: http://www.django-rest-framework.org/api-guide/serializers/
Serialization: Converting complex data types like queryset to native data types like dictionary.
DeSerialization: Converting python native data types to Complex data types.
so both the actions are accomplished with the same serializer. basically when we call a serializer with parameter data (like serializer=Serializer(data=data_dict), it return a complex object. and when we call it with complex object, it gives a dictionary.
So basically there is nothing like deserializer. Serializers perform both serialization and deserialization depending on the parameter we pass and that's how we call it.
we serializer data to send over the network so that it can be easily interpreted and rendered easily in the browser. we deserialize data to get complex object and to probably save it to database.
Thanks! Hope you find this helpful.:)
Related
Is to "deserialize" and "normalize" the same thing or are there differences? A straightforward question. I believe this is generic to front-end models communication to backend database sources/APIs. But if not, this is in the context of Ember Data.
There are differences. When you deseralize your intention is to change from one form say a javascript object and turning it into another form say a ember-data model. When you normalize your intention is to manipulate the data and/or structure of the current form.
So with ember-data you have RESTSerializer with:
serialize which takes a model and turns it into a plain javascript object.
normalize which takes a plain javascript object and returns a plain javascript object.
Compared to transforms/date.js:
deserialize Given a number or string returns a Date object.
serialize Returns a string representation of a Date object.
To put both of these things in context, from the docs on the applyTransforms method on JSONSerializer (this is where the transforms work on the plain javascript object):
Given a subclass of DS.Model and a JSON object this method will
iterate through each attribute of the DS.Model and invoke the
DS.Transform#deserialize method on the matching property of the
JSON object. This method is typically called after the
serializer's normalize method.
I've run into a snag in my views.
Here "filtered_posts" is array of Django objects coming back from the model.
I am having a little trouble figuring out how to get as text data that I can
later pack into json instead of using serializers.serialize...
What results is that the data comes double-escaped (escaped once by serializers.serialize and a second time by json.dumps).
I can't figure out how to return the data from the db in the same way that it would come back if I were using the MySQLdb lib directly, in other words, as strings, instead of references to objects. As it stands if I take out the serializers.serialize, I get a list of these django objects, and it doesn't even list them all (abbreviates them with '...(remaining elements truncated)...'.
I don't think I should, but should I be using the __unicode__() method for this? (and if so, how should I be evoking it?)
JSONtoReturn = json.dumps({
'lowest_id': user_posts[limit - 1].id,
'user_posts': serializers.serialize("json", list(filtered_posts)),
})
The Django Rest Framework looks pretty neat. I've used Tastypie before, too.
I've also done RESTful APIs that don't include a framework. When I do that, I define toJSON methods on my objects, that return dictionaries, and cascade the call to related elements. Then I call json.dumps() on that. It's a lot of work, which is why the frameworks are worth looking at.
What you're looking for is Django Rest Framework. It handles related objects in exactly thew way you're expecting it to (you can include a nested object, like in your example, or simply have it output the PK of the related object for the key).
I have a Django server responsible for serving JSON formatted files to a controller machine. My server gets data from POST requests and serve JSON files on GET requests. I'm wondering what is the most RESTful and efficient way of creating this bridge on the server?
I'm considering creating a JSON model and storing instances on my database for each POST request and pack the instances into JSON files dynamically and serve them on GET requests. The alternative would be creating JSON files on POST requests, saving them in a folder on the server, and serving these files on GET requests.
Which way is better and why? Or am I failing to see a completely better method?
Why are you creating files? You can let a Django view return a JSON response instead of a HTML response:
import json
# in your view:
data = {}
return HttpResponse(json.dumps(data), mimetype="application/json")
Construct the JSON data dynamically from the available data and if the JSON responses are big, add caching (e.g., memcached or Varnish).
This will prevent certain problems (what to do if a GET request is done but there is no JSON file yet?). This way you can generate the JSON based on the data that you have or return JSON with an error message instead.
I looked into json serialization with natural keys and dependencies etc. to control the fields that are serialized. I also tried using the middleware wodofstuff to allow for deeper foreign key serialization. However, I decided to use templates to render the JSON response.
Some pitfalls are
I am responsible for formatting the JSON (more prone to mistakes like a missing semi-colon)
I am responsible for escaping characters
Renders slower than buit-in serialization?
Some advantages are
I have control over what is serialized (even if the underlying models is changed)
I can format many to many or foreign key relationships on the JSON file however I like
TLDL; In my case, the format of the JSON file I needed was very custom. It had dictionary in list in dictionary. Some fields are iterative so I have for loops in the templates to render them. However, the format requires some of the fields in the iterated object to be encapsulated in a list as oppose to a dictionary.
This was an obstacle I ran into while considering simplejson. By using something like
import simplejson as json
def encode_complex(obj):
if isinstance(obj, complex):
return [obj.real, obj.imag]
raise TypeError(repr(o) + " is not JSON serializable")
json.dumps(2 + 1j, default=encode_complex)
'[2.0, 1.0]'
I could manage to return iterated data; however, I needed iteration in iteration and custom object types (list or dict) to encapsulate certain iterations. Finally, (may it be lack of knowledge or lack of patience) I decided to just do it in the templates.
I feel like rendering it through templates is not the most scalable or 'smartest' way of doing it, could it possibly be done in a better way? Please feel free to prove me right or wrong.
I'm currently working on a REST api, using django. I started using the nice djangorestframework, which I loved to use the "View" class.
But, I'm facing with the serialization problem.
I do not like the Serialization using the Serializer classes.
The main goal is to prepare a sort of giant dict, with all the infos, and give it to a renderer class which translate it into an xml, json, yaml, depending on the "Accept:" HTTP header. The goal is classy, but 60% of the CPU time is spend on creating the "GIANT DICT".
This dict can be created using django Models, but I think using on the fly instanciated classes and object is VERY un-efficient ? I'm trying to use some QuerySet methods to filter which models member I want to have, and getting a simple dict : the ::values() method, but unfornately, I can't access the m2m and foreignkey from my models.
Did you already tried this ? Any though ?
You could use the QuerySet's iterator method:
... For a QuerySet which returns a large number of objects that you only
need to access once, this can results in better performance and a
significant reduction in memory.
Your code should looks like:
for obj in SomeModel.objects.values_list('id', 'name').iterator():
# do something
I've found myself unsatisfied with Django's ability to render JSON data. If I use built in serializes then database foreign key relationships are not included in the data (only the keys). Also, it seems to be impossible to include custom data in the json feed that isn't part of the model being serialized.
As a test I implemented a template that rendered some JSON for the resultset of a particular model. I was able to include/exclude whatever parts of the model I wanted and was able to include custom data as well.
The test seemed to work well and wasn't slower than the recommended serialization methods.
Are there any pitfalls to this using this method of serialization?
While it's hard to say definitively whether this method has any pitfalls, it's the method we use in production as you control everything that is serialized, even if the underlying model is changed. We've been running a high traffic application in for almost two years using this method.
Hope this helps.
One problem might be escaping metacharacters like ". Django's template system automatically escapes dangerous characters, but it's set up to do that for HTML. You should look up exactly what the template escaping does, and compare that to what's dangerous in JSON. Otherwise, you could cause XSS problems.
You could think about constructing a data structure of dicts and lists, and then running a JSON serializer on that, rather than directly on your database model.
I don't understand why you see the choice as being either 'use Django serializers' or 'write JSON in templates'. The middle way, which to my mind is much more robust and fits your use case well, is to build up your data as Python lists/dictionaries and then simply use simplejson.dumps() to convert it to a JSON string.
We use this method to get custom JSON format consumed by datatables.net
It was the easiest method we find to accomplish this task and it looks very fine with no problems so far.
You can find details here: http://datatables.net/development/server-side/django
So far, generating JSON from templates, we've run into the need to escape newlines. Looking at doing simplejson.dumps() next.