I've run into a snag in my views.
Here "filtered_posts" is array of Django objects coming back from the model.
I am having a little trouble figuring out how to get as text data that I can
later pack into json instead of using serializers.serialize...
What results is that the data comes double-escaped (escaped once by serializers.serialize and a second time by json.dumps).
I can't figure out how to return the data from the db in the same way that it would come back if I were using the MySQLdb lib directly, in other words, as strings, instead of references to objects. As it stands if I take out the serializers.serialize, I get a list of these django objects, and it doesn't even list them all (abbreviates them with '...(remaining elements truncated)...'.
I don't think I should, but should I be using the __unicode__() method for this? (and if so, how should I be evoking it?)
JSONtoReturn = json.dumps({
'lowest_id': user_posts[limit - 1].id,
'user_posts': serializers.serialize("json", list(filtered_posts)),
})
The Django Rest Framework looks pretty neat. I've used Tastypie before, too.
I've also done RESTful APIs that don't include a framework. When I do that, I define toJSON methods on my objects, that return dictionaries, and cascade the call to related elements. Then I call json.dumps() on that. It's a lot of work, which is why the frameworks are worth looking at.
What you're looking for is Django Rest Framework. It handles related objects in exactly thew way you're expecting it to (you can include a nested object, like in your example, or simply have it output the PK of the related object for the key).
Related
If I perform a prefetch_related('toppings') for a queryset, and I want to later filter(spicy=True) by fields in the related table, Django ignores the cached info and does a database query. I found that this is documented (under the Note box) and seems to happen for all forms of caching (select_related(), already evaluated querysets, etc.) when another filter() is performed.
However, is there some sort of super secret hidden time-saving shortcut to filter locally (using the cache and not hitting the database) without having to write the python code to loop the queryset (using list/dict comprehension, etc.)? Maybe something like a filter_locally(spicy=True)?
EDIT:
One of the reasons why a list/comprehension doesn't work well for me is because a list/dict does not have the queryset methods. In my case, the first level M2M field, toppings, isn't the end goal for me and I need to check a 2nd related M2M field (which I have already pre-fetched as well). While this is also possible using list comprehension, it's just much simpler to have something such as filter_locally(spicy=True, origin__country='Spain') because:
it allows accessing many levels of related fields with minimal effort
it allows chaining other queryset methods
it's easier to read because it's consistent with the familiar filter()
it's easier to modify existing code using filter() without prefetch to add this optimization in without much changes.
But from the responses, Django has no such support :(
You have to write the python code to loop through the queryset (a list/dict comprehension is ideal). All the filter() code knows how to do is add filtering language to the SQL sent to the database. Filtering locally is a totally different problem than filtering remotely, so the solutions to those two separate problems won't be able to share any logic.
A list comprehension one-liner would be pretty straightforward, though; the syntax might not be much more complex than with filter().
If you're filtering on a boolean doing the list comprehension is pretty easy. You can also swap out the topping.spicy==True for a string comparison or whatever.
I would do something like:
qs = Pizza.objects.all().prefetch_related('toppings')
res = list(qs)
def get_spicy(qs):
res = list(qs)
return [pizza for pizza in res if any(topping.spicy==True for
topping in pizza.toppings.all())]
That is if you want to return the pizza object if any of its toppings is spicy. You can also replace the any() with all() to check for all, and do a lot of pretty powerful queries with this syntax. I'm somewhat surprised that there is no easy way to do this in django. It seems like a lot of these simple queries should be easy to implement in a generic manner.
The above code assumes a many2many. It should be easy to modify to work with a simple FK relationship such as a one2one or one2many.
Hope this was helpful.
So I have this situation where i want to serialize models with non-model values. I got the serialization of the models [both queryset and single model itself] but trying to combine that with non-model values seem problematic.
for e.g. I want to JSONify User with some status of the request.
Assume model_to_JSON does model=>JSON, so
# it messes the 'user' json by further escaping it,
#which becomes unparseble on client since its a string now
dumps({ 'user': model_to_JSON(user_obj), 'status': 'ok'})
I could do couple of hacks, by first doing loads on the converted model-json. but thats such a hacky way and so much time is just wasted in dumps=>loads=>dumps
second option is string concatenation by doing individuals dumps and then concatenate the strings by stripping '}' of the leftmost string and '{' of the rightmost string with ','.
The Django serializers are very specifically written towards models/queryset, so I can't really override them.
So has anyone faced this problem before and any solutions you could share
You can look at Django Full Serializers, which was the approach I used a long, long time ago.
Another alternative would be to simply write your own serialisation function, that allows you to pass in attribute names (that will be looked up at serialisation time). I've done this, too. If you go down this approach, be aware there is already a django function model_to_dict, which does the pre-serialisation.
A third method might be to use django's forms as an intermediate to serialisation: this might be useful if you may be deserialising data back from the user, too.
I'm currently working on a REST api, using django. I started using the nice djangorestframework, which I loved to use the "View" class.
But, I'm facing with the serialization problem.
I do not like the Serialization using the Serializer classes.
The main goal is to prepare a sort of giant dict, with all the infos, and give it to a renderer class which translate it into an xml, json, yaml, depending on the "Accept:" HTTP header. The goal is classy, but 60% of the CPU time is spend on creating the "GIANT DICT".
This dict can be created using django Models, but I think using on the fly instanciated classes and object is VERY un-efficient ? I'm trying to use some QuerySet methods to filter which models member I want to have, and getting a simple dict : the ::values() method, but unfornately, I can't access the m2m and foreignkey from my models.
Did you already tried this ? Any though ?
You could use the QuerySet's iterator method:
... For a QuerySet which returns a large number of objects that you only
need to access once, this can results in better performance and a
significant reduction in memory.
Your code should looks like:
for obj in SomeModel.objects.values_list('id', 'name').iterator():
# do something
In my python code in views.py, i have made a list of Object instances. For example, if one of my objects in models.py is Sentence, then I at one point have an list of Sentences:
sentenceList = [Sentence1,Sentence2,...,SentenceN]
I have been happily using the template language in Django to send this off to a template where I can loop through the sentences in a template, but I am now implamenting some AJAX functionality, and would like to send something similar to this list "sentenceList" through a serialization through an AJAX portal.
It seems that I am between two worlds here though: the regular python JSON seiralizer, which serializes lists and python object, and simplejson, which only deals with pure Django Model queries.
Does anyone have a suggestion of how I should handle this (including not using lists of object instances if it is for some reason not good form)?
Thank you!
-Andrew
The Django serializer should be able to deal with a regular list of Django objects, according to the docs:
(Actually, the second argument can be any iterator that yields Django objects, but it'll almost always be a QuerySet).
I've found myself unsatisfied with Django's ability to render JSON data. If I use built in serializes then database foreign key relationships are not included in the data (only the keys). Also, it seems to be impossible to include custom data in the json feed that isn't part of the model being serialized.
As a test I implemented a template that rendered some JSON for the resultset of a particular model. I was able to include/exclude whatever parts of the model I wanted and was able to include custom data as well.
The test seemed to work well and wasn't slower than the recommended serialization methods.
Are there any pitfalls to this using this method of serialization?
While it's hard to say definitively whether this method has any pitfalls, it's the method we use in production as you control everything that is serialized, even if the underlying model is changed. We've been running a high traffic application in for almost two years using this method.
Hope this helps.
One problem might be escaping metacharacters like ". Django's template system automatically escapes dangerous characters, but it's set up to do that for HTML. You should look up exactly what the template escaping does, and compare that to what's dangerous in JSON. Otherwise, you could cause XSS problems.
You could think about constructing a data structure of dicts and lists, and then running a JSON serializer on that, rather than directly on your database model.
I don't understand why you see the choice as being either 'use Django serializers' or 'write JSON in templates'. The middle way, which to my mind is much more robust and fits your use case well, is to build up your data as Python lists/dictionaries and then simply use simplejson.dumps() to convert it to a JSON string.
We use this method to get custom JSON format consumed by datatables.net
It was the easiest method we find to accomplish this task and it looks very fine with no problems so far.
You can find details here: http://datatables.net/development/server-side/django
So far, generating JSON from templates, we've run into the need to escape newlines. Looking at doing simplejson.dumps() next.