JSON Serializing Django Models with simplejson - django

I'd like to use simplejson to serialize a Django model. Django's serializer doesn't support dictionaries... and simplejson doesn't support Django Querysets. This is quite a conundrum.
In the model there's sponsors that have a Foreign Key to sponsor level, I'm trying to group all the sponsors that belong to a certain sponsor level together. Here's the code that generates the list:
from django.shortcuts import get_list_or_404
from special_event.models import Sponsor, SponsorLevel
sponsor_dict = {}
roadie_sponsors = get_list_or_404(Sponsor, level__category = SponsorLevel.ROADIE_CHOICE)
for item in roadie_sponsors:
try:
sponsor_dict[item.level.name].append(item)
except KeyError:
sponsor_dict[item.level.name] = [item]
Here's what sponsor_dict looks like once it's "made"
{
'Fan': [<Sponsor: Fan Sponsor>],
'VIP': [<Sponsor: VIP Sponsor>],
'Groupie': [<Sponsor: Groupie Sponsor>],
'Silver': [<Sponsor: Silver Sponsor>],
'Bronze': [<Sponsor: Another Bronze Sponsor>, <Sponsor: Bronze Sponsor>]
}
I only added one sponsor in each level, except for bronze, just to show how it works. All I want to do is get it "all" into JSON so jQuery can interpret it easily. Can Django's other serializers (like XML or YAML) accomplish this? Can I "extend" the Django JSON Serializer to handle dictionaries or "extend" simplejson to handle Django QuerySet objects?

I would go with extending simplejson. Basically, you want to plug in django's serialization when the JSON encoder encounters a QuerySet. You could use something like:
from json import dumps, loads, JSONEncoder
from django.core.serializers import serialize
from django.db.models.query import QuerySet
from django.utils.functional import curry
class DjangoJSONEncoder(JSONEncoder):
def default(self, obj):
if isinstance(obj, QuerySet):
# `default` must return a python serializable
# structure, the easiest way is to load the JSON
# string produced by `serialize` and return it
return loads(serialize('json', obj))
return JSONEncoder.default(self,obj)
# partial function, we can now use dumps(my_dict) instead
# of dumps(my_dict, cls=DjangoJSONEncoder)
dumps = curry(dumps, cls=DjangoJSONEncoder)
For more info on default method, have a look at simplejson documentation. Put that in a python module, then import dumps and you're good to go. But note that this function will only help you serializing QuerySet instances, not Model instances directly.

A really flexible way to serialize most structures in django is to use the serializer class found here

based on Clement's answer, I did this to get models into JSON as well.
def toJSON(obj):
if isinstance(obj, QuerySet):
return simplejson.dumps(obj, cls=DjangoJSONEncoder)
if isinstance(obj, models.Model):
#do the same as above by making it a queryset first
set_obj = [obj]
set_str = simplejson.dumps(simplejson.loads(serialize('json', set_obj)))
#eliminate brackets in the beginning and the end
str_obj = set_str[1:len(set_str)-2]
return str_obj

Related

Django rest framwork - how to prevent 1062, "Duplicate entry"

at my Django application (DRF only), I'm trying to create a new object where one of the fields is setup like this:
resource_name = models.CharField(verbose_name="Resource Name", blank=False, null=False, max_length=50, unique=True)
If I now try to create an Object with the same resource_name twice, I'm always running into the following exception:
django.db.utils.IntegrityError: (1062, "Duplicate entry
'test_ressource123' for key 'resource_name'")
Is there any good solution that I can apply to all kinds of these situations?
Would be awesome to simply make the API response that the object already exists, kinda strange that this is not already a built-in of DRF. Can I maybe overwrite the def create function call of the serializer ? Any good advice welcome.
Generally, you should develop a serializer for each model to understand how you work with each one. Later you can forget about this "common" solution and look for a bug for a long time.
Nevertheless, I see two approaches to achieve that behavior.
Overriden ModelViewSet
We can redefine the create method of default ModelViewSet and then inherit your ModelViewSets from this class. Like this:
from django.db import IntegrityError
from rest_framework import viewsets
from rest_framework.exceptions import APIException
from rest_framework.status import HTTP_400_BAD_REQUEST
class SupressIntegrityErrorModelViewSet(viewsets.ModelViewSet):
def create(self, request, *args, **kwargs):
try:
return super().create(request, *args, **kwargs)
except IntegrityError as e:
raise APIException(detail=str(e), code=HTTP_400_BAD_REQUEST)
More information here.
DRF custom exception handling
Firstly you need to make a custom_exception_handler method; DRF documentation suggests making it in <application-directory>/utils.py:
from django.http import HttpResponseBadRequest
from rest_framework.views import exception_handler
def custom_exception_handler(exc, context):
if type(exc).__name__ == 'IntegrityError':
return HttpResponseBadRequest(str(e))
return exception_handler(exc, context)
Then you have to set EXCEPTION_HANDLER in your settings.py:
REST_FRAMEWORK = {
'EXCEPTION_HANDLER': 'my_project.my_app.utils.custom_exception_handler'
}
It is better to add UniqueValidator to your serializer.
resource_name = serializers.CharField(max_length=50, validators=[UniqueValidator(queryset=YourModel.objects.all())], requierd=False)

Django rest framework complex return info for a single api

How to return complex information in a single api under Django rest framework?
Assuming I have a model:
class Apple(models.Model):
color
size
shape
With a single api: /api/get-apples, I want to return a json like below:
{"red": {"round":[0.1,0.2,0.3],"spigold":[0.3,0.4,0.5]},
"yellow":{"round":[0.1,0.2,0.4],"spigold":[0.2,0.4,0.5]}}
What will be the best way to achieve this?
Create a serializers.py in your app's folder and add this code into it.
from rest_framework import serializers
class AppleSerializer(serializers.ModelSerializer):
class Meta:
model = Apple
fields = ('color', 'size', 'shape',)
In your views.py:
from rest_framework.generics import ListAPIView
from .serializers import AppleSerializer
class get_apples(ListAPIView):
serializer_class = AppleSerializer
def get_queryset(self):
# Here create your queryset.
queryset = Apple.objects.all()
return queryset
In your urls.py:
url(r'^api/get_apples/', views.get_apples.as_view(), name='get_apples'),
And you are good to go.
Output will be like this.
Let's say you have 2 apples.
{{"color": "red", "size": "blabla", "shape": "round"},{...(another apple json)}}
I'd edit my previous answer but I think it is a good example of using serializers with covering view,urls and serializer parts. Therefore, I didn't want to delete it :)
Here is how to return a complex json structure.
As I mentioned before, as far as I know, we can't do something like that by using rest framework's serializers class because it need comparison and grouping. I'll use rest_framework's api_view structure.
Also, I didn't understand types of size and shape in your model and what is your desired output. Therefore, this might be wrong but you'll get the idea anyway.
from rest_framework.decorators import api_view
from django.http import HttpResponse
import json
#api_view(['GET'])
def get_apples(request):
# define your queryset here as you want.
basket = Apple.objects.all()
data = {} # empty dictionary
for apple in basket:
if data.get(apple.color, None) is not None: # if same color of apple exists in basket
# check if shape exists or not.
if data.get(apple.color).get(apple.shape, None) is not None:
data.get(apple.color).get(apple.shape).append(apple.size)
else:
data.get(apple.color)[apple.shape] = [apple.size]
else:
data[apple.color] = {apple.shape: [apple.size]}
return HttpResponse(json.dumps(data), content_type='application/json; charset=utf-8')
I didn't test this code but probably this will work. Let me know if this works or not!

Django Haystack Results to Django Rest Framework Serializer

I am attempting to do a search using Django Haystack and then upon retrieving results I need to pass these results to my Django Rest Framework serializer.
The Django Rest Framework serializers.ModelSerializer requires that a queryset of objects gets sent for the serializer to be able to serialize these objects along with their database fields.
When I create my API view and use search to get results haystack returns a searchqueryset.
How could I get this searchqueryset into a django queryset without doing something like:
article_queryset = Article.objects.filter(id__in=[i.object for i in searchqueryset])
As you could imagine, sometimes search can return excess of 1000 search results which means that the above would be very inefficient.
Right now the Django rest framework allows me to paginate my returned objects. I am paginating by 30 objects on each page. How would it be possible for me to do the same with my Haystack searchqueryset?
Any advice or ideas on how to use Haystack along with Django Rest Framework would be great. Examples of how others have done a similar thing would be cool too :)
You could use Haystack's SearchView (instead of a DRF view) which exposes the page object for pagination. Then you can just pass that to your serializer.
E.G. Awhile back I was working on something that required the json object of the rendered results on the current HTML page, so I wrote a filter tag to do this (we were using FacetedSearchView for the UI, and DRF for a more general purpose RESTful API). Called like so in the template:
var articlesJson = {{ page.object_list|article_jsonify }}
Actual filter tag:
import collections
from django.template import Library
from django.utils.safestring import mark_safe
from rest_framework.renderers import JSONRenderer
register = Library()
def article_jsonify(object):
if isinstance(object, collections.Iterable):
many = True
ids = [obj.object.id for obj in object]
else:
many = False
ids = [object.id]
articles = Article.objects.filter(pk__in=ids)
serializer = ArticleSerializer(articles, many=many)
content = JSONRenderer().render(serializer.data)
return mark_safe(content)
register.filter('article_jsonify', article_jsonify)
You could also write a view inherited from generics.ListAPIView and override the get_queryset method, where you would pass the request's query parameter to a SearchQuerySet, then output it using the Serializer. More info here:
http://www.django-rest-framework.org/api-guide/filtering
Of course you might not be able to use the ModelSerializer this way, unless you do something like you mentioned. However DRF has an example on using the paginator on a queryset like so:
http://www.django-rest-framework.org/api-guide/pagination#paginating-querysets
UPDATE
I ended up eventually using a DRF view that uses a Haystack SearchQuerySet as the queryset, and then passing it to a Paginator. Here is a simplified example (I'm sure it can be streamlined some), but I hope it helps someone gets started.
class ArticleList(ListCreateAPIView):
"""
List, Create files
"""
model = Article
def get(self, request, *args, **kwargs):
# simplified filtering of an SQS
q = request.get('q', '')
sqs = SearchQuerySet().filter(content=Clean(q))
paginator = Paginator(sqs, 20)
page = request.QUERY_PARAMS.get('page')
try:
articles = paginator.page(page)
except PageNotAnInteger:
# If page is not an integer, deliver first page
articles = paginator.page(1)
except PageNotAnInteger:
# If page is out of range, deliver last page
articles = paginator.page(paginator.num_pages)
serializer_context = {'request': request}
serializer = PaginatedArticleSerializer(articles, context=serializer_context)
return Response(serializer.data)
class ArticleSerializer(serializers.ModelSerializer):
"""
Base Article Serializer
"""
class Meta:
model = Article
class PaginatedArticleSerializer(pagination.PaginationSerializer):
"""
Serializes page objects of article querysets.
"""
start_index = serializers.SerializerMethodField('get_start_index')
end_index = serializers.SerializerMethodField('get_end_index')
num_pages = serializers.Field(source='paginator.num_pages')
class Meta:
object_serializer_class = ArticleSerializer
def get_start_index(self, page):
return page.start_index()
def get_end_index(self, page):
return page.end_index()
def get_curr_page(self, page):
return page.number

Django Rest Framework bulk updates inserting instead of updating

I'm trying to build out a bulk update view for a specific model using Django Rest Framework. In the short term, it only needs to update one field (toggling an invite from submitted=False to submitted=True), but I'd like it to be able to provide more functionality in the future. Whenever I test the view, however, a new object is being created instead of the current one being modified.
I feel like this must be a simple mistake on my part, but I can't figure out what's going on. The serializer object appears to be ignoring the value for "id" passed in through JSON, which may be contributing to the issue. Current code is:
class InviteBulkUpdateView(generics.UpdateAPIView):
def get_queryset(self):
order = self.kwargs['order']
invite = get_objects_for_user(self.request.user, 'sourcing.view_invite')
return invite.filter(order=order)
serializer_class = InviteInputSerializer
def put(self, request, *args, **kwargs):
data = request.DATA
serializer = InviteInputSerializer(data=data, many=True)
if serializer.is_valid():
serializer.save()
return Response(status=status.HTTP_200_OK)
else:
return Response(status=status.HTTP_400_BAD_REQUEST)
class InviteInputSerializer(serializers.ModelSerializer):
class Meta:
model = Invite
fields = ('id', 'order', 'team', 'submitted')
Can anybody shed some light onto what I might be doing wrong?
Just in case somebody is looking for a library to handle this, I wrote a Django-REST-Framework-bulk which allows to do that in a couple of lines (the example only does bulk update but the library also allows bulk create and delete):
from rest_framework_bulk import ListCreateBulkUpdateAPIView
class FooView(ListCreateBulkUpdateAPIView):
model = FooModel
You're not passing object instances to your serializer. (Thus it will create new instances rather than update.) See the docs on dealing with multiple objects in serializers where you'll see your QuerySet passed in.
Django has update method to handle that. You may want to read full info from django documentation.
Here is a sample code where you can use to update given field for multiple records:
from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import status
from rest_framework.exceptions import APIException
class Room_Update_ViewSet(APIView):
def put(self, request,*args, **kwargs):
hotel_id = self.kwargs.get('hotel_id')
room_ids = self.request.query_params.get('room_ids')
room_ids = list(map(int, room_ids.split(',')))
try:
Room.objects.filter(hotel_id=hotel_id,id__in=room_ids).update(booked_status=False)
instances = Room.objects.filter(hotel_id=hotel_id,id__in=room_ids)
serializer = RoomSerializer(instance=instances, many=True)
return Response(serializer.data,status=status.HTTP_200_OK)
except Exception as e:
print("Error udating rooms-->",e)
raise APIException

Override Django Object Serializer to get rid of specified model

I need to convert a Django Queryset Object into a Json string. The built in Django Serialization library works great. Although it specifies the name of the Model from where it was created. Since I don't need this, how do I get rid of it? What else do I need to override to be able to use the overridden end_object method below?
class Serializer(PythonSerializer):
def end_object(self, obj):
self.objects.append({
"model" : smart_unicode(obj._meta), # <-- I want to remove this
"pk" : smart_unicode(obj._get_pk_val(), strings_only=True),
"fields" : fields
})
self._current = None
Sorry I had totally forgot about this question. This is how I ended up solving it (with thanks to FunkyBob on #django):
from django.core.serializers.python import Serializer
class MySerialiser(Serializer):
def end_object( self, obj ):
self._current['id'] = obj._get_pk_val()
self.objects.append( self._current )
# views.py
serializer = MySerialiser()
data = serializer.serialize(some_qs)
Here's a serializer that removes all metadata (pk, models) from the serialized output, and moves the fields up to the top-level of each object. Tested in django 1.5
from django.core.serializers import json
class CleanSerializer(json.Serializer):
def get_dump_object(self, obj):
return self._current
Serializer = CleanSerializer
Edit:
In migrating my app to an older version of django (precipitated by a change in hosting provider), the above serializer stopped working. Below is a serializer that handles the referenced versions:
import logging
import django
from django.core.serializers import json
from django.utils.encoding import smart_unicode
class CleanSerializer148(json.Serializer):
def end_object(self, obj):
current = self._current
current.update({'pk': smart_unicode(obj._get_pk_val(),
strings_only=True)})
self.objects.append(current)
self._current = None
class CleanSerializer151(json.Serializer):
def get_dump_object(self, obj):
self._current['pk'] = obj.pk
return self._current
if django.get_version() == '1.4.8':
CleanSerializer = CleanSerializer148
else:
CleanSerializer = CleanSerializer151
Serializer = CleanSerializer
Override JSON serializer class:
from django.core.serializers.json import Serializer, DjangoJSONEncoder
from django.utils import simplejson
class MySerializer(Serializer):
"""
Convert QuerySets to JSONS, overrided to remove "model" from JSON
"""
def end_serialization(self):
# little hack
cleaned_objects = []
for obj in self.objects:
del obj['model']
cleaned_objects.append(obj)
simplejson.dump(cleaned_objects, self.stream, cls=DjangoJSONEncoder, **self.options)
In the view:
JSONSerializer = MySerializer
jS = JSONSerializer()