Right way to pass Django objects to ExtJS - django

Django has a built in serialization functionality which allows you to serialize any query result set into JSON:
json_serializer = serializers.get_serializer("json")()
json_serializer.serialize(queryset, ensure_ascii=False)
This produces output such as:
[
{
"pk": 1,
"model": "app_name.model_name",
"fields": {
"field_name": "value",
(...)
}
}
]
If you want to pass this JSON object over to an ExtJS driven application you run into a problem, because ExtJS expects its JSON to be formatted differently:
{
"total": 100,
"success": true,
"objects": [
{
"id": 1,
"field_name": "value",
(...)
}
]
}
There are 2 main differences: the additional meta-data (success, total) and the IDs of the objects which are provided together with other fields in Ext, but not in Django.
There are many possible ways to make one or the other format conform with the second, but what do you consider to be the best way to make this work? Is it a special serializer on the Django side, or maybe a special reader on the ExtJS side...
What do you think is the best way to solve this problem?

Better idea: use a custom serialiser.
settings.py
SERIALIZATION_MODULES = {
'extjson': 'extjs.serialiser.json'
}
extjs\serialiser\json.py
from django.core.serialisers.json import Serialiser
class ExtJSONSerialiser(Serializer)
"""
Serializes a QuerySet to basic Python objects.
"""
def end_object(self, obj):
self._current.update({"id": smart_unicode(obj._get_pk_val(), strings_only=True),})
self.objects.append(self._current)
self._current = None
def getvalue(self):
return {
'total': len(self.objects),
'success': True,
'objects': self.objects,
}
yourcode.py
json_serializer = serializers.get_serializer("extjson")()
json_serializer.serialize(queryset, ensure_ascii=False)

I found a solution which is able to serialize objects, which contain QuerySet as attributes. It's comes from traddicts.org blog, but you can now find it on GitHub:
https://github.com/datamafia/django-query-json-serializer/blob/master/JSONSerializer.py
I further modified the code to work recursively, so in effect I can do the following:
users = User.objects.all()
response = {}
response['success'] = True
response['users'] = users
json_serialize(response)
json_serialize(response, serialize_related=True)
json_serialize(response, serialize_related=True, ignored=['users.groups'])
json_serialize(response, serialize_related=True, ignored=['users.groups.permissions'])
I like your answer Thomas, but I needed something which would be even more flexible.

Related

Bulk insertion in django rest framework

When I am trying to create multiple instances in a single request it shows error if one instance is not correct in the batch. But how can I write my create method to insert correct instances in the batch. That means only correct instances will insert in the db and also show the errors message for the wrong instances.
[
{
"name": "oil",
"unit_price": 200
},
{
"name": "meat",
"unit_type": "raw",
"unit_price": 1000
}
"name": "salt",
"unit_type": "raw",
"unit_price": -100
}
]
I want to insert first two instances will be insert in the db and for last one it will throws an error like this.
"errors":
[
{
"unit_price": [
"Enter positive number."
]
}
]
Here is my serializer
class ProductSerializer(serializers.ModelSerializer):
def validate_unit_price(self, value):
if (value) > 0:
return value
raise serializers.ValidationError("Enter positive number.")
class Meta:
model = Product
fields = [ 'name', 'unit_type', 'unit_price']
Also my views function is
#api_view(['POST'])
def store(request):
serializer = ProductSerializer(data = request.data,many=True)
if serializer.is_valid():
serializer.save()
return Response({'response_code': '500', 'response': status.HTTP_500_INTERNAL_SERVER_ERROR, 'message': 'Something went wrong', 'data': request.data, 'errors': serializer.errors})
Since you're using one ProductSerializer instance, you won't be able to save if is_valid returns False. If you want to create all valid datasets and return the ones that errored, you might want to consider creating a serializer instance per entry, and keeping track of the data returned. You can build out a list of response values for the user that way, so partial failure is allowed.
My suggestion will not make this a "bulk" operation, but it technically isn't with many=True. many=True just wraps your serializer with a ListSerializer, which just calls create using the wrapped serializer per attribute. So you're performing a save() per instance of data as is.

How to parse a string list query param in DRF serializers?

I'm building a REST API to handle user requests in the following form:
localhost:8000/search/?categories=<category1>,<category2>&parts=<id1>,<id2>
Where , is supposed to be the delimiter for the parser.
My view processes the request and passes the query params to the serializer, but I just cannot get the raw strings parsed to a list of strings.
My attempt so far:
class StringListField(serializers.ListField):
child = serializers.CharField()
class LookupPartsSerializer(serializers.Serializer):
categories = StringListField()
parts = StringListField()
class LookupParts(APIView):
def get(self, request, format=None):
serializer = LookupPartsSerializer(data=request.query_params)
serializer.is_valid()
return Response(serializer.validated_data)
My desired output is like:
{
"categories": [
"<category1>",
"<category2>"
],
"parts": [
"<id1>",
"<id2>"
]
}
But now I'm getting:
{
"categories": [
"<category1>,<category2>"
],
"parts": [
"<id1>,<id2>"
]
}
So basically I'm looking for an option to pass a delimiter argument to the StringListField or add some custom parsing method.
NOTE:
I'm aware, that if I change the query pattern from ?categories=<category1>,<category2>... to ?categories=<category1>&categories=<category2>... then I'd get the desired results, but I'd like to stick to my original approach.
I think the best way is to implement a customer to_representation():
def to_representation(self, instance):
data = super(StringListField, self).to_representation(instance)
data['parts'] = data['parts'].split(',') if isinstance(data['parts'], str) else data['parts']
data['categories'] = data['categories'].split(',') if isinstance(data['categories'], str) else data['categories']
return data
Hope it helps!

Using views to change DRF response

I would like to alter the response from an API.
However, it does not alter the result properly. I get a KeyError: 'game'.
I am not sure why, as my API response (via URL) seems to have the value game in it. I may be getting confused with the JSON response, and the python object.
I have a sample of the API response below
results from API
{
"pk": 995,
"game": [
{
"name": "Finance",
"gamelevel": 3
},
{
"name": "Data",
"gamelevel": 1
}
]
},
views.py
class TagList(viewsets.ModelViewSet):
queryset = Task.objects.filter(game__isnull=False).all()
serializer_class = TagSortSerializer
def get_queryset(self):
test = self.queryset.values('title', 'game__name')
result = defaultdict(set)
for item in queryset:
parent = {'name': 'NoLevel_1'}
children = []
for game in item['game']:
if game['gamelevel'] == 1:
parent = game
else:
children.append((game['gamelevel'], game['name']))
result[parent['name']].update(children)
result = [
{
'name': parent,
'game_child': [
{'name': name, 'gamelevel': gamelevel} for gamelevel, name in games
],
'gamelevel': 1,
} for parent, games in result.items()
]
return result
You're using the values queryset method to get only a selection of fields from the model, and the only fields you've specified are title and tag__name. So you won't get game or any of the other keys you've used.
You certainly don't want to use values here in the first place; just do a normal query and access fields via dot lookup rather than dictionary.

How to integrate Haystack with Django Rest Framework for making GET REST API for searching?

model.py
class Item(models.Model):
name=models.CharField(max_length=50)
company=models.CharField(max_length=100)
search_indexes.py
class ItemIndex(indexes.SearchIndex, indexes.Indexable):
text = indexes.CharField(document=True, use_template=True)
name=indexes.CharField(model_attr='name')
company=indexes.CharField(model_attr='company')
def get_model(self):
return Item
def index_queryset(self, using=None):
return self.get_model().objects.all()
serializer.py
class ItemSearchSerializer(serializers.Serializer):
text = serializers.CharField()
name=serializers.CharField()
company=serializers.CharField()
views.py
class ItemSearchViewSet(mixins.ListModelMixin, viewsets.GenericViewSet):
serializer_class = ItemSearchSerializer
permission_classes = (permissions.IsAuthenticatedOrReadOnly,)
def get_queryset(self):
request = self.request
queryset = EmptySearchQuerySet()
if request.GET.get('q', ''):
query = request.GET.get('q', '')
queryset =SearchQuerySet().filter(content=query);
return queryset
And in url.py I added :
router.register(r'searchquery', views.ItemSearchViewSet, base_name='searchquery')
Now on making GET request from postman like :
http://127.0.0.1:8000/searchquery/?q=app, I am getting the response as desired as show below.
{
"count": 2,
"next": null,
"previous": null,
"results": [
{
"text": "apple\ndjflkj",
"id": 14,
"name": "apple",
"mrp": "45.000000",
"company": "djflkj",
"imageid": "jfhi",
"edible": false,
"discount": "0.000000",
"deliverable": true,
"seller_uid": "ljhkh",
"category": "ldjhgjfdk"
},
{
"text": "app\nhuhiu",
"id": 16,
"name": "app",
"mrp": "78.000000",
"company": "huhiu",
"imageid": "iyiugiy",
"edible": false,
"discount": "45.000000",
"deliverable": true,
"seller_uid": "hjh",
"category": "hhl"
}
]
}
But the reponse time is very slow it takes around 2700 ms everytime ,
and I want to make it fast. As response of elastic search is much fast
but I don't know what I am doing wrong. Not sure but may be due to
these reasons I am getting this delay : 1) Haystack is made for
django, so on integrating it with django rest framework , it may be
getting slow. 2) I am using free Bonsai Elastic search heroku add on
and it has just 125 mb memory.
This is how I am connecting to Bonsai elastic search (setting.py)
ES_URL = urlparse('https://******#pine-1455731.us-east1.bonsaisearch.net')
print ES_URL
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
'URL': ES_URL.scheme + '://' + ES_URL.hostname + ':443',
'INDEX_NAME': 'haystack',
},
}
if ES_URL.username:
HAYSTACK_CONNECTIONS['default']['KWARGS'] = {"http_auth": ES_URL.username + ':' + ES_URL.password}
Any help will be appreciated. I am new to elastic search. I want to do elastic search to search products by name for my android application.
I even don't know whether this is the correct approach to do searching. I thought I would enter name of product I want to search and then i will send a GET request and get all the products which are related.
I did Python Profile please look it here: gist
If any one could suggest me any other way of achieving this I will appreciate your help.
The search response is slow because of this code:
def index_queryset(self, using=None):
return self.get_model().objects.all()
index_queryset is supposed to return query set, you are actually returning all model objects. This method is called for every item which is returned in search.

POST new nested object

I'm trying to implement this scheme:
http://127.0.0.1:8000/api/get_work/
{
"type": "dns",
"source_alerts": [
{
"source": "alehop.com",
"alerts": [
{
"dns_server": "8.8.4.4",
"ip_addr": "134.211.190.5",
},
{
"dns_server": "7.7.2.2",
"ip_addr": "224.110.70.3",
}
]
}
]
}
And then be able to GET all alerts nested into a source:
** The source will be unique
http://127.0.0.1:8000/api/set_work/dns/alehop.com/
"alerts": [
{
"dns_server": "8.8.4.4",
"ip_addr": "134.211.190.5",
},
{
"dns_server": "7.7.2.2",
"ip_addr": "224.110.70.3",
}
And POST a single alert into that source:
{
"dns_server": "7.7.2.2",
"ip_addr": "224.110.70.3",
}
My question is: is possible to implement a list/create viewset of a route with parameters?
router.register(r'set_work/(?P<type>.+)/(?P<source>.+)', views.SetWorkViewSet)
In that case, how can I use that parameters in the viewset in order to filter the queryset?
Thank you in advance. Any other approaches will be very welcome, I'm very new to python/django.
Sure you can!
Django REST Framework (DRF) for model viewsets (which I assume you are using) implements standard get_object and get_queryset methods. Since you added additional parameters to the url regex, you can reference them via self.kwargs inside the viewset:
def get_queryset(self):
qs = super(...).get_queryset()
return qs.filter(type=self.kwargs['type'], source=self.kwargs['source'])
This will do the filtering which will make list work. As for the create, you will probably need to adjust the serializer in order to use the values from the url kwargs. There are a couple of ways to do that:
add the url kwargs to the data being passed to the serializer when instantiating it
class MyViewSet(ViewSet):
def get_serializer(self, *args, **kwargs):
# add the url kwargs to request data so that serializer will see it
self.request.data.update(self.kwargs)
return super(...).get_serializer(*args, **kwargs)
This technically should work however feels a bit hackish to me so I would not recommend this approach.
adjust the validation logic within the serializer to add the url parameters
class MySerializer(ModelSerializer):
class Meta(object):
model = MyModel
def to_internal_value(self, data):
if hasattr(self, 'internal_data') and 'view' in self.context:
data.update(self.context['view'].kwargs)
return super(...).to_internal_value(data)
I personally feel this approach is cleaner even though it leaks a little of information about the viewset to the serializer.
Please note the code is not tested so you will need to do some testing but it should get you started.