Bulk insertion in django rest framework - django

When I am trying to create multiple instances in a single request it shows error if one instance is not correct in the batch. But how can I write my create method to insert correct instances in the batch. That means only correct instances will insert in the db and also show the errors message for the wrong instances.
[
{
"name": "oil",
"unit_price": 200
},
{
"name": "meat",
"unit_type": "raw",
"unit_price": 1000
}
"name": "salt",
"unit_type": "raw",
"unit_price": -100
}
]
I want to insert first two instances will be insert in the db and for last one it will throws an error like this.
"errors":
[
{
"unit_price": [
"Enter positive number."
]
}
]
Here is my serializer
class ProductSerializer(serializers.ModelSerializer):
def validate_unit_price(self, value):
if (value) > 0:
return value
raise serializers.ValidationError("Enter positive number.")
class Meta:
model = Product
fields = [ 'name', 'unit_type', 'unit_price']
Also my views function is
#api_view(['POST'])
def store(request):
serializer = ProductSerializer(data = request.data,many=True)
if serializer.is_valid():
serializer.save()
return Response({'response_code': '500', 'response': status.HTTP_500_INTERNAL_SERVER_ERROR, 'message': 'Something went wrong', 'data': request.data, 'errors': serializer.errors})

Since you're using one ProductSerializer instance, you won't be able to save if is_valid returns False. If you want to create all valid datasets and return the ones that errored, you might want to consider creating a serializer instance per entry, and keeping track of the data returned. You can build out a list of response values for the user that way, so partial failure is allowed.
My suggestion will not make this a "bulk" operation, but it technically isn't with many=True. many=True just wraps your serializer with a ListSerializer, which just calls create using the wrapped serializer per attribute. So you're performing a save() per instance of data as is.

Related

How to parse a string list query param in DRF serializers?

I'm building a REST API to handle user requests in the following form:
localhost:8000/search/?categories=<category1>,<category2>&parts=<id1>,<id2>
Where , is supposed to be the delimiter for the parser.
My view processes the request and passes the query params to the serializer, but I just cannot get the raw strings parsed to a list of strings.
My attempt so far:
class StringListField(serializers.ListField):
child = serializers.CharField()
class LookupPartsSerializer(serializers.Serializer):
categories = StringListField()
parts = StringListField()
class LookupParts(APIView):
def get(self, request, format=None):
serializer = LookupPartsSerializer(data=request.query_params)
serializer.is_valid()
return Response(serializer.validated_data)
My desired output is like:
{
"categories": [
"<category1>",
"<category2>"
],
"parts": [
"<id1>",
"<id2>"
]
}
But now I'm getting:
{
"categories": [
"<category1>,<category2>"
],
"parts": [
"<id1>,<id2>"
]
}
So basically I'm looking for an option to pass a delimiter argument to the StringListField or add some custom parsing method.
NOTE:
I'm aware, that if I change the query pattern from ?categories=<category1>,<category2>... to ?categories=<category1>&categories=<category2>... then I'd get the desired results, but I'd like to stick to my original approach.
I think the best way is to implement a customer to_representation():
def to_representation(self, instance):
data = super(StringListField, self).to_representation(instance)
data['parts'] = data['parts'].split(',') if isinstance(data['parts'], str) else data['parts']
data['categories'] = data['categories'].split(',') if isinstance(data['categories'], str) else data['categories']
return data
Hope it helps!

Using views to change DRF response

I would like to alter the response from an API.
However, it does not alter the result properly. I get a KeyError: 'game'.
I am not sure why, as my API response (via URL) seems to have the value game in it. I may be getting confused with the JSON response, and the python object.
I have a sample of the API response below
results from API
{
"pk": 995,
"game": [
{
"name": "Finance",
"gamelevel": 3
},
{
"name": "Data",
"gamelevel": 1
}
]
},
views.py
class TagList(viewsets.ModelViewSet):
queryset = Task.objects.filter(game__isnull=False).all()
serializer_class = TagSortSerializer
def get_queryset(self):
test = self.queryset.values('title', 'game__name')
result = defaultdict(set)
for item in queryset:
parent = {'name': 'NoLevel_1'}
children = []
for game in item['game']:
if game['gamelevel'] == 1:
parent = game
else:
children.append((game['gamelevel'], game['name']))
result[parent['name']].update(children)
result = [
{
'name': parent,
'game_child': [
{'name': name, 'gamelevel': gamelevel} for gamelevel, name in games
],
'gamelevel': 1,
} for parent, games in result.items()
]
return result
You're using the values queryset method to get only a selection of fields from the model, and the only fields you've specified are title and tag__name. So you won't get game or any of the other keys you've used.
You certainly don't want to use values here in the first place; just do a normal query and access fields via dot lookup rather than dictionary.

django api post without bring json still work?

Because django rest framework did not support bulk create
So I write one
And I found a strange problem
if I POST the api with a json like :
[{'address':'1','name':'2','start':'3'},
{'address':'10','name':'20','start':'30'}]
it works!
But if I kust POST the api without bring json
I still got bulk create success message.
Why would this happen??
Where do I write wrong??
This is my API view
class BulkTestList(APIView):
def post(self, request, format=None):
duplicateList = []
for data in request.data:
message = {}
if not 'address' in data.keys():
message['address'] = [ "This field is required."]
elif not data['address']:
message["address"] = [ "This field may not be blank."]
if not 'name' in data.keys():
message["name"] = [ "This field is required."]
elif not data['name']:
message["name"]= [ "This field may not be blank."]
if not 'star' in data.keys():
message["star"] = [ "This field is required."]
elif not data['star']:
message["star"]= [ "This field may not be blank."]
if message:
return Response(message, status=status.HTTP_400_BAD_REQUEST)
for data in request.data:
address = data['address'].upper()
bulkCreateObjects = Data(address=address, name=data['name'], star=data['star'], datetime=datetime.datetime.now(pytz.utc))
bulkCreateObjects.save()
message = {"bulk create success"}
return Response(data=message, status=status.HTTP_201_CREATED)
Django REST framework doesn't have bulk out of the box but you have a third party app that does.
Your current view just doesn't call a serializer therefore you won't get the validation at any point. See http://www.django-rest-framework.org/tutorial/3-class-based-views/#rewriting-our-api-using-class-based-views
Note that for bulk you'll add a many=True to the serializer so it will be able to deal with list of data.
Your issue that the view returns a 201 "{bulk create success}" is due to the fact that you iteration over request.data does not check if reuqest.data is actually empty. A for loop over an empty list will just skip over the for block. As Linovia mentions, you need to add some validation to your view.

POST new nested object

I'm trying to implement this scheme:
http://127.0.0.1:8000/api/get_work/
{
"type": "dns",
"source_alerts": [
{
"source": "alehop.com",
"alerts": [
{
"dns_server": "8.8.4.4",
"ip_addr": "134.211.190.5",
},
{
"dns_server": "7.7.2.2",
"ip_addr": "224.110.70.3",
}
]
}
]
}
And then be able to GET all alerts nested into a source:
** The source will be unique
http://127.0.0.1:8000/api/set_work/dns/alehop.com/
"alerts": [
{
"dns_server": "8.8.4.4",
"ip_addr": "134.211.190.5",
},
{
"dns_server": "7.7.2.2",
"ip_addr": "224.110.70.3",
}
And POST a single alert into that source:
{
"dns_server": "7.7.2.2",
"ip_addr": "224.110.70.3",
}
My question is: is possible to implement a list/create viewset of a route with parameters?
router.register(r'set_work/(?P<type>.+)/(?P<source>.+)', views.SetWorkViewSet)
In that case, how can I use that parameters in the viewset in order to filter the queryset?
Thank you in advance. Any other approaches will be very welcome, I'm very new to python/django.
Sure you can!
Django REST Framework (DRF) for model viewsets (which I assume you are using) implements standard get_object and get_queryset methods. Since you added additional parameters to the url regex, you can reference them via self.kwargs inside the viewset:
def get_queryset(self):
qs = super(...).get_queryset()
return qs.filter(type=self.kwargs['type'], source=self.kwargs['source'])
This will do the filtering which will make list work. As for the create, you will probably need to adjust the serializer in order to use the values from the url kwargs. There are a couple of ways to do that:
add the url kwargs to the data being passed to the serializer when instantiating it
class MyViewSet(ViewSet):
def get_serializer(self, *args, **kwargs):
# add the url kwargs to request data so that serializer will see it
self.request.data.update(self.kwargs)
return super(...).get_serializer(*args, **kwargs)
This technically should work however feels a bit hackish to me so I would not recommend this approach.
adjust the validation logic within the serializer to add the url parameters
class MySerializer(ModelSerializer):
class Meta(object):
model = MyModel
def to_internal_value(self, data):
if hasattr(self, 'internal_data') and 'view' in self.context:
data.update(self.context['view'].kwargs)
return super(...).to_internal_value(data)
I personally feel this approach is cleaner even though it leaks a little of information about the viewset to the serializer.
Please note the code is not tested so you will need to do some testing but it should get you started.

Right way to pass Django objects to ExtJS

Django has a built in serialization functionality which allows you to serialize any query result set into JSON:
json_serializer = serializers.get_serializer("json")()
json_serializer.serialize(queryset, ensure_ascii=False)
This produces output such as:
[
{
"pk": 1,
"model": "app_name.model_name",
"fields": {
"field_name": "value",
(...)
}
}
]
If you want to pass this JSON object over to an ExtJS driven application you run into a problem, because ExtJS expects its JSON to be formatted differently:
{
"total": 100,
"success": true,
"objects": [
{
"id": 1,
"field_name": "value",
(...)
}
]
}
There are 2 main differences: the additional meta-data (success, total) and the IDs of the objects which are provided together with other fields in Ext, but not in Django.
There are many possible ways to make one or the other format conform with the second, but what do you consider to be the best way to make this work? Is it a special serializer on the Django side, or maybe a special reader on the ExtJS side...
What do you think is the best way to solve this problem?
Better idea: use a custom serialiser.
settings.py
SERIALIZATION_MODULES = {
'extjson': 'extjs.serialiser.json'
}
extjs\serialiser\json.py
from django.core.serialisers.json import Serialiser
class ExtJSONSerialiser(Serializer)
"""
Serializes a QuerySet to basic Python objects.
"""
def end_object(self, obj):
self._current.update({"id": smart_unicode(obj._get_pk_val(), strings_only=True),})
self.objects.append(self._current)
self._current = None
def getvalue(self):
return {
'total': len(self.objects),
'success': True,
'objects': self.objects,
}
yourcode.py
json_serializer = serializers.get_serializer("extjson")()
json_serializer.serialize(queryset, ensure_ascii=False)
I found a solution which is able to serialize objects, which contain QuerySet as attributes. It's comes from traddicts.org blog, but you can now find it on GitHub:
https://github.com/datamafia/django-query-json-serializer/blob/master/JSONSerializer.py
I further modified the code to work recursively, so in effect I can do the following:
users = User.objects.all()
response = {}
response['success'] = True
response['users'] = users
json_serialize(response)
json_serialize(response, serialize_related=True)
json_serialize(response, serialize_related=True, ignored=['users.groups'])
json_serialize(response, serialize_related=True, ignored=['users.groups.permissions'])
I like your answer Thomas, but I needed something which would be even more flexible.