I'm trying to pass a few arguments form my view to my additional function, but I get error "Object of type 'Request' is not JSON serializable", I don't why is occured with me, cause I didn't pass request object to Response.
#api_view(['POST', ])
#permission_classes([IsAuthenticated])
def users_upload(request):
if request.user.type != 't':
logger.warning(f'Users upload ERROR: -{request.user.username}- not a teacher')
return Response({'message': 'Not a teacher'}, status=403)
try:
user_file = request.FILES['data']
file_format = user_file.name.split('.')[-1]
if file_format not in ['csv', 'xls', 'xlsx']:
logging.error('User upload ERROR: incorrect file format')
return Response({'message': 'Incorrect file format'}, status=400)
auto_file = AutoFile.objects.create(
description='Temp file for user upload',
file=user_file
)
upload_users_from_file.delay(request, auto_file.pk, file_format)
return Response({'message': 'Task created'}, status=200)
except Exception as e:
logging.error(f'User upload ERROR: unable to evaluate file format: {e}')
return Response({'message': 'Error during file check'}, status=400)
What happened?
I can't show function "upload_users_from_file" because it distributes request object to large functions chain, anyway you can see by logger that ERROR occured actually in user_upload function.
When delay() Celery must to serialize all the delay arguments to store in the task database. When Celery-worker gets this task, it deserializes arguments.
You try to pass the Request object as an argument to delay. Celery doesn't known how to serialize this type of objects.
Some advises:
Try to pass to tasks only objects that serializable by default - lists, dicts, strs, ints, etc.
If you need to pass a lot of arguments, combine them into dicts or tuples.
You may want to make your object serializable. Please pay attention to these topics:
How to make a class serializable
How to register a custom JSON encoder to Celery
Related
New to development of API. Any help or article would really help.
my views.py
#api_view(["POST"])
def retrieve_base64_as_pdf(request):
try:
#base_64_input = request.POST.get('ImageBase64String')
body_unicode = request.body.decode('utf-8')
print(len(body_unicode))
body = json.loads(body_unicode)
print(len(body))
base_64_input = body['ImageBase64String']
doc_ref_id = body['DocRefID']
#base_64_input = json.loads(request.body["ImageBase64String"])
pdf = base64.b64decode(base_64_input)
#build PDF
with open(os.path.expanduser('life/lpo-test.pdf'), 'wb') as fout:
fout.write(pdf)
responses = single_page_extractor(file_names='life/lpo-test.pdf',client_doc_ref_id=doc_ref_id)
# return Response(responses, safe=False)
return Response(responses, status=status.HTTP_200_OK)
# responses = json.responses.replace("\'", '')
# return Response(responses, status=status.HTTP_200_OK, content_type='json')
except ValueError as e:
os.remove('life/lpo-test.pdf')
return Response(e, status=status.HTTP_400_BAD_REQUEST)
my process,
1. consume base 64 run my logic and output json.
It works 99% of the time but throws bad request at times.
1. The json input is the same for all inputs.
error message:
exception: Exception Value: Object of type ParserError is not JSON serializable
The only error:
ParserError('Error tokenizing data. C error: Expected 1 fields in line 3, saw 12\n')
Looks like Response class if from django-rest-framework and it will try to jsonify any input data you gave it.
On the last line you are trying to serialize error object and that's why you have a problem:
return Response(e, status=status.HTTP_400_BAD_REQUEST)
Try to wrap error into str to make in json-serializable:
return Response({'error': str(e)}, status=status.HTTP_400_BAD_REQUEST)
Also the problem might be in single_page_extractor if it can return error objects.
I have implemented my custom Renderer like this:
from rest_framework.renderers import JSONRenderer
class CustomJSONRenderer(JSONRenderer):
def render(self, data, accepted_media_type=None, renderer_context=None):
//I am hardcoding status and message for now. Which I have to update according to the response.
data = {'data': data, 'message':'ok', 'status':200 }
return super(CustomJSONRenderer, self).render(data, accepted_media_type, renderer_context)
This is working pretty good. Now I want to update status using HTTP status code of response and thus providing a custom message. So how should I achieve this?
Basically I want the response like this:
{"status":200, "data":[actual data comes here.], "message":"ok"}
Well on a different note I found out that we can get the status information. The renderer_context parameter actually contains the following information-
{'view': <ViewSet object at 0x7ff3dcc3fac0>, 'args': (), 'kwargs': {}, 'request': <rest_framework.request.Request object at 0x7ff3dcc37e20>, 'response': <Response status_code=400, "application/json; charset=utf-8">}
This means the renderer_context parameter is a dictionary and can be exploited in order to modify your response. For example -
def render(self, data, accepted_media_type=None, renderer_context=None):
if renderer_context is not None:
print(renderer_context['response'].status_code)
This is not what renderers are made for. You should use a renderer to transform the response into a certain format (json, html, csv, etc) according to the request. By default it will use the Acceptheader, but you could image to pass a querystring parameter to force a different output.
I think what you are trying to do is a custom error exception http://www.django-rest-framework.org/api-guide/exceptions/#custom-exception-handling
Hope this helps
I'm writing unit tests for my django api written with django-rest-framework, and I'm encountering seemingly inconsistent response data from calls that generate 400_BAD_REQUEST.
When I make a request that fails because it references an invalid primary key, repr(response.data) is a string that I can check for the substring "Invalid pk". But when I make a request that fails because it's trying to create a non-unique object, repr(response.data) contains {'name': [<django.utils.functional.__proxy__ object at 0x7f3ccdcb26d0>]} instead of the expected {'name': ['This field must be unique.']}. When I make an actual POST call to the real server, I get the expected 400 response with {'name': ['This field must be unique.']}.
Here's a code sample:
class GroupViewTests(APITestCase):
def test_post_existing_group(self):
"""
Use POST to update a group at /groups
"""
# create a group
# self.group_data returns a valid group dict
data = self.group_data(1)
url = reverse('group-list')
response = self.client.post(url, data, format='json')
self.assertEqual(response.status_code, status.HTTP_201_CREATED)
# create the same group again
response = self.client.post(url, data, format='json')
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
# this fails
self.assertIn('must be unique', repr(response.data))
def test_create_group_with_nonexistent_user(self):
"""
Create a group with an invalid userid in it.
"""
url = reverse('group-list')
data = self.group_data(5)
data.update({'users': ['testnonexistentuserid']})
response = self.client.post(url, data, format='json')
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
# this succeeds
self.assertIn('Invalid pk', repr(response.data))
I'm using only the default middleware.
From what I've found online, the __proxy__ object is a representation of a lazy translation, which can be rendered as a string by calling unicode(obj) or obj.__unicode__. And indeed, response.data['name'][0].__unicode__ does return the expected value, but calling unicode(response.data) instead of repr(response.data) still returns the object identified by memory address.
Can anyone explain what's going on here, and how to fix it? I can work around the issue for now, but I'd like to know the "real way" to fix the problem. Is it possibly a bug in django or django-rest-framework?
Use response.content instead of response.data.
response.content contains the json output just like your client will receive it. It's also useful if you have custom renderer that change the response because response.data is not rendered, but response.content yes.
E.g :
import json
# we have to parse the output since it no more a dict
data = json.loads(response.content)
# if `response.content` is a byte string, decode it
data = json.loads(response.content.decode('utf-8'))
According to https://docs.djangoproject.com/en/dev/ref/forms/validation/
# Good
ValidationError(
_('Invalid value: %(value)s'),
params={'value': '42'},
)
# Bad
ValidationError(_('Invalid value: %s') % value)
The docs doesnt really explain why it is bad / good. Can someone give a concrete example?
Furthermore, when I inspect form.errors, I get something like 'Invalid: %(value)s'. How do I get the params from the Validation error and interpolate them into the error msg?
Edited
So is this considered good?
ValidationError(
_('Invalid value: %(value)s') % {'value': '42'},
)
I think the real question is: why pass the variables separately via the params argument? Why not interpolate directly into the error msg (ignore named or positional interpolation for now)???
Edited
Ok, From the source # https://github.com/django/django/blob/stable/1.5.x/django/forms/forms.py
I don't think there is any way to retrieve ValidationError's params since the Form does not even save the ValidationError object itself. See code below.
class ValidationError(Exception):
"""An error while validating data."""
def __init__(self, message, code=None, params=None):
import operator
from django.utils.encoding import force_text
"""
ValidationError can be passed any object that can be printed (usually
a string), a list of objects or a dictionary.
"""
if isinstance(message, dict):
self.message_dict = message
# Reduce each list of messages into a single list.
message = reduce(operator.add, message.values())
if isinstance(message, list):
self.messages = [force_text(msg) for msg in message]
else:
self.code = code
self.params = params
message = force_text(message)
self.messages = [message]
class Form:
....
def _clean_fields(...):
....
except ValidationError as e:
self._errors[name] = self.error_class(e.messages) # Save messages ONLY
if name in self.cleaned_data:
del self.cleaned_data[name]
If you have multiple parameters, they might appear in a different order when you translate the error message.
Named arguments allow you to change the order in which the arguments appear, without changing params. With a tuple of arguments, the order is fixed.
Note that you are linking to the development version of the Django docs. The validation error is not interpolating the parameters because you are using Django 1.5 or earlier. If you try your code in the 1.6 beta, then the parameters are interpolated into the error message.
ValidationError is caught by the form validation routine and though it can just show a message, it's better to save the possibility of getting params of error; eg. field name, value that caused error and so on. It's stated just before the example you've provided.
In order to make error messages flexible and easy to override
What I meant exactly was, I would like to have JSON response when I modify the obj_create(). I've implemented the UserSignUpResource(ModelResource) and inside the obj_create(), I did some validation and when it fails, I raise BadRequest(). However, this doesn't throw out JSON. It throws out String instead.
Any idea if I can make it throw out {'error': 184, 'message': 'This username already exists'} format? Or am I not suppose to modify obj_create()? Or what should I do instead?
Many help, thanks.
Cheers,
Mickey
easy that, i've just created a little helper method in tastypies http module:
import json
#tastypies HttpResponse classes here...
def create_json_response(data, http_response_class):
return http_response_class(content=json.dumps(data), content_type="application/json; charset=utf-8")
then you can simply say:
from tastypie.http import HttpNotFound, create_json_response
#choose HttpNotFound, HttpCreated whatever...
raise ImmediateHttpResponse(create_json_response({"error":"resource not found"}, HttpNotFound))
You should use the error_response method from the resource.
Something like:
def obj_create(self, bundle, **kwargs):
# Code that finds Some error
my_errors = {"error": ["Some error"]}
raise ImmediateHttpResponse(response=self.error_response(bundle.request, my_errors))
Usually you would call super and the errors should arise from the tastypie validation process. The exception would be automatically thrown (with the errors dictionary being saved on the bundle.errors property).