Django Rest Framework api test upload text file application/x-www-form-urlencoded - django

I'm trying to test my Django REST API for file uploading.
The catch is that my server tries to validate the file so that the file has a recognised file type. Currently only text-based filetypes are allowed, like: docs, pdf, txt.
I've tried using a temporary file and now I just tried to read a file from the disk then gave up.
Whenever I use a temporary file the server responds with:
{
"cover_letter": [
"The submitted data was not a file. Check the encoding type on the form."
],
"manuscript": [
"The submitted data was not a file. Check the encoding type on the form."
]
}
This is my test:
def test_user_can_submit_a_paper(self):
"""
Ensure that an user is able to upload a paper.
This test is not working.. yet
"""
tmp_file = open("__init__.py", "w")
data = {
"title": "paper",
"authors": "me",
"description": "ma detailed description",
"manuscript": base64.b64encode(tmp_file.read()).decode(),
"cover_letter": base64.b64encode(tmp_file.read()).decode()
}
tmp_file.close()
response = self.client.post(self.papers_submitted, data=urlencode(MultiValueDict(data)), content_type='application/x-www-form-urlencoded',
HTTP_AUTHORIZATION=self.authorization_header)
print(response.data)
self.assertEqual(response.status_code, status.HTTP_200_OK)
del data["cover_letter"]
response = self.client.post(self.papers_submitted, data=urlencode(MultiValueDict(data)), content_type='application/x-www-form-urlencoded',
HTTP_AUTHORIZATION=self.authorization_header)
print(response.data)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
I have successfully tested this endpoint via postman but I don't know how to do it in Python.

If I understand what you are asking, you'd like to test your DRF endpoint with a file upload, using python. You should use RequestFactory, which simulates a request -- it will be more like using Postman.
This answer Django: simulate HTTP requests in shell has an example of using RequestFactory, and there are multiple answers here django RequestFactory file upload for specific solutions on how to include uploaded files with RequestFactory.

I've managed to come up with a solution thanks to Mark's answer.
Here's the code if anyone is interested:
def test_user_can_submit_a_paper(self):
from api.journal import PaperListSubmitted
from django.test.client import RequestFactory
from django.core.files import temp as tempfile
"""
Ensure that an user is able to upload a paper.
"""
request_factory = RequestFactory()
manuscript = tempfile.NamedTemporaryFile(suffix=".txt")
cover_letter = tempfile.NamedTemporaryFile(suffix=".txt")
manuscript.write(b"This is my stupid paper that required me to research writing this test for over 5h")
cover_letter.write(b"This is my stupid paper that required me to research writing this test for over 5h")
manuscript.seek(0)
cover_letter.seek(0)
post_data = {
"title": "My post title",
"description": "this is my paper description",
"authors": "no authors",
"manuscript": manuscript,
"cover_letter": cover_letter
}
request = request_factory.post(self.papers_submitted, HTTP_AUTHORIZATION=self.authorization_header,
data=post_data)
response = PaperListSubmitted.as_view()(request)
self.assertEqual(response.status_code, status.HTTP_200_OK)

Related

Trying to write a script to upload files to a django project

I have a django 3.x project where I can upload multiple files and associated form data through the admin pages to a model called Document. However, I need to upload a large number of files, so I wrote a small python script to automate that process.
I am having one problem with the script. I can't seem to set the name of the file as it is set when uploaded through the admin page.
Here is the script...I had a few problems getting the csrf token working correctly, so there may be some redundant code for that.
import requests
# Set up the urls to login to the admin pages and access the correct add page
URL1='http://localhost:8000/admin/'
URL2='http://localhost:8000/admin/login/?next=/admin/'
URL3 = 'http://localhost:8090/admin/memorabilia/document/add/'
USER='admin'
PASSWORD='xxxxxxxxxxxxx'
client = requests.session()
# Retrieve the CSRF token first
client.get(URL1) # sets the cookie
csrftoken = client.cookies['csrftoken']
print("csrftoken1=%s" % csrftoken)
login_data = dict(username=USER, password=PASSWORD, csrfmiddlewaretoken=csrftoken)
r = client.post(URL2, data=login_data, headers={"Referer": "foo"})
r = client.get(URL3)
csrftoken = client.cookies['csrftoken']
print("csrftoken2=%s" % csrftoken)
cookies = dict(csrftoken= csrftoken)
headers = {'X-CSRFToken': csrftoken}
file_path = "/media/mark/ea00fd8e-4330-4d76-81d8-8fe7dde2cb95/2017/Memorable/20047/Still Images/Photos/20047_Phillips_Photo_052_002.jpg"
data = {
"csrfmiddlewaretoken": csrftoken,
"documentType_id": '1',
"rotation" : '0',
"TBD": '350',
"Title": "A test title",
"Period": "353",
"Source Folder": '258',
"Decade": "168",
"Location": "352",
"Photo Type": "354",
}
file_data = None
with open(file_path ,'rb') as fr:
file_data = fr.read()
# storage_file_name is the name of the FileField in the Document model.
#response_1 = requests.post(url=URL3, data=data, files={'storage_file_name': file_data,}, cookies=cookies)
response_2 = client.post(url=URL3, data=data, files={'storage_file_name': file_data, 'name': "20047_Phillips_Photo_052_002.jpg"}, cookies=cookies,)
When I upload using the admin page, the name of the file is "20047_Phillips_Photo_052_002.jpg", as it should be (i.e. storage_file_name.name = 20047_Phillips_Photo_052_002.jpg).
When I run the script using files={'storage_file_name': file_data,} (see response_1 at the bottom of the script), the files uploads correctly except that the name of the file is "storage_file_name" and not "20047_Phillips_Photo_052_002.jpg" (i.e. storage_file_name.name = "storage_file_name").
When I upload using files={'storage_file_name': file_data, 'name': "20047_Phillips_Photo_052_002.jpg"} the name of the file is still "storage_file_name" (i.e. storage_file_name.name = "storage_file_name").
I looked in the request.FILES object when uploading a file through the admin page, and the _name field for each object is the name of the file being uploaded. The documentation for the django File object says it has a field called name.
What am I missing to get my script to upload a file the same way as the admin page does? By that I mean, the name of the file is not "storage_file_name".
When I change the last response= line to
response = client.post(url=URL3, data=metadata, files= {'storage_file_name': open(file_path ,'rb'),}, cookies=cookies, headers=headers)
the file upload works and the file name is correctly displayed.

Headers content included in file uploaded in django rest framework

I'm having an issue in my django api file upload.
When I upload file using FileUploadParser everything goes well
but the file upload contains the header of the request stuff like Content-Disposition
when i try to open the uploaded file it is broken. I searched for a while for some solution but no chance. Decided to use MultiPartparser but this way nothing is included in the request.data dict. How can I go around this ? Can somebody show me a code or a way to successfully upload file or image to my api without having them broken ? Thanks for any hint.
Here's the code I have so far
class EstablishmentMediaUploadView(views.APIView):
permission_classes = (IsAuthenticated,)
authentication_class = JSONWebTokenAuthentication
parser_classes = (FileUploadParser,)
serializer_class = MediaSerializer
name = 'establishment-media-file-upload'
def put(self, request, **kwargs):
print(request.data)
if 'file' not in request.data:
raise ParseError("Empty media file for establishment")
establishmentid = kwargs.get('establishmentid')
if establishmentid is None:
return Response({"error": "You didn't specify the establishmentid"}, status=400)
mediaFile = request.data.get('file')
media = Media.objects.create(mediatitle=mediaFile.name)
establishment = Establishment.objects.get(id=establishmentid)
media.establishmentlogo.save(mediaFile.name, mediaFile, save=False)
media.establishment = establishment
media.save()
return Response({"message": "Logo added for this establishment"}, status=200)
I first test it by uploading from Insomnia api test client and Vscode thunder client extension. For both the headers are included.
Then I did the test in my angular frontend. Here's the code of the service method in charge of the upload :
setEstablishmentLogo(establishment: Establishment, media: Media): Observable<Object> {
let formdata = new FormData();
formdata.set("establishmentlogo", media.establishmentlogo);
var url = `${endpoints.establishment_media_upload_uri_base}/${establishment.id}`;
console.log(url)
return this.http.put(url, formdata, {
headers: {
"Accept": "*/*",
"Content-Disposition": `attachment; filename=${media.establishmentlogo.name}`,
'Authorization': `Bearer ${token}`
}
});
}
Got the same behavior. Perhaps I'm getting something wrong in all this. But can't figure it out.
Well i use flutter framework and i upload images from mobile to django and guess what, it updates image field without doing anything from my end. Just make sure where ever you are uploading this image from do it accordingly to http library in that framework.
So just to avoid this headache to another folk. Here is what I found finally.
I was using FileUploadParser in my uploadview. But the point is that when using the 'FileUploadParser' the request.data dict
is populated with the uploaded content. The uploaded content in that case, when it comes from clients like Postman
or Insomnia or Thunder Client of vscode, it contains the single file you uploaded (We assume that you select the option binary request
in those clients) and that way, nothing wrong happens, the files are safe. But when you upload files from the browser, let's say from angular and the parser_class in the view is set to FileUploadParser
then you will successfully upload the file but it would be broken because the whole request (file + headers and browser boundaries stuffs) is parsed as a single file
by FileUploadParser(It's indeed it's job) and thus you end up with broken files on your server or backend. So the way to go around that is to set parser_classes to MultipartParser and FormParser optionally
That way your uploaded files are fine. And much, you don't need to specify Content-Disposition: attachment; filename='some file name' in your request header. Bear in mind that when you use FileUploadParser, your file is in request.data['file'] and when it is MultiPartParser (and FormParser) the file is in request.FILES['file'].
With all this said, the working version of the upload view I posted in my question looks like below :
class EstablishmentMediaUploadView(views.APIView):
permission_classes = (IsAuthenticated,)
authentication_class = JSONWebTokenAuthentication
parser_classes = (MultiPartParser, FormParser,)
name = 'mtp-establishment-media-file-upload'
def post(self, request, *args, **kwargs):
if 'file' not in request.FILES:
return Response({"message": "Please provide a file"}, status=status.HTTP_400_BAD_REQUEST)
establishmentid = kwargs.get('establishmentid')
if establishmentid is None:
return Response({"error": "You didn't specify the establishmentid"}, status=status.HTTP_400_BAD_REQUEST)
mediaFile = request.FILES['file']
media = Media.objects.create(mediatitle=mediaFile.name)
media.establishmentlogo.save(mediaFile.name, mediaFile, save=True)
try:
establishment = Establishment.objects.get(id=establishmentid)
except Establishment.DoesNotExist:
return Response({"error": "Establishment does not exist"}, status=status.HTTP_404_NOT_FOUND)
media.establishment = establishment
media.save()
return Response({"message": "Logo added for this establishment"}, status=status.HTTP_201_CREATED)

Test cases for Django Rest Framework; struggling to get a correct response

Update: Solved my own problem: I've learnt that Django creates its own test database, and as such, it needs to be populated with data. Ran my importer in my test cases and it all worked. So, if you're also wondering why you're tests don't work, check that you've got some data in the test db!
End Update
I am writing tests for my Django Rest Framework API but I am struggling to get my code to return a 200 OK. At the moment, my test case continually returns a 404 Not Found.
I'm in the early stages of writing tests, and have a lot to learn. I'm currently following https://www.django-rest-framework.org/api-guide/testing/
I'm trying to test an endpoint at the following URL
# Not shown here, is that all URLs here will be prepended with /api/v1
path('case/<int:pk>/', EntireCaseView.as_view(), name='case'),
I have an object in my database with an ID (primary key) of 1. I can successful query the API by going to http://localhost:8000/api/v1/case/1/
I receive a valid JSON response (Trampe is a rabbit)
{
"id": 1,
"total_points": 5000,
"passing_points": 3700,
"budget": 5000,
"description": "Saving Trampe from Trauma",
"name": "Trampe",
"signalment": "8yr, intact male, mixed breed.",
"problem": "Respiratory difficulty",
"image": {
"id": 1,
"file": "http://localhost:8000/media/images/trampe.jpg",
"description": "A lovely picture of Trampe"
},
My API requires authentication, and as such I am providing authentication in my test case.
class CaseTests(APITestCase):
def test_status_code(self):
"""
ensure that case/1 returns 200 OK
"""
# Create a test user
test_user = User(username='jim', password='monkey123', email='jim#jim.com')
test_user.save()
# build a factory and get our user Jim
factory = APIRequestFactory()
user = User.objects.get(username='jim')
# Get our view to test and the url, too
view = EntireCaseView.as_view()
url = reverse('case', kwargs={'pk': '1'})
print(url.__str__())
# Make an authenticated request to the view...
request = factory.get(url)
print(request.get_full_path())
force_authenticate(request, user=user)
response = view(request, "1")
print(response.data)
self.assertEqual(response.status_code, status.HTTP_200_OK)
Of interest here (at least to me) are the lines
url = reverse('case', kwargs={'pk': '1'})
and
response = view(request, "1")
If I leave out either the kwargs argument in url =r everse('case', kwargs={'pk': '1'}) or the "1" in response = view(request, "1") I will receive an error saying that the get() method requires 2 positional arguments but only given.
Here is the signature of the get() method in my view.
class EntireCaseView(APIView):
def get(self, request, pk):
If I run my test, Django reports that it fails because of a 404.
self.assertEqual(response.status_code, status.HTTP_200_OK)
AssertionError: 404 != 200
What I am trying to work out is why this is the case. Printing print(url.__str__()) outputs /api/v1/case/1/ as does print(request.get_full_path())
So in summary, I am trying to understand why I'm receiving this 404, and ultimately, how I can test this, and other endpoints.
Any and all help is appreciated.
Cheers,
C

How to properly send data from APIClient in Django (rest_framework) for a POST request

I've encountered some strange behavior in a Django unittest. Specifically, I'm using the APIClient module from rest_framework.test to simulate GET/POST requests from a unittest.
The issue occurs when updating/creating a new object in the Django ORM via a POST request (see the code below):
def test_something(self):
data = {
"name": 'unit testing',
"data": {}
}
response = self.api_client.post(reverse('save_model'), data=data, format='json')
self.assertEqual(response.status_code, 200)
#api_view(['GET', 'POST'])
def save_model(request):
obj, created = MyModel.objects.update_or_create(
user_id=request.user,
**request.data
)
return JsonResponse({
'id': obj.id,
'name': obj.name,
'user_id': obj.user_id.id
})
The error i receive when running the test case:
Error binding parameter 1 - probably unsupported type
Based on other stack posts involving this error, i would assume i have a type issue for the second parameter (the data field). However when the same exact data is used to store an object in Django shell, it works every time. Additionally, when the request is made from the client (with the same data) the request succeeds every time.
If i print the data in the unittest request i get the following:
(, u'{}')
(, u'unit testing')
Model code is below:
class MyModel(models.Model):
id = models.AutoField(primary_key=True)
name = models.CharField(max_length=255)
user_id = models.ForeignKey(AUTH_USER_MODEL)
data = JSONField()
So i thought this might be a unicode issue. But once again, storing the object with unicode data in the shell works just fine. One subtle difference to note, the django unittest will create a new test db for the models, whereas running in the shell does not.
Im out of answers, so if someone can shed some light on what's occuring here, that would be amazing.

Send file from backbone Model to django

I want to upload files from Backbone to Django File upload system.
First of all I've follow the https://stackoverflow.com/a/10916733/1590377 explanation. I've do a FileModel and with the above indication I have a model with this information:
attributes: Object
data: "data:image/png;base64,iVBORw ..."
file: "image2012-06-12 13:36:45.png"
now I save the model to the URL where I have the upload view in django like this:
def upload_file_64(request):
if request.method == 'POST':
file = cStringIO.StringIO(base64.b64decode(request.POST['data']))
#method to save the file
response_data={"result":"ok"}
return HttpResponse(simplejson.dumps(response_data), mimetype='application/json')
else:
response_data={"success": "No a post request"}
return HttpResponse(simplejson.dumps(response_data), mimetype='application/json')
but the response that the django sistem give me is:
"MultiValueDictKeyError at /api/upload64/↵'Key \'data\' not found in <QueryDict: {u\'base64,iVBORw0KG....
The POST http request is:
POST:
base64,iVBORw0KG ..."} = u''
{"file":"Captura de pantalla de 2012-06-12 13:36:45.png","data":"data:image/png = u''
How I can fix this so that I can upload a file to django. I use a multi-part method to upload files from another platforms how android but with backbone I can't upload a file.
Can someone help me eith this problem?
Thanks!!
I've coded another solution. I've used a jquery upload pluging to upload the file, and get the response.
The plugin is : http://lagoscript.org/jquery/upload/demo?locale=en and the code that I used in my backbone view is:
events : {
'change #file1' : 'upload'
},
upload : function(){
$('#file1').upload('http://192.168.0.195/api/upload/', function(res) {
console.log(res)
//now I use the res to create a model :)
}, 'html');
},