I have Python flask webservice that takes in a file:
Headers:
Content-type: multipart/formdata
Content:
"fileTest": UPLOADED FILE
When I pass the file to another service using requests lib, I get issue where the uploaded file is not passed.
My Code:
files = {}
for form_file_param in request.files:
fs = request.files[form_file_param] # type: FileStorage
files[form_file_param] = (fs.filename, fs.read())
req_headers = {
"content-type": u "multipart/form-data; boundary=X-INSOMNIA-BOUNDARY",
}
r = requests.request(method='POST',
url=url,
headers=req_headers,
files=files)
I contact my other service directly through postman and it works successfully. I cannot seem to figure out what I am doing wrong in the above code.
You need to follow requests document.
http://docs.python-requests.org/en/master/user/quickstart/#post-a-multipart-encoded-file
url = 'https://httpbin.org/post'
files = {'file': ('report.xls', open('report.xls', 'rb'), 'application/vnd.ms-excel', {'Expires': '0'})}
r = requests.post(url, files=files)
r.text
Change . after watching OP response , the issue caused by header - Content-Type.
This is a special content type which can be visualized as multiple sub-requests in one big request. Each of those sub-requests (one form-data element) has their own set of headers. The content type of the actual data is in there.1
Note : there are no different between fs and fs.read()
#models.py line 149
if isinstance(fp, (str, bytes, bytearray)):
fdata = fp
else:
data = fp.read()
Related
I'm currently trying to save a file via requests, it's rather large, so I'm instead streaming it.
I'm unsure how to specifically do this, as I keep getting different errors. This is what I have so far.
def download_file(url, matte_upload_path, matte_servers, job_name, count):
local_filename = url.split('/')[-1]
url = "%s/static/downloads/%s_matte/%s/%s" % (matte_servers[0], job_name, count, local_filename)
with requests.get(url, stream=True) as r:
r.raise_for_status()
fs = FileSystemStorage(location=matte_upload_path)
print(matte_upload_path, 'matte path upload')
with open(local_filename, 'wb') as f:
for chunk in r.iter_content(chunk_size=8192):
f.write(chunk)
fs.save(local_filename, f)
return local_filename
but it returns
io.UnsupportedOperation: read
I'm basically trying to have requests save it to the specific location via django, any help would be appreciated.
I was able to solve this, by using a tempfile to save the python requests, then saving it via the FileSystemStorage
local_filename = url.split('/')[-1]
url = "%s/static/downloads/%s_matte/%s/%s" % (matte_servers[0], job_name, count, local_filename)
response = requests.get(url, stream=True)
fs = FileSystemStorage(location=matte_upload_path)
lf = tempfile.NamedTemporaryFile()
# Read the streamed image in sections
for block in response.iter_content(1024 * 8):
# If no more file then stop
if not block:
break
# Write image block to temporary file
lf.write(block)
fs.save(local_filename, lf)
I am receiving an http request from a desktop application with a screenshot. I cannot speak with the developer or see source code, so all I have is the http request I am getting.
The file isn't in request.FILES, it is in request.POST.
#csrf_exempt
def create_contract_event_handler(request, contract_id, event_type):
keyboard_events_count = request.POST.get('keyboard_events_count')
mouse_events_count = request.POST.get('mouse_events_count')
screenshot_file = request.POST.get('screenshot_file')
barr2 = bytes(screenshot_file.encode(encoding='utf8'))
with open('.test/output.jpeg', 'wb') as f:
f.write(barr2)
f.close()
The file is corrupted.
The binary starts like this, I don't know if that helps:
����JFIFHH��C
%# , #&')*)-0-(0%()(��C
(((((((((((((((((((((((((((((((((((((((((((((((((((�� `"��
Also, if I try to open the image with PIL, I get the following error:
from PIL import Image
im = Image.open('./test/output.jpg')
#OSError: cannot identify image file './test/output.jpg'
Finally, I managed to touch the code in the other hand, the 'filename' was missing in the header and for that reason I was getting the file in the POST instead of in the FILES dictionary.
Right now I'm just checking the response of the link like so:
self.client = Client()
response = self.client.get(url)
self.assertEqual(response.status_code, 200)
Is there a Django-ic way to test a link to see if a file download event actually takes place? Can't seem to find much resource on this topic.
If the url is meant to produce a file rather than a "normal" http response, then its content-type and/or content-disposition will be different.
the response object is basically a dictionary, so you could so something like
self.assertEquals(
response.get('Content-Disposition'),
"attachment; filename=mypic.jpg"
)
more info:
https://docs.djangoproject.com/en/dev/ref/request-response/#telling-the-browser-to-treat-the-response-as-a-file-attachment
UPD:
If you want to read the actual contents of the attached file, you can use response.content. Example for a zip file:
try:
f = io.BytesIO(response.content)
zipped_file = zipfile.ZipFile(f, 'r')
self.assertIsNone(zipped_file.testzip())
self.assertIn('my_file.txt', zipped_file.namelist())
finally:
zipped_file.close()
f.close()
I have taken a look at other questions related to multipart/form POST requests in Python but unfortunately, they don't seem to address my exact question. Basically, I normally use CURL in order to hit an API service that allows me to upload zip files in order to create HTML5 assets. The CURL command I use looks like this:
curl -X POST -H "Authorization: api: 222111" --form "type=html" --form "file=Folder1/Folder2/example.zip" "https://example.api.com/upload?ins_id=123"
I am trying to use a python script to iterate through a folder of zip files in order to upload all of these files and receive a "media ID" back. This is what my script looks like:
import os
import requests
import json
ins_id = raw_input("Please enter your member ID: ")
auth = raw_input("Please enter your API authorization token: ")
for filename in os.listdir("zips"):
if filename.endswith(".zip"):
file_path = os.path.abspath(filename)
url = "https://example.api.com/upload?
ins_id="+str(ins_id)
header = {"Authorization": auth}
response = requests.post(url, headers=header, files={"form_type":
(None, "html"), "form_file_upload": (None, str(file_path))})
api_response = response.json()
print api_response
This API service requires the file path to be included when submitting the POST. However, when I use this script, the response indicates that "file not provided". Am I including this information correctly in my script?
Thanks.
Update:
I think I am heading in the right direction now (thanks to the answer provided) but now, I receive an error message stating that there is "no such file or directory". My thinking is that I am not using os.path correctly but even if I change my code to use "relpath" I still get the same message. My script is in a folder and I have a completely different folder called "zips" (in the same directory) which is where all of my zip files are stored.
To upload files with the request library, you can include the file handler directly in the JSON as described in the documentation. This is the corresponding example that I have taken from there:
url = 'http://httpbin.org/post'
files = {'file': open('path_to_your_file', 'rb')}
r = requests.post(url, files=files)
If we integrate this in your script, it would look as follows (I also made it slightly more pythonic):
import os
import requests
import json
folder = 'zips'
ins_id = raw_input("Please enter your member ID: ")
auth = raw_input("Please enter your API authorization token: ")
url = "https://example.api.com/upload?"
header = {"Authorization": auth}
for filename in os.listdir(folder):
if not filename.endswith(".zip"):
continue
file_path = os.path.abspath(os.path.join(folder, filename))
ins_id="+str(ins_id)"
response = requests.post(
url, headers=header,
files={"form_type": (None, "html"),
"form_file_upload": open(file_path, 'rb')}
)
api_response = response.json()
print api_response
As I don't have the API end point, I can't actually test this code block - but it should be something along these lines.
Right now I'm just checking the response of the link like so:
self.client = Client()
response = self.client.get(url)
self.assertEqual(response.status_code, 200)
Is there a Django-ic way to test a link to see if a file download event actually takes place? Can't seem to find much resource on this topic.
If the url is meant to produce a file rather than a "normal" http response, then its content-type and/or content-disposition will be different.
the response object is basically a dictionary, so you could so something like
self.assertEquals(
response.get('Content-Disposition'),
"attachment; filename=mypic.jpg"
)
more info:
https://docs.djangoproject.com/en/dev/ref/request-response/#telling-the-browser-to-treat-the-response-as-a-file-attachment
UPD:
If you want to read the actual contents of the attached file, you can use response.content. Example for a zip file:
try:
f = io.BytesIO(response.content)
zipped_file = zipfile.ZipFile(f, 'r')
self.assertIsNone(zipped_file.testzip())
self.assertIn('my_file.txt', zipped_file.namelist())
finally:
zipped_file.close()
f.close()