I would like some help about my code. My goal is to send at the same time string variables as a ini plain text and a bmp file in an httpResponse.
For the moment I insert the decoded bytes of the bmp file in an ini parameter, take into account that I communicate with an interphone which is only client but not server so I can only make httpresponses but no requests.
If I base64 encode my image, I'll need to change the software of our interphone to decode it, for the moment I can't, can you tell me if base64 encode bytes is mandatory in my case ?
I made some researches on the web and I saw that people base64 encode their images or they make multipart response.
Could you help me to implement a multipart response please, even hand made, that would interest me ?
I show you how I do for the moment, I put the image in the "string" ini parameter:
def send_bmp():
outputConfig = io.StringIO()
outputConfig.write('[RETURN_INFO]\r\n')
outputConfig.write('config_id=255\r\n')
outputConfig.write('config_type=2\r\n')
outputConfig.write('action=3\r\n')
outputConfig.write('[DATABASE]\r\n')
file = open(django_settings.TMP_DIR+'/qrcode.bmp', 'rb').read()
outputConfig.write('size_all='+str(len(file))+'\r\n')
outputConfig.write('string='+file.decode('iso-8859-1')+'\r\n')
outputConfig.write('csum='+str(sum(file))+'\r\n')
body = outputConfig.getvalue()
httpR = HttpResponse(body, content_type='text/plain;charset=iso-8859-1')
httpR['Content-Length'] = len(body)
return httpR
Here is the response I get :
https://gist.github.com/Ezekiah/e6fd50f13c05f338f27a
If you need to mix the image file content with the rest of the response I think you have to use Base64 encoding. If it is possible to return the ini parameters in one request and the file in another Django provides a FileResponse class(subclass of StreamingHttpResponse) that you can use to return the bmp file in chunks, like this:
from django.http import FileResponse
def send_bmp(request):
file = open(django_settings.TMP_DIR+'/qrcode.bmp', 'rb')
return FileResponse(file)
Related
I want to upload large files using flask. Rather than try to load the entire file into memory, I've implemented the request.stream.read() method to stream the file to disk in chunks, as per the following code, which is very similar to answers given to many similar questions I have found:
#app.route("/uploadData", methods=["POST"])
def uploadData():
filename = uuid.uuid4().hex + '.nc'
filePath = os.path.join("/tmp", filename)
with open(filePath, "wb+") as f:
chunk_size = 4096
while True:
chunk = flask.request.stream.read(chunk_size)
if len(chunk) == 0:
break
f.write(chunk)
return flask.jsonify({'success': True, 'filename': filename})
This works well, except that it "wraps" the file in post data, like the following:
------WebKitFormBoundaryoQ8GPdNkcfUNrKBd
Content-Disposition: form-data; name="inputFile"; filename="some_file_upload.nc"
Content-Type: application/x-netcdf
<Actual File content here>
------WebKitFormBoundaryoQ8GPdNkcfUNrKBd--
How can I stream the file to disk without getting the form boundary stuff?
In theory, I could call flask.request.file or the like to get the file correctly, but as that loads the entire file into memory (or more likely a temporary file), and is quite slow relative to the stream method, I don't like it as a solution.
If it makes a difference, I'm initiating the file upload using the following javascript:
var formData=new FormData($('#fileform')[0])
$.ajax({
url:'/uploadData',
data:formData,
processData:false,
contentType:false,
type:'POST'
})
EDIT: I've managed to work around the issue by using readline() rather than read(), discarding the first four lines, and then checking for chunk starting with "---" to discard the last line, which works. However, this feels both kludgy and fragile, so if there is a better solution, I would love to hear it.
I am trying to upload an image file to cloudinary which I have sent from my django template to a function in views.py
The file is in request.FILES['image']
cloudinary.config(
cloud_name="p*****",
api_key="33************",
api_secret="4***-S***_o*********"
)
img_obj = request.FILES['image']
cloudinary_response = cloudinary.uploader.upload(img_obj)
image_url = cloudinary_response['url']
Printing img_obj gives the name of the image (Like : "tree.jpg")
cloudinary upload doc is as follows https://cloudinary.com/documentation/image_upload_api_reference#upload
The type of img_obj is InMemoryUploadedFile. Now is there a way to convert it to base64 or somthing like that so I can upload.
Or any other solution ??
You have a few options:
img_obj.file is the actual file object, so you could try uploading that directly.
Since an InMemoryUploadedFile is a subclass of File, you can also just open(mode='rb') the file, using standard python file io functions.
Or you can try img_obj.file.read()
I would go for the second option:
import base64
with img_obj.open("rb") as image_file:
encoded_string = base64.b64encode(image_file.read())
My server is uWSGI and Python. I send myself an image from a file upload on the web page. How do I parse that file on the server?
I was able to handle a CSV because it's just text and I sent it by itself, but I have no idea how to handle images, or if I send the text file with other data. I'll add sample POST data to clarify when I'm back at my computer.
Part of my problem is the previous developer did some weird things with parsing POST data, so instead of being able to let uWSGI turn it into usable data, I have to do that myself in Python.
I assume you were handeling url encoded data by doing read on environ['wigs.imput'], something like this.
try:
request_body_size = int(environ.get('CONTENT_LENGTH', 0))
except (ValueError):
request_body_size = 0
request_body = environ['wsgi.input'].read(request_body_size)
dP = parse_qs(request_body)
For multipart/form-data data you need to use cgi.FieldStorage.
d = cgi.FieldStorage(environ=environ, fp=environ['wsgi.input'], keep_blank_values=True)
For Normal values in your form you can do
firstName = d.getvalue("firstName")
For the file you can get it by
file_data = d['imageFile'].file.read()
filename = d['imageFile'].filename
I'm writing a Django function that takes some user input, and generates a pdf for the user. However, the process for generating the pdf is quite intensive, and I'll get a lot of repeated requests so I'd like to store the generated pdfs on the server and check if they already exist before generating them.
The problem is that django-wkhtmltopdf (which I'm using for generation) is meant to return to the user directly, and I'm not sure how to store it on the file.
I have the following, which works for returning a pdf at /pdf:
urls.py
urlpatterns = [
url(r'^pdf$', views.createPDF.as_view(template_name='site/pdftemplate.html', filename='my_pdf.pdf'))
]
views.py
class createPDF(PDFTemplateView):
filename = 'my_pdf.pdf'
template_name = 'site/pdftemplate.html'
So that works fine to create a pdf. What I'd like is to call that view from another view and save the result. Here's what I've got so far:
#Create pdf
pdf = createPDF.as_view(template_name='site/pdftemplate.html', filename='my_pdf.pdf')
pdf = pdf(request).render()
pdfPath = os.path.join(settings.TEMP_DIR,'temp.pdf')
with open(pdfPath, 'w') as f:
f.write(pdf.content)
This creates temp.pdf and is about the size I'd expect but the file isn't valid (it renders as a single completely blank page).
Any suggestions?
Elaborating on the previous answer given: to generate a pdf file and save to disk do this anywhere in your view:
...
context = {...} # build your context
# generate response
response = PDFTemplateResponse(
request=self.request,
template=self.template_name,
filename='file.pdf',
context=context,
cmd_options={'load-error-handling': 'ignore'})
# write the rendered content to a file
with open("file.pdf", "wb") as f:
f.write(response.rendered_content)
...
I have used this code in a TemplateView class so request and template fields were set like that, you may have to set it to whatever is appropriate in your particular case.
Well, you need to take a look to the code of wkhtmltopdf, first you need to use the class PDFTemplateResponse in wkhtmltopdf.views to get access to the rendered_content property, this property get us access to the pdf file:
response = PDFTemplateResponse(
request=<your_view_request>,
template=<your_template_to_render>,
filename=<pdf_filename.pdf>,
context=<a_dcitionary_to_render>,
cmd_options={'load-error-handling': 'ignore'})
Now you could use the rendered_content property to get access to the pdf file:
mail.attach('pdf_filename.pdf', response.rendered_content, 'application/pdf')
In my case I'm using this pdf to attach to an email, you could store it.
I have a lot of user uploaded content and I want to validate that uploaded image files are not, in fact, malicious scripts. In the Django documentation, it states that ImageField:
"Inherits all attributes and methods from FileField, but also validates that the uploaded object is a valid image."
Is that totally accurate? I've read that compressing or otherwise manipulating an image file is a good validation test. I'm assuming that PIL does something like this....
Will ImageField go a long way toward covering my image upload security?
Django validates the image uploaded via form using PIL.
See https://code.djangoproject.com/browser/django/trunk/django/forms/fields.py#L519
try:
# load() is the only method that can spot a truncated JPEG,
# but it cannot be called sanely after verify()
trial_image = Image.open(file)
trial_image.load()
# Since we're about to use the file again we have to reset the
# file object if possible.
if hasattr(file, 'reset'):
file.reset()
# verify() is the only method that can spot a corrupt PNG,
# but it must be called immediately after the constructor
trial_image = Image.open(file)
trial_image.verify()
...
except Exception: # Python Imaging Library doesn't recognize it as an image
raise ValidationError(self.error_messages['invalid_image'])
PIL documentation states the following about verify():
Attempts to determine if the file is broken, without actually decoding
the image data. If this method finds any problems, it raises suitable
exceptions. This method only works on a newly opened image; if the
image has already been loaded, the result is undefined. Also, if you
need to load the image after using this method, you must reopen the
image file.
You should also note that ImageField is only validated when uploaded using form. If you save the model your self (e.g. using some kind of download script), the validation is not performed.
Another test is with the file command. It checks for the presence of "magic numbers" in the file to determine its type. On my system, the file package includes libmagic as well as a ctypes-based wrapper /usr/lib64/python2.7/site-packages/magic.py. It looks like you use it like:
import magic
ms = magic.open(magic.MAGIC_NONE)
ms.load()
type = ms.file("/path/to/some/file")
print type
f = file("/path/to/some/file", "r")
buffer = f.read(4096)
f.close()
type = ms.buffer(buffer)
print type
ms.close()
(Code from here.)
As to your original question: "Read the Source, Luke."
django/core/files/images.py:
"""
Utility functions for handling images.
Requires PIL, as you might imagine.
"""
from django.core.files import File
class ImageFile(File):
"""
A mixin for use alongside django.core.files.base.File, which provides
additional features for dealing with images.
"""
def _get_width(self):
return self._get_image_dimensions()[0]
width = property(_get_width)
def _get_height(self):
return self._get_image_dimensions()[1]
height = property(_get_height)
def _get_image_dimensions(self):
if not hasattr(self, '_dimensions_cache'):
close = self.closed
self.open()
self._dimensions_cache = get_image_dimensions(self, close=close)
return self._dimensions_cache
def get_image_dimensions(file_or_path, close=False):
"""
Returns the (width, height) of an image, given an open file or a path. Set
'close' to True to close the file at the end if it is initially in an open
state.
"""
# Try to import PIL in either of the two ways it can end up installed.
try:
from PIL import ImageFile as PILImageFile
except ImportError:
import ImageFile as PILImageFile
p = PILImageFile.Parser()
if hasattr(file_or_path, 'read'):
file = file_or_path
file_pos = file.tell()
file.seek(0)
else:
file = open(file_or_path, 'rb')
close = True
try:
while 1:
data = file.read(1024)
if not data:
break
p.feed(data)
if p.image:
return p.image.size
return None
finally:
if close:
file.close()
else:
file.seek(file_pos)
So it looks like it just reads the file 1024 bytes at a time until PIL says it's an image, then stops. This obviously does not integrity-check the entire file, so it really depends on what you mean by "covering my image upload security": illicit data could be appended to an image and passed through your site. Someone could DOS your site by uploading a lot of junk or a really big file. You could be vulnerable to an injection attack if you don't check any uploaded captions or make assumptions about the image's uploaded filename. And so on.