I want to return some files in a HttpResponse and I'm using the following function. The file that is returned always has a filesize of 1kb and I do not know why. I can open the file, but it seems that it is not served correctly. Thus I wanted to know how one can return files with django/python over a HttpResponse.
#login_required
def serve_upload_files(request, file_url):
import os.path
import mimetypes
mimetypes.init()
try:
file_path = settings.UPLOAD_LOCATION + '/' + file_url
fsock = open(file_path,"r")
#file = fsock.read()
#fsock = open(file_path,"r").read()
file_name = os.path.basename(file_path)
file_size = os.path.getsize(file_path)
print "file size is: " + str(file_size)
mime_type_guess = mimetypes.guess_type(file_name)
if mime_type_guess is not None:
response = HttpResponse(fsock, mimetype=mime_type_guess[0])
response['Content-Disposition'] = 'attachment; filename=' + file_name
except IOError:
response = HttpResponseNotFound()
return response
Edit:
The bug is actually not a bug ;-)
This solution is working in production on an apache server, thus the source is ok.
While writing this question I tested it local with the django development server and was wondering why it does not work. A friend of mine told me that this issue could arise if the mime types are not set in the server. But he was not sure if this is the problem. But one thing for sure.. it has something to do with the server.
Could it be that the file contains some non-ascii characters that render ok in production but not in development?
Try reading the file as binary:
fsock = open(file_path,"rb")
Try passing the fsock iterator as a parameter to HttpResponse(), rather than to its write() method which I think expects a string.
response = HttpResponse(fsock, mimetype=...)
See http://docs.djangoproject.com/en/dev/ref/request-response/#passing-iterators
Also, I'm not sure you want to call close on your file before returning response. Having played around with this in the shell (I've not tried this in an actual Django view), it seems that the response doesn't access the file until the response itself is read. Trying to read a HttpResponse created using a file that is now closed results in a ValueError: I/O operation on closed file.
So, you might want to leave fsock open, and let the garbage collector deal with it after the response is read.
Try disabling "django.middleware.gzip.GZipMiddleware" from your MIDDLEWARE_CLASSES in settings.py
I had the same problem, and after I looked around the middleware folder, this middleware seemed guilty to me and removing it did the trick for me.
Related
I have a website that has a button. When it's clicked, it returns a number of pandas dataframers into an excel file and returns that excel file automatically as download.
It seems to work ok, except when I open the file, it seems to be corrupted. It asks if some of the tabs should be recovered. I'm using the code below. Any suggestions are appreciated what could be the cause for this.
import io
from flask.helpers import make_response
from pandas.io.excel import ExcelWriter
output = io.BytesIO()
writer = ExcelWriter(output)
dfs = [df1,df2....]
tabs ['tab1','tab2',....]
for df, tab_name in zip(dfs, tab_names):
df.to_excel(writer, tab_name)
writer.close()
resp = make_response(output.getvalue())
resp.headers['Content-Disposition'] = 'attachment; filename=output.xlsx'
resp.headers["Content-type"] = "text/csv"
return resp
You'll need to to add
output.seek(0)
after you close the writer.
You might also find it easier to write
return send_file(output, attachment_filename="output.xlsx", as_attachment=True)
(after importing send_file from flask)
Right now I'm just checking the response of the link like so:
self.client = Client()
response = self.client.get(url)
self.assertEqual(response.status_code, 200)
Is there a Django-ic way to test a link to see if a file download event actually takes place? Can't seem to find much resource on this topic.
If the url is meant to produce a file rather than a "normal" http response, then its content-type and/or content-disposition will be different.
the response object is basically a dictionary, so you could so something like
self.assertEquals(
response.get('Content-Disposition'),
"attachment; filename=mypic.jpg"
)
more info:
https://docs.djangoproject.com/en/dev/ref/request-response/#telling-the-browser-to-treat-the-response-as-a-file-attachment
UPD:
If you want to read the actual contents of the attached file, you can use response.content. Example for a zip file:
try:
f = io.BytesIO(response.content)
zipped_file = zipfile.ZipFile(f, 'r')
self.assertIsNone(zipped_file.testzip())
self.assertIn('my_file.txt', zipped_file.namelist())
finally:
zipped_file.close()
f.close()
I have a setup that lets users download files that are stored in the DB as BYTEA data. Everything works OK, except the download speed is very slow...it seems to download in 33KB chunks, one chunk per second.
Is there a setting I can specify to speed this up?
views.py
from django.http import FileResponse
def getFileResponse(filedata, filename, filesize, contenttype):
response = FileResponse(filedata, content_type=contenttype)
response['Content-Disposition'] = 'attachment; filename=%s' % filename
response['Content-Length'] = filesize
return response
return getFileResponse(
filedata = myfile.filedata, # Binary data from DB
filename = myfile.filename + myfile.fileextension,
filesize = myfile.filesize,
contenttype = myfile.filetype
)
Previously, I had the binary data returned as an HttpResponse and it downloaded like a normal file, with normal speeds. This worked fine locally, but when I pushed to Heroku, it wouldn't download the file -- instead displaying <Memory at XXX> in the download file.
And another side issue...when I include a text file with non-ASCII data (i.e. รก), I get an error as well:
UnicodeEncodeError: 'ascii' codec can't encode characters...: ordinal not in range(128)
How can I handle files with Unicode data?
Update
Anyone know why the download speed gets so slow when changing from HTTPResponse to FileResponse? Or alternatively, why the HTTPResponse to return a file doesn't work on Heroku?
Update - Google Drive
I re-worked my application and hooked it up with a Google Drive back-end for serving files. It employs BytesIO() suggested by Eric below:
def download_file(self, fileid, mimetype=None):
# Get binary file data
request = self.get_file(fileid=fileid, mediaflag=True)
stream = io.BytesIO()
downloader = MediaIoBaseDownload(stream, request)
done = False
# Retry if we received HTTPError
for retry in range(0, 5):
try:
while done is False:
status, done = downloader.next_chunk()
print("Download %d%%." % int(status.progress() * 100))
return stream.getvalue()
except (HTTPError) as error:
return ('API error: {}. Try # {} failed.'.format(error.response, retry))
I think the difference you observe between HttpResponse vs. FileResponse is caused by the spec: https://www.python.org/dev/peps/pep-3333/#buffering-and-streaming
In your previous code, an HttpResponse was created with one huge byte string containing your whole file, and the first iteration pass returned the complete response body. With a a FileResponse, the file is iterated in chunks (of 4kb, 8kb or other depending on your WSGI app server), which (I think) are streamed immediately upstream (to the reverse proxy then client), which may add overhead (more communication over process boundaries?).
It would help to know the app server used (uwsgi, gunicorn, waitress, other) and its relevant config. Also more details about the heroku error in case that can be solved!
why you store whole file in database.
best case is to store file on hard and store only path on database
then according to your web server you can let web server to serve file.
web services serve file better than Django.
if files have no access check store them on media
if your files have access control you according to your web server you can use some response headers
if you use Nginx must use X-Accel-Redirect and use any alternative on other web services tutorial on https://wellfire.co/learn/nginx-django-x-accel-redirects/
I looked at the various questions similar to mine, but I could not find anything a fix for my problem.
In my code, I want to serve a freshly generated excel file residing in my app directory in a folder named files
excelFile = ExcelCreator.ExcelCreator("test")
excelFile.create()
response = HttpResponse(content_type='application/vnd.ms-excel')
response['Content-Disposition'] = 'attachment; filename="test.xls"'
return response
So when I click on the button that run this part of the code, it sends to the user an empty file. By looking at my code, I can understand that behavior because I don't point to that file within my response...
I saw some people use the file wrapper (which I don't quite understand the use). So I did like that:
response = HttpResponse(FileWrapper(excelFile.file),content_type='application/vnd.ms-excel')
But then, I receive the error message from server : A server error occurred. Please contact the administrator.
Thanks for helping me in my Django quest, I'm getting better with all of your precious advices!
First, you need to understand how this works, you are getting an empty file because that is what you are doing, actually:
response = HttpResponse(content_type='application/vnd.ms-excel')
response['Content-Disposition'] = 'attachment; filename="test.xls"'
HttpResponse receives as first arg the content of the response, take a look to its contructor:
def __init__(self, content='', mimetype=None, status=None, content_type=None):
so you need to create the response with the content that you wish, is this case, with the content of your .xls file.
You can use any method to do that, just be sure the content is there.
Here a sample:
import StringIO
output = StringIO.StringIO()
# read your content and put it in output var
out_content = output.getvalue()
output.close()
response = HttpResponse(out_content, mimetype='application/vnd.ms-excel')
response['Content-Disposition'] = 'attachment; filename="test.xls"'
I would recommend you use:
python manage.py runserver
to run your application from the command line. From here you will see the console output of your application and any exceptions that are thrown as it runs. This may provide a quick resolution to your problem.
Right now I'm just checking the response of the link like so:
self.client = Client()
response = self.client.get(url)
self.assertEqual(response.status_code, 200)
Is there a Django-ic way to test a link to see if a file download event actually takes place? Can't seem to find much resource on this topic.
If the url is meant to produce a file rather than a "normal" http response, then its content-type and/or content-disposition will be different.
the response object is basically a dictionary, so you could so something like
self.assertEquals(
response.get('Content-Disposition'),
"attachment; filename=mypic.jpg"
)
more info:
https://docs.djangoproject.com/en/dev/ref/request-response/#telling-the-browser-to-treat-the-response-as-a-file-attachment
UPD:
If you want to read the actual contents of the attached file, you can use response.content. Example for a zip file:
try:
f = io.BytesIO(response.content)
zipped_file = zipfile.ZipFile(f, 'r')
self.assertIsNone(zipped_file.testzip())
self.assertIn('my_file.txt', zipped_file.namelist())
finally:
zipped_file.close()
f.close()