I'm trying to make a script for downloading the uploaded files, on the user's machine. The problem is that the download simply doesn't work (it either downloads me an empty file, or gives me some errors).
the last error is:
coercing to Unicode: need string or buffer, FieldFile found
def download_course(request, id):
course = Courses.objects.get(pk = id).course
path_to_file = 'root/cFolder'
filename = course # Select your file here.
wrapper = FileWrapper(file(course))
content_type = mimetypes.guess_type(filename)[0]
response = HttpResponse(wrapper, content_type = content_type)
response['Content-Length'] = os.path.getsize(filename)
response['Content-Disposition'] = 'attachment; filename=%s/' % smart_str(course)
return response
how can I declare properly the filename so that it will know each time what file to be downloading:
the filename is actually 'course' as declared above
Thanks !
edited
I think that you need to extract path value from FileField object:
def download_course(request, id):
course = Courses.objects.get(pk = id).course
path = course.path # Get file path
wrapper = FileWrapper( open( path, "r" ) )
content_type = mimetypes.guess_type( path )[0]
response = HttpResponse(wrapper, content_type = content_type)
response['Content-Length'] = os.path.getsize( path ) # not FileField instance
response['Content-Disposition'] = 'attachment; filename=%s/' % \
smart_str( os.path.basename( path ) ) # same here
return response
Why is that:
Let's say I have (well, I actually have) Model:
class DanePracodawcy( DaneAdresowe, DaneKontaktowe ):
# other fields
logo = ImageWithThumbnailsField( upload_to = 'upload/logos/',
thumbnail = {'size': (180, 90)},
blank = True )
ImageWithThumbnailsField is subclass of FileField, so it behaves the same way. Now, when I do SELECT:
mysql> select logo from accounts_danepracodawcy;
+-----------------------------+
| logo |
+-----------------------------+
| upload/logos/Lighthouse.jpg |
+-----------------------------+
1 row in set (0.00 sec)
it shows (relative to MEDIA_ROOT) path of stored file. But when I access logo Model attribute:
[D:projekty/pracus]|1> from accounts.models import DanePracodawcy
[D:projekty/pracus]|4> DanePracodawcy.objects.get().logo
<4> <ImageWithThumbnailsFieldFile: upload/logos/Lighthouse.jpg>
[D:projekty/pracus]|5> type( _ )
<5> <class 'sorl.thumbnail.fields.ImageWithThumbnailsFieldFile'>
I get instance of some object. If I try to pass that instance to os.path.getsize:
[D:projekty/pracus]|8> import os.path
[D:projekty/pracus]|9> os.path.getsize( DanePracodawcy.objects.get().logo )
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
D:\projekty\pracus\<ipython console> in <module>()
C:\Python26\lib\genericpath.pyc in getsize(filename)
47 def getsize(filename):
48 """Return the size of a file, reported by os.stat()."""
---> 49 return os.stat(filename).st_size
50
51
TypeError: coercing to Unicode: need string or buffer, ImageWithThumbnailsFieldFile found
I get TypeError, like you. So I need file path as string, which can be obtained with path attribute:
[D:projekty/pracus]|13> os.path.getsize( DanePracodawcy.objects.get().logo.path )
<13> 561276L
Alternatively, I could get name attribute and os.path.join it with MEDIA_ROOT setting:
[D:projekty/pracus]|11> from django.conf import settings
[D:projekty/pracus]|12> os.path.getsize( os.path.join( settings.MEDIA_ROOT, DanePracodawcy.objects.get().logo.name ) )
<12> 561276L
But that's unnecessary typing.
Last thing to note: because path is absolute path, I need to extract filename to pass it to Content-Disposition header:
[D:projekty/pracus]|16> DanePracodawcy.objects.get().logo.path
<16> u'd:\\projekty\\pracus\\site_media\\upload\\logos\\lighthouse.jpg'
[D:projekty/pracus]|17> os.path.basename( DanePracodawcy.objects.get().logo.path )
<17> u'lighthouse.jpg'
Unless you are letting the user download a dynamically generated file, I don't see why you need to do all that.
You can just let this view redirect to the appropriate path, and the respective headers are set by the server serving the static files; typically apache or nginx
I'd do your this view as follows:
from django.conf import settings
def download_course(request,id):
course = get_object_or_404(Course,id=id)
filename = course.course
return redirect('%s/%s'%(settings.MEDIA_URL,filename))
Enjoy :)
Related
I was trying to create some products in ecommerce project in django and i had the data file ready and just wanted to loop throw the data and save to the database with Product.objects.create(image='', ...) but i couldnt upload the images from local directory to database!
I tried these ways:
1
with open('IMAGE_PATH', 'rb') as f:
image = f.read()
Product.objects.create(image=image)
2
image = open('IMAGE_PATH', 'rb')
Product.objects.create(image=image)
3
module_dir = dir_path = os.path.dirname(os.path.realpath(__file__))
for p in products:
file_path = os.path.join(module_dir, p['image'])
Product.objects.create()
product.image.save(
file_path,
File(open(file_path, 'rb'))
)
product.save()
none worked for me.
After some searching, I got the answer.
the code to use would be like this:
from django.core.files import File
for p in products:
product = Product.objects.create()
FILE_PATH = p['image']
local_file = open(f'./APP_NAME/{FILE_PATH}', "rb")
djangofile = File(local_file)
product.image.save('FILE_NAME.jpg', djangofile)
local_file.close()
from django.core.files import File
import urllib
result = urllib.urlretrieve(image_url) # image_url is a URL to an image
model_instance.photo.save(
os.path.basename(self.url),
File(open(result[0], 'rb'))
)
self.save()
Got the answer from here
I'm making a REST api that files can be uploaded based in MODEL-VIEW in flask-appbuilder like this.
But I don't know how to call REST API (POST /File).
I tried several different ways. but I couldn't.
Let me know the correct or the alternative ways.
[client code]
file = {'file':open('test.txt', 'rb'),'description':'test'}
requests.post(url, headers=headers, files=file)
==> Failed
model.py
class Files(Model):
__tablename__ = "project_files"
id = Column(Integer, primary_key=True)
file = Column(FileColumn, nullable=False)
description = Column(String(150))
def download(self):
return Markup(
'<a href="'
+ url_for("ProjectFilesModelView.download", filename=str(self.file))
+ '">Download</a>'
)
def file_name(self):
return get_file_original_name(str(self.file))
view.py
class FileApi(ModelRestApi):
resource_name = "File"
datamodel = SQLAInterface(Files)
allow_browser_login = True
appbuilder.add_api(FileApi)
FileColumn is only a string field that saves the file name in the database. The actual file is saved to config['UPLOAD_FOLDER'].
This is taken care of by flask_appbuilder.filemanager.FileManager.
Furthermore, ModelRestApi assumes that you are POSTing JSON data. In order to upload files, I followed Flask's documentation, which suggests to send a multipart/form-data request. Because of this, one needs to override ModelRestApi.post_headless().
This is my solution, where I also make sure that when a Files database row
is deleted, so is the relative file from the filesystem.
from flask_appbuilder.models.sqla.interface import SQLAInterface
from flask_appbuilder.api import ModelRestApi
from flask_appbuilder.const import API_RESULT_RES_KEY
from flask_appbuilder.filemanager import FileManager
from flask import current_app, request
from marshmallow import ValidationError
from sqlalchemy.exc import IntegrityError
from app.models import Files
class FileApi(ModelRestApi):
resource_name = "file"
datamodel = SQLAInterface(Files)
def post_headless(self):
if not request.form or not request.files:
msg = "No data"
current_app.logger.error(msg)
return self.response_400(message=msg)
file_obj = request.files.getlist('file')
if len(file_obj) != 1:
msg = ("More than one file provided.\n"
"Please upload exactly one file at a time")
current_app.logger.error(msg)
return self.response_422(message=msg)
else:
file_obj = file_obj[0]
fm = FileManager()
uuid_filename = fm.generate_name(file_obj.filename, file_obj)
form = request.form.to_dict(flat=True)
# Add the unique filename provided by FileManager, which will
# be saved to the database. The original filename can be
# retrieved using
# flask_appbuilder.filemanager.get_file_original_name()
form['file'] = uuid_filename
try:
item = self.add_model_schema.load(
form,
session=self.datamodel.session)
except ValidationError as err:
current_app.logger.error(err)
return self.response_422(message=err.messages)
# Save file to filesystem
fm.save_file(file_obj, item.file)
try:
self.datamodel.add(item, raise_exception=True)
return self.response(
201,
**{API_RESULT_RES_KEY: self.add_model_schema.dump(
item, many=False),
"id": self.datamodel.get_pk_value(item),
},
)
except IntegrityError as e:
# Delete file from filesystem if the db record cannot be
# created
fm.delete_file(item.file)
current_app.logger.error(e)
return self.response_422(message=str(e.orig))
def pre_delete(self, item):
"""
Delete file from filesystem before removing the record from the
database
"""
fm = FileManager()
current_app.logger.info(f"Deleting {item.file} from filesystem")
fm.delete_file(item.file)
You can use this.
from app.models import Project, ProjectFiles
class DataFilesModelView(ModelView):
datamodel = SQLAInterface(ProjectFiles)
label_columns = {"file_name": "File Name", "download": "Download"}
add_columns = ["file", "description", "project"]
edit_columns = ["file", "description", "project"]
list_columns = ["file_name", "download"]
show_columns = ["file_name", "download"]
Last add the view to the menu.
appbuilder.add_view(DataFilesModelView,"File View")
I'm following this solution (Serving dynamically generated ZIP archives in Django) to serve some zip files from django.
The idea is to select the files from a database using some check boxes, but I'm trying to make the example work with just 2 images.
import os
import zipfile
import StringIO
from django.http import HttpResponse
def getfiles(request):
# Files (local path) to put in the .zip
# FIXME: Change this (get paths from DB etc)
filenames = ["/home/../image1.png", "/home/../image2.png"]
# Folder name in ZIP archive which contains the above files
# E.g [thearchive.zip]/somefiles/file2.txt
# FIXME: Set this to something better
zip_subdir = "somefiles"
zip_filename = "%s.zip" % zip_subdir
# Open StringIO to grab in-memory ZIP contents
s = StringIO.StringIO()
# The zip compressor
zf = zipfile.ZipFile(s, "w")
for fpath in filenames:
# Calculate path for file in zip
fdir, fname = os.path.split(fpath)
zip_path = os.path.join(zip_subdir, fname)
# Add file, at correct path
zf.write(fpath, zip_path)
# Must close zip for all contents to be written
zf.close()
# Grab ZIP file from in-memory, make response with correct MIME-type
resp = HttpResponse(s.getvalue(), mimetype = "application/x-zip-compressed")
# ..and correct content-disposition
resp['Content-Disposition'] = 'attachment; filename=%s' % zip_filename
return resp
I wrote the getfile(request) on my views.py and i make a call from the index view
def index(request):
if request.method == 'POST': # If the form has been submitted...
resp = getfiles(request)
form = FilterForm(request.POST) # A form bound to the POST data
# do some validation and get latest_events from database
context = {'latest_events_list': latest_events_list, 'form': form}
return render(request, 'db_interface/index.html', context)
I know the getfile() method is called, because if I put names of unexistents files I got an error, but I dont get any download neither an error if the filenames are correct (I put the full path /home/myuser/xxx/yyy/Project/app/static/app/image1.png).
I tried with the django server and with the apache2/nginx server I have for production
I also tried using content_type = 'application/force-download'
Thanks
[I apologize in advance for the length of this question.]
I'm using Django 1.2.3-3+squeeze1 on Debian squeeze.
I am writing an application which uploads zip files to disk in a
temporary location, unzips them, and then saves the results to a
permanent location. The unzipped files are registered in the database as a class called
FileUpload after they are unzipped. The uploaded zipped files also correspond to a class,
but I'll ignore that for the purposes of this question. FileUpload looks like this.
class FileUpload(models.Model):
folder = models.ForeignKey(FolderUpload, null=True, blank=True, related_name='parentfolder')
upload_date = models.DateTimeField(default=datetime.now(), blank=True, editable=False)
upload = models.FileField(upload_to=file_upload_path)
name = models.CharField(max_length=100)
description = models.CharField(blank=True, max_length=200)
def save(self):
if not self.id:
if self.folder == None:
pass
else:
self.path = self.folder.path
super(FileUpload, self).save()
I'm also using a form defined by
from django.forms import ChoiceField, Form, ModelForm
class FileUploadForm(ModelForm):
class Meta:
model = FileUpload
The function that takes the unzipped files on the disk. registers them
with the database, and moves them to the correct place is called
addFile. I was previously using this:
def addFile(name, filename, description, content, folder_id=None):
#f = open(filename, 'r')
#content = f.read()
from forms import FileUploadForm
from django.core.files.uploadedfile import SimpleUploadedFile
if folder_id == None:
data = {'name':name, 'description':description}
else:
data = {'name':name, 'description':description, 'folder':str(folder_id)}
file_data = {'upload': SimpleUploadedFile(filename, content)}
ff = FileUploadForm(data, file_data)
try:
zf = ff.save(commit=False)
zf.save()
except:
raise RuntimeError, "Form error is %s."%(ff.errors)
return zf
This worked, but the problem was that it dumped the entire file into
memory. With large files, and especially given Python isn't known for
it's memory economy, this consumed huge amounts of memory. So I
switched to this:
from django.core.files.uploadedfile import UploadedFile
class UnzippedFile(UploadedFile):
def __init__(self, file, filepath, content_type='text/plain', charset=None):
import os
self.filepath = filepath
self.name = os.path.basename(filepath)
self.size = os.path.getsize(filepath)
self.content_type = content_type
self.charset = charset
super(UnzippedFile, self).__init__(file, self.name, content_type, self.size, charset)
def temporary_file_path(self):
"""
Returns the full path of this file.
"""
return self.filepath
def addFile(filepath, description, file, folder_id=None):
import os, sys
from forms import FileUploadForm
from django.core.files.uploadedfile import UploadedFile
name = os.path.basename(filepath)
if folder_id == None:
data = {'name':name, 'description':description}
else:
data = {'name':name, 'description':description, 'folder':str(folder_id)}
file_data = {'upload': UnzippedFile(file, filepath)}
ff = FileUploadForm(data, file_data)
try:
zf = ff.save(commit=False)
zf.save()
except:
raise
return zf
I was forced to subclass UploadedFile, since none of the derived
classes that were already there (in
django/core/files/uploadedfile.py) seemed to do what I wanted.
The temporary_file_path function is there because the Django File Uploads docs say
UploadedFile.temporary_file_path()
Only files uploaded onto disk will have this method; it returns the
full path to the temporary uploaded file.
It seems the FileSystemStorage class looks for this attribute in
_save function as described later.
If n is the relative path of the file in the zip archive, then the
usage is
name = os.path.normpath(os.path.basename(n)) # name of file
pathname = os.path.join(dirname, n) # full file path
description = "from zip file '" + zipfilename + "'" # `zipfilename` is the name of the zip file
fname = open(pathname) # file handle
f = addFile(pathname, description, fname)
This works, but I traced through the code, and found that the code was
using streaming, when clearly the optimal thing to do in this case
would be to just copy the file from the temporary location to the
permanent location. The code in question is in
django/core/files/storage.py, in the _save function of the
FileSystemStorage class. In _save, name is the relative path of
the destination, and content is a File object.
def _save(self, name, content):
full_path = self.path(name)
directory = os.path.dirname(full_path)
[...]
while True:
try:
# This file has a file path that we can move.
if hasattr(content, 'temporary_file_path'):
file_move_safe(content.temporary_file_path(), full_path)
content.close()
# This is a normal uploadedfile that we can stream.
else:
# This fun binary flag incantation makes os.open throw an
# OSError if the file already exists before we open it.
fd = os.open(full_path, os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, 'O_BINARY', 0))
try:
locks.lock(fd, locks.LOCK_EX)
for chunk in content.chunks():
os.write(fd, chunk)
finally:
locks.unlock(fd)
os.close(fd)
except OSError, e:
if e.errno == errno.EEXIST:
# Ooops, the file exists. We need a new file name.
name = self.get_available_name(name)
full_path = self.path(name)
else:
raise
else:
# OK, the file save worked. Break out of the loop.
break
The _save function is looking for the attribute
temporary_file_path. I believe this code is intended to be triggered
by the temporary_file_path function mentioned earlier in
django/core/files/uploadedfile.py. However, the class that is
actually passed (corresponding to the content argument) is <class
'django.db.models.fields.files.FieldFile'>, and here is what the
attribute dict (content.__dict__) for this object looks like:
{'_committed': False, 'name': u'foo', 'instance': <FileUpload: foo>,
'_file': <UnzippedFile: foo (text/plain)>, 'storage':<django.core.files.storage.DefaultStorage object at 0x9a70ccc>,
'field': <django.db.models.fields.files.FileField object at0x9ce9b4c>, 'mode': None}
The temporary_file_path is attached to the
UnzippedFile class, which is inside the _file data member. So
content._file has a temporary_file_path attribute, not content
itself.
This is what a regular file upload looks like. As you can see, it is similar.
[Fri Jun 17 08:05:33 2011] [error] type of content is <class 'django.db.models.fields.files.FieldFile'>
[Fri Jun 17 08:05:33 2011] [error] {'_committed': False, 'name': u'behavior.py',
'instance': <FileUpload: b>, '_file': <TemporaryUploadedFile: behavior.py (text/x-python)>,
'storage': <django.core.files.storage.DefaultStorage object at 0xb8d7fd8c>,
'field': <django.db.models.fields.files.FileField object at 0xb8eb584c>, 'mode': None}
It is difficult for me to follow in any detail how the code gets from
the FileUploadForm save to the Storage object. The Django form
code in particular is quite obscure.
Anyway, my question, after all this setup is, how/when is the first
option below, with file_move_safe supposed to be activated? I'm
seeing a mismatch here. Is this a bug? Can anyone clarify?
if hasattr(content, 'temporary_file_path')
The above will never equal true with this conditional since you state that content does not have a temporary_file_path identifier. However since content._file does you can use the following to get the functionality you are looking for
if hasattr(content, '_file'):
if hasattr(content._file,'temporary_file_path'):
I'm using the following code to serve uploaded files from a login secured view in a django app.
Do you think that there is a security vulnerability in this code? I'm a bit concerned about that the user could place arbitrary strings in the url after the upload/ and this is directly mapped to the local filesystem.
Actually I don't think that it is a vulnerability issue, since the access to the filesystem is restricted to the files in the folder defined with the UPLOAD_LOCATION setting.
UPLOAD_LOCATION = is set to a not publicly available folder on the webserver
url(r'^upload/(?P<file_url>[/,.,\s,_,\-,\w]+)', 'project_name.views.serve_upload_files', name='project_detail'),
#login_required
def serve_upload_files(request, file_url):
import os.path
import mimetypes
mimetypes.init()
try:
file_path = settings.UPLOAD_LOCATION + '/' + file_url
fsock = open(file_path,"r")
file_name = os.path.basename(file_path)
file_size = os.path.getsize(file_path)
print "file size is: " + str(file_size)
mime_type_guess = mimetypes.guess_type(file_name)
if mime_type_guess is not None:
response = HttpResponse(fsock, mimetype=mime_type_guess[0])
response['Content-Disposition'] = 'attachment; filename=' + file_name
#response.write(file)
except IOError:
response = HttpResponseNotFound()
return response
EDIT: Updated the source according Ignacio Vazquez-Abrams comments:
import os.path
import mimetypes
#login_required
def serve_upload_files(request, file_url):
mimetypes.init()
try:
file_path = os.path.join(settings.UPLOAD_LOCATION, file_url)
#collapse possibly available up-level references
file_path = os.path.normpath(file_path)
#check if file path still begins with settings.UPLOAD_LOCATION, otherwise the user tampered around with up-level references in the url
#for example this input: http://127.0.0.1:8000/upload/..\test_upload.txt results having the user access to a folder one-level higher than the upload folder
#AND check if the common_prefix ends with a dir separator, Because although '/foo/barbaz' starts with '/foo/bar'
common_prefix = os.path.commonprefix([settings.UPLOAD_LOCATION, file_path])
if common_prefix == settings.UPLOAD_LOCATION and common_prefix.endswith(os.sep):
fsock = open(file_path,"r")
file_name = os.path.basename(file_path)
mime_type_guess = mimetypes.guess_type(file_name)
if mime_type_guess is not None:
response = HttpResponse(fsock, mimetype=mime_type_guess[0])
response['Content-Disposition'] = 'attachment; filename=' + file_name
else:
response = HttpResponseNotFound()
else:
print "wrong directory"
response = HttpResponseNotFound()
except IOError:
response = HttpResponseNotFound()
return response
A few tips:
Use os.path.join() to join the path together.
Use os.path.normpath() to get the actual path with no ".." references.
Use os.path.commonprefix() against UPLOAD_LOCATION and the generated path, and verify that the result starts with UPLOAD_LOCATION.
Make sure that UPLOAD_LOCATION ends with a dir separator.
TL;DR: Use os.path.