I want to upload a file with filename as email of the uploader. I know this is not a OS file system issue, nor a unsupported python operation because I did save images with this format in python. Now I want to save the images in model(Specifically ImageField).
I'm using OverwriteStorage to overwrite images if there is some existing image with same name.
Saving the image
model = Model.objects.create(email=email)
# Blob to image
tempfile_io = io.BytesIO()
img.save(tempfile_io, 'JPEG')
image_file = InMemoryUploadedFile(tempfile_io, None, email + ".jpeg",'image/jpeg',tempfile_io.getbuffer().nbytes, None)
print("email is", email)
model.pic.save(email + ".jpg",image_file)
Model
class Model(models.Model):
email = models.EmailField()
pic = models.ImageField(upload_to="pics", storage=OverwriteStorage())
OverwriteStorage
class OverwriteStorage(FileSystemStorage):
def get_available_name(self, name, *args, **kwargs):
print("name is", name)
if self.exists(name):
self.delete(name)
return name
But for some reason, I don't get the exact name in OverwriteStorage-get_available_name.
email is xyz#example.com
name is xyzexamle.com.jpg
Notice how the # sign is brutally removed.
Is there any file name string check i need to disable. How can I make Django use the exact file name given(whenever possible)?
You will have to override get_valid_name(name) as documented
get_valid_name(name)
Returns a filename suitable for use with the underlying storage
system. The name argument passed to this method is either the original
filename sent to the server or, if upload_to is a callable, the
filename returned by that method after any path information is
removed. Override this to customize how non-standard characters are
converted to safe filenames.
From source
def get_valid_name(self, name):
"""
Return a filename, based on the provided filename, that's suitable for
use in the target storage system.
"""
return get_valid_filename(name)
Related
I am building a Django project where users can upload pictures. I am wondering what I should do to not show the original picture name.
I want the url to be something like /pic/randomnumber, and when the picture is downloaded from the website, it would have the name randomnumber.jpg. For example, all the pictures on Tumblr have the name tumblr_blabla.jpg.
I think this is something that should be done in models.py, but I am not quite sure how to implement it.
IMO you should write method save in your model
Something like that:
from PIL import Image
import os
class YOURS_MODEL_NAME(models.Model):
photo = models.ImageField(upload_to="photos")
def save(self, miniature=True):
super(YOURS_MODEL_NAME, self).save()
if miniature:
filepath = self.photo.path
image = Image.open(filepath)
new_filepath = filepath.split('.')
new_filepath = '.'.join("HERE YOU CAN ADD EVERYTHING TO PATH TO THIS PHOTO") + "." + new_filepath[-1].lower()
try:
image.save(new_filepath, quality=90, optimize=1)
except:
image.save(new_filepath, quality=90)
photo_name = self.photo.name.split('.')
photo_name = '.'.join("HERE YOU CAN ADD EVERYTHING YOU WANT TO 'PHOTO NAME'") + "." + photo_name[-1].lower()
self.photo = photo_name
self.save(miniature=False)
# remove old image
os.remove(filepath)
The upload_to argument in your Model definition can be a callable function which you use to customize the name of the file. Taken from the Django docs on
FileField (of which ImageField is a subclass):
upload_to takes two arguments: instance and filename, (where filename is the original filename, which you may also chose to ignore).
Something similar to this in models.py should do the trick:
def random_filename(instance, filename):
file_name = "random_string" # use your choice for generating a random string!
return file_name
class SomeModel(models.Model):
file = models.ImageField(upload_to=random_filename)
(this is similar to the answer this question about FileFields).
If you are going down this path, I would recommend that you use either the hash/checksum or date/time of the file upload. Something along these lines should work (although I haven't tested it myself!):
from hashlib import sha1
def unique_filename(instance, field):
filehash = sha1()
for chunk in getattr(instance, field).chunks():
filehash.update(chunk)
return filehash
class SomeModel(models.Model):
file = models.ImageField(upload_to=unique_filename(field='file'))
Hope this helps!
[I apologize in advance for the length of this question.]
I'm using Django 1.2.3-3+squeeze1 on Debian squeeze.
I am writing an application which uploads zip files to disk in a
temporary location, unzips them, and then saves the results to a
permanent location. The unzipped files are registered in the database as a class called
FileUpload after they are unzipped. The uploaded zipped files also correspond to a class,
but I'll ignore that for the purposes of this question. FileUpload looks like this.
class FileUpload(models.Model):
folder = models.ForeignKey(FolderUpload, null=True, blank=True, related_name='parentfolder')
upload_date = models.DateTimeField(default=datetime.now(), blank=True, editable=False)
upload = models.FileField(upload_to=file_upload_path)
name = models.CharField(max_length=100)
description = models.CharField(blank=True, max_length=200)
def save(self):
if not self.id:
if self.folder == None:
pass
else:
self.path = self.folder.path
super(FileUpload, self).save()
I'm also using a form defined by
from django.forms import ChoiceField, Form, ModelForm
class FileUploadForm(ModelForm):
class Meta:
model = FileUpload
The function that takes the unzipped files on the disk. registers them
with the database, and moves them to the correct place is called
addFile. I was previously using this:
def addFile(name, filename, description, content, folder_id=None):
#f = open(filename, 'r')
#content = f.read()
from forms import FileUploadForm
from django.core.files.uploadedfile import SimpleUploadedFile
if folder_id == None:
data = {'name':name, 'description':description}
else:
data = {'name':name, 'description':description, 'folder':str(folder_id)}
file_data = {'upload': SimpleUploadedFile(filename, content)}
ff = FileUploadForm(data, file_data)
try:
zf = ff.save(commit=False)
zf.save()
except:
raise RuntimeError, "Form error is %s."%(ff.errors)
return zf
This worked, but the problem was that it dumped the entire file into
memory. With large files, and especially given Python isn't known for
it's memory economy, this consumed huge amounts of memory. So I
switched to this:
from django.core.files.uploadedfile import UploadedFile
class UnzippedFile(UploadedFile):
def __init__(self, file, filepath, content_type='text/plain', charset=None):
import os
self.filepath = filepath
self.name = os.path.basename(filepath)
self.size = os.path.getsize(filepath)
self.content_type = content_type
self.charset = charset
super(UnzippedFile, self).__init__(file, self.name, content_type, self.size, charset)
def temporary_file_path(self):
"""
Returns the full path of this file.
"""
return self.filepath
def addFile(filepath, description, file, folder_id=None):
import os, sys
from forms import FileUploadForm
from django.core.files.uploadedfile import UploadedFile
name = os.path.basename(filepath)
if folder_id == None:
data = {'name':name, 'description':description}
else:
data = {'name':name, 'description':description, 'folder':str(folder_id)}
file_data = {'upload': UnzippedFile(file, filepath)}
ff = FileUploadForm(data, file_data)
try:
zf = ff.save(commit=False)
zf.save()
except:
raise
return zf
I was forced to subclass UploadedFile, since none of the derived
classes that were already there (in
django/core/files/uploadedfile.py) seemed to do what I wanted.
The temporary_file_path function is there because the Django File Uploads docs say
UploadedFile.temporary_file_path()
Only files uploaded onto disk will have this method; it returns the
full path to the temporary uploaded file.
It seems the FileSystemStorage class looks for this attribute in
_save function as described later.
If n is the relative path of the file in the zip archive, then the
usage is
name = os.path.normpath(os.path.basename(n)) # name of file
pathname = os.path.join(dirname, n) # full file path
description = "from zip file '" + zipfilename + "'" # `zipfilename` is the name of the zip file
fname = open(pathname) # file handle
f = addFile(pathname, description, fname)
This works, but I traced through the code, and found that the code was
using streaming, when clearly the optimal thing to do in this case
would be to just copy the file from the temporary location to the
permanent location. The code in question is in
django/core/files/storage.py, in the _save function of the
FileSystemStorage class. In _save, name is the relative path of
the destination, and content is a File object.
def _save(self, name, content):
full_path = self.path(name)
directory = os.path.dirname(full_path)
[...]
while True:
try:
# This file has a file path that we can move.
if hasattr(content, 'temporary_file_path'):
file_move_safe(content.temporary_file_path(), full_path)
content.close()
# This is a normal uploadedfile that we can stream.
else:
# This fun binary flag incantation makes os.open throw an
# OSError if the file already exists before we open it.
fd = os.open(full_path, os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, 'O_BINARY', 0))
try:
locks.lock(fd, locks.LOCK_EX)
for chunk in content.chunks():
os.write(fd, chunk)
finally:
locks.unlock(fd)
os.close(fd)
except OSError, e:
if e.errno == errno.EEXIST:
# Ooops, the file exists. We need a new file name.
name = self.get_available_name(name)
full_path = self.path(name)
else:
raise
else:
# OK, the file save worked. Break out of the loop.
break
The _save function is looking for the attribute
temporary_file_path. I believe this code is intended to be triggered
by the temporary_file_path function mentioned earlier in
django/core/files/uploadedfile.py. However, the class that is
actually passed (corresponding to the content argument) is <class
'django.db.models.fields.files.FieldFile'>, and here is what the
attribute dict (content.__dict__) for this object looks like:
{'_committed': False, 'name': u'foo', 'instance': <FileUpload: foo>,
'_file': <UnzippedFile: foo (text/plain)>, 'storage':<django.core.files.storage.DefaultStorage object at 0x9a70ccc>,
'field': <django.db.models.fields.files.FileField object at0x9ce9b4c>, 'mode': None}
The temporary_file_path is attached to the
UnzippedFile class, which is inside the _file data member. So
content._file has a temporary_file_path attribute, not content
itself.
This is what a regular file upload looks like. As you can see, it is similar.
[Fri Jun 17 08:05:33 2011] [error] type of content is <class 'django.db.models.fields.files.FieldFile'>
[Fri Jun 17 08:05:33 2011] [error] {'_committed': False, 'name': u'behavior.py',
'instance': <FileUpload: b>, '_file': <TemporaryUploadedFile: behavior.py (text/x-python)>,
'storage': <django.core.files.storage.DefaultStorage object at 0xb8d7fd8c>,
'field': <django.db.models.fields.files.FileField object at 0xb8eb584c>, 'mode': None}
It is difficult for me to follow in any detail how the code gets from
the FileUploadForm save to the Storage object. The Django form
code in particular is quite obscure.
Anyway, my question, after all this setup is, how/when is the first
option below, with file_move_safe supposed to be activated? I'm
seeing a mismatch here. Is this a bug? Can anyone clarify?
if hasattr(content, 'temporary_file_path')
The above will never equal true with this conditional since you state that content does not have a temporary_file_path identifier. However since content._file does you can use the following to get the functionality you are looking for
if hasattr(content, '_file'):
if hasattr(content._file,'temporary_file_path'):
Can someone illuminate me as to exactly when an uploaded file is actually written to the location returned by "upload_to" in the FileField, in particular with regards to the order of field, model, and form validation and cleaning?
Right now I have a "clean" method on my model which assumes the uploaded file is in place, so it can do some validation on it. It looks like the file isn't yet saved, and may just be held in a temporary location or in memory. If that is the case, how do I "open" it or find a path to it if I need to execute some external process/program to validate the file?
Thanks,
Ian
The form cleansing has nothing to do with actually saving the file, or with saving any other data for that matter. The file isn't saved until to you run the save() method of the model instance (note that if you use ModelName.objects.create() this save() method is called for you automatically).
The bound form will contain an open File object, so you should be able to do any validation on that object directly. For example:
form = MyForm(request.POST, request.FILES)
if form.is_valid():
file_object = form.cleaned_data['myFile']
#run any validation on the file_object, or define a clean_myFile() method
# that will be run automatically when you call form.is_valid()
model_inst = MyModel('my_file' = file_object,
#assign other attributes here....
)
model_inst.save() #file is saved to disk here
What do you need to do on it? If your validation will work without a temporary file, you can access the data by calling read() on what your file field returns.
def clean_field(self):
_file = self.cleaned_data.get('filefield')
contents = _file.read()
If you do need it on the disk, you know where to go from here :) write it to a temporary location and do some magic on it!
Or write it as a custom form field. This is the basic idea how I go about verification of an MP3 file using the 'mutagen' library.
Notes:
first check the file size then if correct size write to tmp location.
Will write the file to temporary location specified in SETTINGS check its MP3 and then delete it.
The code:
from django import forms
import os
from mutagen.mp3 import MP3, HeaderNotFoundError, InvalidMPEGHeader
from django.conf import settings
class MP3FileField(forms.FileField):
def clean(self, *args, **kwargs):
super(MP3FileField, self).clean(*args, **kwargs)
tmp_file = args[0]
if tmp_file.size > 6600000:
raise forms.ValidationError("File is too large.")
file_path = getattr(settings,'FILE_UPLOAD_TEMP_DIR')+'/'+tmp_file.name
destination = open(file_path, 'wb+')
for chunk in tmp_file.chunks():
destination.write(chunk)
destination.close()
try:
audio = MP3(file_path)
if audio.info.length > 300:
os.remove(file_path)
raise forms.ValidationError("MP3 is too long.")
except (HeaderNotFoundError, InvalidMPEGHeader):
os.remove(file_path)
raise forms.ValidationError("File is not valid MP3 CBR/VBR format.")
os.remove(file_path)
return args
I've got a model like this:
class Talk(BaseModel):
title = models.CharField(max_length=200)
mp3 = models.FileField(upload_to = u'talks/', max_length=200)
seconds = models.IntegerField(blank = True, null = True)
I want to validate before saving that the uploaded file is an MP3, like this:
def is_mp3(path_to_file):
from mutagen.mp3 import MP3
audio = MP3(path_to_file)
return not audio.info.sketchy
Once I'm sure I've got an MP3, I want to save the length of the talk in the seconds attribute, like this:
audio = MP3(path_to_file)
self.seconds = audio.info.length
The problem is, before saving, the uploaded file doesn't have a path (see this ticket, closed as wontfix), so I can't process the MP3.
I'd like to raise a nice validation error so that ModelForms can display a helpful error ("You idiot, you didn't upload an MP3" or something).
Any idea how I can go about accessing the file before it's saved?
p.s. If anyone knows a better way of validating files are MP3s I'm all ears - I also want to be able to mess around with ID3 data (set the artist, album, title and probably album art, so I need it to be processable by mutagen).
You can access the file data in request.FILES while in your view.
I think that best way is to bind uploaded files to a form, override the forms clean method, get the UploadedFile object from cleaned_data, validate it anyway you like, then override the save method and populate your models instance with information about the file and then save it.
a cleaner way to get the file before be saved is like this:
from django.core.exceptions import ValidationError
#this go in your class Model
def clean(self):
try:
f = self.mp3.file #the file in Memory
except ValueError:
raise ValidationError("A File is needed")
f.__class__ #this prints <class 'django.core.files.uploadedfile.InMemoryUploadedFile'>
processfile(f)
and if we need a path, ther answer is in this other question
You could follow the technique used by ImageField where it validates the file header and then seeks back to the start of the file.
class ImageField(FileField):
# ...
def to_python(self, data):
f = super(ImageField, self).to_python(data)
# ...
# We need to get a file object for Pillow. We might have a path or we might
# have to read the data into memory.
if hasattr(data, 'temporary_file_path'):
file = data.temporary_file_path()
else:
if hasattr(data, 'read'):
file = BytesIO(data.read())
else:
file = BytesIO(data['content'])
try:
# ...
except Exception:
# Pillow doesn't recognize it as an image.
six.reraise(ValidationError, ValidationError(
self.error_messages['invalid_image'],
code='invalid_image',
), sys.exc_info()[2])
if hasattr(f, 'seek') and callable(f.seek):
f.seek(0)
return f
I would like to create dynamically the destination of my uploaded files.
But, it seems that the 'upload_to' option is only available for a models, not for forms. So the following code is wrong.
class MyForm(forms.Form):
fichier = forms.FileField(**upload_to='files/%m-%Y/'**)
In the view handling the uploaded file, the destination is static. How can I make it dynamic ?
Thank you.
class YourFileModel(models.Model)
def upload_path(self, name):
name = do_sth_with_name(name)
folder = generate_folder_name(self.id, self.whatever_field)
return 'uploads/' + folder + '/' + name
file = models.FileField(upload_to=upload_path)
edit after comment
def handle_uploaded_file(file):
# generate dynamic path
# save file to that path
example here http://docs.djangoproject.com/en/dev/topics/http/file-uploads/#handling-uploaded-files
if form from model, override the save() method
class YourForm(forms.ModelForm):
fichier = forms.FileField()
def save(self):
if self.cleaned_data['fichier']:
file = handle_uploaded_file(self.cleaned_data['fichier'])
super(YourForm, self).save()
if not form from model, call the upload handler in your view
def your_view(request):
#####
if form.is_valid():
file = handle_uploaded_file(form.cleaned_data['fichier'])
Instead of a string, supply a callable -- i.e. the name of a function that takes the model instance and a string, and returns the desired name. See FileField docs for specifics. One thing they don't say (at least I can't find it in the docs) is that if the returned filename starts with '/' then it is an absolute path, otherwise it is relative to your /media directory.