I have changed upload_to attribute of my models image field. How can I re-upload all images to new paths?
So, I think 're-uploading' is the wrong way to think about it -- re-uploading the images will still leave the old ones lying around, which (depending on how many images you have) could be a massive waste of space. One way to do this instead would be by the following two step process:
1) Move the files manually, on your server, to the new upload_to location via whatever method is OS appropriate. This could probably all be done with one mv command on linux, if that's what you're hosting on.
2) If you just changed the upload_to attribute, and didn't change the MEDIA_ROOT settings or anything else, what you need to change is the Imagefield's name property. An ImageField's name properly usually is a joining of your upload_to string and your image's filename (this then gets appended to MEDIA_URL to form the images url or MEDIA_ROOT to form the actual upload path). So you could update the models in your Django shell by typing something like this:
import os
from my_app import MyModel
newpath = 'your/new/upload_to/'
for obj in MyModel.objects.all():
image_name = os.path.split(obj.my_img_field.name)[1]
obj.my_img_field.name = newpath + image_name
obj.save()
You can check to see if everything worked properly by calling obj.my_img_field.url and seeing if that points where it should.
Here's a little snippet that I made when I needed to do this on many models and didn't want to do this on th OS level.
For use with strftime this have to be modified though.
models = (YourModel1, YourModel2)
for Model in models:
for field in Model._meta.get_fields():
if not hasattr(field, 'upload_to'):
continue
for instance in Model.objects.all():
f = getattr(instance, field.name)
if not f:
continue
if field.upload_to not in str(f):
filename = os.path.basename(f.name)
new_path = os.path.join(field.upload_to, filename)
os.makedirs(
os.path.join(
settings.MEDIA_ROOT,
field.upload_to
),
exist_ok=True
)
try:
shutil.move(
os.path.join(settings.MEDIA_ROOT, f.name),
os.path.join(settings.MEDIA_ROOT, new_path)
)
setattr(instance, field.name, new_path)
except FileNotFoundError as e:
logger.error("Not found {}".format(field.name))
logger.error(str(e))
else:
instance.save()
Related
I am building a Django project where users can upload pictures. I am wondering what I should do to not show the original picture name.
I want the url to be something like /pic/randomnumber, and when the picture is downloaded from the website, it would have the name randomnumber.jpg. For example, all the pictures on Tumblr have the name tumblr_blabla.jpg.
I think this is something that should be done in models.py, but I am not quite sure how to implement it.
IMO you should write method save in your model
Something like that:
from PIL import Image
import os
class YOURS_MODEL_NAME(models.Model):
photo = models.ImageField(upload_to="photos")
def save(self, miniature=True):
super(YOURS_MODEL_NAME, self).save()
if miniature:
filepath = self.photo.path
image = Image.open(filepath)
new_filepath = filepath.split('.')
new_filepath = '.'.join("HERE YOU CAN ADD EVERYTHING TO PATH TO THIS PHOTO") + "." + new_filepath[-1].lower()
try:
image.save(new_filepath, quality=90, optimize=1)
except:
image.save(new_filepath, quality=90)
photo_name = self.photo.name.split('.')
photo_name = '.'.join("HERE YOU CAN ADD EVERYTHING YOU WANT TO 'PHOTO NAME'") + "." + photo_name[-1].lower()
self.photo = photo_name
self.save(miniature=False)
# remove old image
os.remove(filepath)
The upload_to argument in your Model definition can be a callable function which you use to customize the name of the file. Taken from the Django docs on
FileField (of which ImageField is a subclass):
upload_to takes two arguments: instance and filename, (where filename is the original filename, which you may also chose to ignore).
Something similar to this in models.py should do the trick:
def random_filename(instance, filename):
file_name = "random_string" # use your choice for generating a random string!
return file_name
class SomeModel(models.Model):
file = models.ImageField(upload_to=random_filename)
(this is similar to the answer this question about FileFields).
If you are going down this path, I would recommend that you use either the hash/checksum or date/time of the file upload. Something along these lines should work (although I haven't tested it myself!):
from hashlib import sha1
def unique_filename(instance, field):
filehash = sha1()
for chunk in getattr(instance, field).chunks():
filehash.update(chunk)
return filehash
class SomeModel(models.Model):
file = models.ImageField(upload_to=unique_filename(field='file'))
Hope this helps!
I have written an email form class in Django with a FileField. I want to check the uploaded file for its type via checking its mimetype. Subsequently, I want to limit file types to pdfs, word, and open office documents.
To this end, I have installed python-magic and would like to check file types as follows per the specs for python-magic:
mime = magic.Magic(mime=True)
file_mime_type = mime.from_file('address/of/file.txt')
However, recently uploaded files lack addresses on my server. I also do not know of any method of the mime object akin to "from_file_content" that checks for the mime type given the content of the file.
What is an effective way to use magic to verify file types of uploaded files in Django forms?
Stan described good variant with buffer. Unfortunately the weakness of this method is reading file to the memory. Another option is using temporary stored file:
import tempfile
import magic
with tempfile.NamedTemporaryFile() as tmp:
for chunk in form.cleaned_data['file'].chunks():
tmp.write(chunk)
print(magic.from_file(tmp.name, mime=True))
Also, you might want to check the file size:
if form.cleaned_data['file'].size < ...:
print(magic.from_buffer(form.cleaned_data['file'].read()))
else:
# store to disk (the code above)
Additionally:
Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).
So you might want to handle it like so:
import os
tmp = tempfile.NamedTemporaryFile(delete=False)
try:
for chunk in form.cleaned_data['file'].chunks():
tmp.write(chunk)
print(magic.from_file(tmp.name, mime=True))
finally:
os.unlink(tmp.name)
tmp.close()
Also, you might want to seek(0) after read():
if hasattr(f, 'seek') and callable(f.seek):
f.seek(0)
Where uploaded data is stored
Why no trying something like that in your view :
m = magic.Magic()
m.from_buffer(request.FILES['my_file_field'].read())
Or use request.FILES in place of form.cleaned_data if django.forms.Form is really not an option.
mime = magic.Magic(mime=True)
attachment = form.cleaned_data['attachment']
if hasattr(attachment, 'temporary_file_path'):
# file is temporary on the disk, so we can get full path of it.
mime_type = mime.from_file(attachment.temporary_file_path())
else:
# file is on the memory
mime_type = mime.from_buffer(attachment.read())
Also, you might want to seek(0) after read():
if hasattr(f, 'seek') and callable(f.seek):
f.seek(0)
Example from Django code. Performed for image fields during validation.
You can use django-safe-filefield package to validate that uploaded file extension match it MIME-type.
from safe_filefield.forms import SafeFileField
class MyForm(forms.Form):
attachment = SafeFileField(
allowed_extensions=('xls', 'xlsx', 'csv')
)
In case you're handling a file upload and concerned only about images,
Django will set content_type for you (or rather for itself?):
from django.forms import ModelForm
from django.core.files import File
from django.db import models
class MyPhoto(models.Model):
photo = models.ImageField(upload_to=photo_upload_to, max_length=1000)
class MyForm(ModelForm):
class Meta:
model = MyPhoto
fields = ['photo']
photo = MyPhoto.objects.first()
photo = File(open('1.jpeg', 'rb'))
form = MyForm(files={'photo': photo})
if form.is_valid():
print(form.instance.photo.file.content_type)
It doesn't rely on content type provided by the user. But
django.db.models.fields.files.FieldFile.file is an undocumented
property.
Actually, initially content_type is set from the request, but when
the form gets validated, the value is updated.
Regarding non-images, doing request.FILES['name'].read() seems okay to me.
First, that's what Django does. Second, files larger than 2.5 Mb by default
are stored on a disk. So let me point you at the other answer
here.
For the curious, here's the stack trace that leads to updating
content_type:
django.forms.forms.BaseForm.is_valid: self.errors
django.forms.forms.BaseForm.errors: self.full_clean()
django.forms.forms.BaseForm.full_clean: self._clean_fields()
django.forms.forms.BaseForm._clean_fiels: field.clean()
django.forms.fields.FileField.clean: super().clean()
django.forms.fields.Field.clean: self.to_python()
django.forms.fields.ImageField.to_python
I have been playing around with django-photologue for a while, and find this a great alternative to all other image handlings apps out there.
One thing though, I also use django-cumulus to push my uploads to my CDN instead of running it on my local machine / server.
When I used imagekit, I could always pass a upload_to='whatever' but I cannot seem to do this with photologue as it automatically inserts the imagefield. How would I go about achieving some sort of an overwrite?
Regards
I think you can hook into the pre_save signal of the Photo model, and change the upload_to field, just before the instance is saved to the database.
Take a look at this:
http://docs.djangoproject.com/en/dev/topics/signals/
Managed to find a workaround for it, however this requires you to make the changes in photologue/models.py
if PHOTOLOGUE_PATH is not None:
if callable(PHOTOLOGUE_PATH):
get_storage_path = PHOTOLOGUE_PATH
else:
parts = PHOTOLOGUE_PATH.split('.')
module_name = '.'.join(parts[:-1])
module = __import__(module_name)
get_storage_path = getattr(module, parts[-1])
else:
def get_storage_path(instance, filename):
dirname = 'photos'
if hasattr(instance, 'upload_to'):
if callable(instance.upload_to):
dirname = instance.upload_to()
else: dirname = instance.upload_to
return os.path.join(PHOTOLOGUE_DIR,
os.path.normpath( force_unicode(datetime.now().strftime(smart_str(dirname))) ),
filename)
And then in your apps models do the following:
class MyModel(ImageModel):
text = ...
name = ...
def upload_to(self):
return 'yourdirectorynamehere'
You can use the setting PHOTOLOGUE_PATH to provide your own callable. Define a method which takes instance and filename as parameters then return whatever you want. For example in your settings.py:
import photologue
...
def PHOTOLOGUE_PATH(instance, filename):
folder = 'myphotos' # Add your logic here
return os.path.join(photologue.models.PHOTOLOGUE_DIR, folder, filename)
Presumably (although I've not tested this) you could find out more about the Photo instance (e.g. what other instances it relates to) and choose the folder accordingly.
I've got a model like this:
class Talk(BaseModel):
title = models.CharField(max_length=200)
mp3 = models.FileField(upload_to = u'talks/', max_length=200)
seconds = models.IntegerField(blank = True, null = True)
I want to validate before saving that the uploaded file is an MP3, like this:
def is_mp3(path_to_file):
from mutagen.mp3 import MP3
audio = MP3(path_to_file)
return not audio.info.sketchy
Once I'm sure I've got an MP3, I want to save the length of the talk in the seconds attribute, like this:
audio = MP3(path_to_file)
self.seconds = audio.info.length
The problem is, before saving, the uploaded file doesn't have a path (see this ticket, closed as wontfix), so I can't process the MP3.
I'd like to raise a nice validation error so that ModelForms can display a helpful error ("You idiot, you didn't upload an MP3" or something).
Any idea how I can go about accessing the file before it's saved?
p.s. If anyone knows a better way of validating files are MP3s I'm all ears - I also want to be able to mess around with ID3 data (set the artist, album, title and probably album art, so I need it to be processable by mutagen).
You can access the file data in request.FILES while in your view.
I think that best way is to bind uploaded files to a form, override the forms clean method, get the UploadedFile object from cleaned_data, validate it anyway you like, then override the save method and populate your models instance with information about the file and then save it.
a cleaner way to get the file before be saved is like this:
from django.core.exceptions import ValidationError
#this go in your class Model
def clean(self):
try:
f = self.mp3.file #the file in Memory
except ValueError:
raise ValidationError("A File is needed")
f.__class__ #this prints <class 'django.core.files.uploadedfile.InMemoryUploadedFile'>
processfile(f)
and if we need a path, ther answer is in this other question
You could follow the technique used by ImageField where it validates the file header and then seeks back to the start of the file.
class ImageField(FileField):
# ...
def to_python(self, data):
f = super(ImageField, self).to_python(data)
# ...
# We need to get a file object for Pillow. We might have a path or we might
# have to read the data into memory.
if hasattr(data, 'temporary_file_path'):
file = data.temporary_file_path()
else:
if hasattr(data, 'read'):
file = BytesIO(data.read())
else:
file = BytesIO(data['content'])
try:
# ...
except Exception:
# Pillow doesn't recognize it as an image.
six.reraise(ValidationError, ValidationError(
self.error_messages['invalid_image'],
code='invalid_image',
), sys.exc_info()[2])
if hasattr(f, 'seek') and callable(f.seek):
f.seek(0)
return f
I would like to create dynamically the destination of my uploaded files.
But, it seems that the 'upload_to' option is only available for a models, not for forms. So the following code is wrong.
class MyForm(forms.Form):
fichier = forms.FileField(**upload_to='files/%m-%Y/'**)
In the view handling the uploaded file, the destination is static. How can I make it dynamic ?
Thank you.
class YourFileModel(models.Model)
def upload_path(self, name):
name = do_sth_with_name(name)
folder = generate_folder_name(self.id, self.whatever_field)
return 'uploads/' + folder + '/' + name
file = models.FileField(upload_to=upload_path)
edit after comment
def handle_uploaded_file(file):
# generate dynamic path
# save file to that path
example here http://docs.djangoproject.com/en/dev/topics/http/file-uploads/#handling-uploaded-files
if form from model, override the save() method
class YourForm(forms.ModelForm):
fichier = forms.FileField()
def save(self):
if self.cleaned_data['fichier']:
file = handle_uploaded_file(self.cleaned_data['fichier'])
super(YourForm, self).save()
if not form from model, call the upload handler in your view
def your_view(request):
#####
if form.is_valid():
file = handle_uploaded_file(form.cleaned_data['fichier'])
Instead of a string, supply a callable -- i.e. the name of a function that takes the model instance and a string, and returns the desired name. See FileField docs for specifics. One thing they don't say (at least I can't find it in the docs) is that if the returned filename starts with '/' then it is an absolute path, otherwise it is relative to your /media directory.