We use Python Django.
I want to store user-uploaded files in the following way:
/<INTEGER>/<HASH>/<FILE>
<INTEGER> is a subfolder which is meant to contain max 1000 files.
<HASH> is a secret code individual for each file, to prevent hackers to download a somebody's other file from our HTTPS server.
<FILE> is the filename.
How to write a Django storage implementing this?
First, can I derive my class from django.core.files.storage.FileSystemStorage or I need to derived from django.core.files.storage.Storage directly?
Second, which methods should I override? Should I override save? It seems yes, because it is best to generate the hash only when saving, but I am unsure I can do in save what I need. What about overriding _save? Also: what's about overriding get_available_name and/or generate_filename? They seem not be quite tailored to my needs, because it would be better to generate the hash only when actually saving the file, not just when considering its name.
So, what to do?
I would go for generate_filename in django.core.files.storage.FileSystemStorage. I find it more appropriate since than I still have the nice safety cushion in get_available_name.
I also wouldn't worry about getting filename in cases when the file is not being saved, just need to make sure that the hash generating is cheap enough.
# settings.py
DEFAULT_FILE_STORAGE = 'app.storage.CustomStorage'
# app/storage.py
from datetime import datetime
import hashlib
import random
from django.core.files.storage import FileSystemStorage
class CustomStorage(FileSystemStorage):
def generate_filename(self, filename):
# implement smarter hash creation method here
some_hash = hashlib.md5('{}-{}'.format('filename', datetime.now().isoformat())).hexdigest()
return '{0}/{1}/{2}'.format(random.randint(0, 1000), some_hash, filename)
i guess you can have your own function to handle the way you want to create folder structure and you can have in models.py as attachment = models.FileField(upload_to=change_file_path, blank=True, null=True)
where change_file_path is your custom function to create the desired folder structure
Using both storage and upload_to arguments for setting the upload directory.More info FileField django docs
Here I am using uuid as <HASH> (you could change using your own hash generator),max_num is <integer> and senfile is <FILE> in Django model
models.py
from django.core.files.storage import FileSystemStorage
import uuid
storeFile = FileSystemStorage(location='SensitiveFiles/')
def content_file_upload(instance, filename):
return '{0}/{1}/{2}'.format(instance.max_num,str(uuid.uuid4()), filename)
class SensitiveFiles(models.Model):
user = models.ForeignKey(User)
senfile = models.FileField(storage=storeFile,upload_to=content_file_upload,validators=[validation_for_max_num_files])
max_num=models.IntegerField(default=1000)
def get_file_url(self):
return self.senfile.url
Create your validators for checking current number of files in uploaded directory.
Related
I have a model which has an ImageField.
class A(model.Model):
blah = CharField(max_length=10)
profile = ImageField(upload_to='uploads/', null=True, blank=True)
I want to set a default image for existing data, so I tried saving the file in shell like the following:
>>> myapp.models import A
>>> from django.core.files import File
>>> for a in A.objects.all():
>>> a.profile.save('default.jpg', File(open('/Users/myusername/Pictures/default.jpg', 'rb')))
I was quite happy, until I found out that in myprojectroot/media/uploads/ there are as many defaultblablah.jpg files as the number of A objects.
I'd like to know if there's any way to set the same image file with all objects as you do with update() method.
Could someone shed some lights on it?
Thank you in advance.
Yes, you can store a file default.jpg in the uploads/ directory, and then update the objects with:
A.objects.filter(profile=None).update(profile='uploads/default.jpg')
I would also advise to make the field non-nullable, and use as default an value:
class A(model.Model):
blah = CharField(max_length=10)
profile = ImageField(
upload_to='uploads/',
default='uploads/default.jpg',
blank=True
)
Behind the curtains an ImageField is a varchar field in the database, that stores a path to the image you use. The Django ImageField has extra logic such that it presents it as a FieldFile [Django-doc] with some logic, but it is stored in the database as a string.
The reason why it will generate multiple images, is because you can later decide to edit one of the images. If that is the case, all of the other images would see these edits. So if the filename "clashes", it will make copies to prevent this. If you however make use of a default image, the idea is to not modify that image. Therefore it might also be a good idea to mark the file as readonly.
We need to store a few smallish files to the database (yes, I'm well aware of the counterarguments, but setting up e.g. FileField to work in several environments seems very tedious for a couple of files, and having files on the database will also solve backup requirements).
However, I was surprised to find out that even though BinaryField can be set editable, Django Admin does not create a file upload widget for it.
The only functionality we need for the BinaryField is the possibility to upload a file and replace the existing file. Other than that, the Django Admin fulfills all our requirements.
How can we do this modification to Django Admin?
You will want to create a custom Widget specifically for BinaryField which has to read the file contents before putting them into the database.
class BinaryFileInput(forms.ClearableFileInput):
def is_initial(self, value):
"""
Return whether value is considered to be initial value.
"""
return bool(value)
def format_value(self, value):
"""Format the size of the value in the db.
We can't render it's name or url, but we'd like to give some information
as to wether this file is not empty/corrupt.
"""
if self.is_initial(value):
return f'{len(value)} bytes'
def value_from_datadict(self, data, files, name):
"""Return the file contents so they can be put in the db."""
upload = super().value_from_datadict(data, files, name)
if upload:
return upload.read()
And then you need to use it in admin in the following way:
class MyModelAdmin(admin.ModelAdmin):
formfield_overrides = {
models.BinaryField: {'widget': BinaryFileInput()},
}
fields = ('name', 'your_binary_file')
Note:
BinaryField doesn't have a url or a file name so you will not be able to check what's in the db
After uploading the file you will be able to see just the byte size of the value stored in the db
You might want to extend the widget to be able to download the file
by reading it's contents
In my views.py file, there are some functions like follows:
def upload_file(request):
"""
upload a file and store it into the database,
and the file will be predicted in another view immediately.
"""
def predict(request):
"""
make prediction of the file uploaded in the 'upload_file' function.
"""
How to access the file which uploaded in upload_file function in the predict function? Here is my thought:
1. read the last row in the database, but this sounds a little silly.
2. use a cache system and retrieve the file from cache?
Is there any useful solution for this problem? please give me any hints or other resources.
Thanks for anyone's help.
As it is described here, you can acces the storage url as Follows :
Class MyModel(models.Model):
photo = models.ImageField()
model = MyModel.objects.get(pk=1) # for instance
model.photo.url
>>> 'https://media.example.com/mymodels/example_image.jpg'
I have a Django model with a customized image field. The image field creates some thumbnail sizes on upload. Code may look like this:
from django.db import models
from utils import CustomImageField
class Photo(models.Model):
image = CustomImageField()
Now I modify the original image, let's say I rotate it. And now I want to trigger the save method of my image field again, in order to overwrite the thumbnails and create rotated versions. So, I don't need to rotate the thumbnails elsewhere in my code (DRY).
Any thoughts? Something along those lines - but how exactly?
p = Photo.objects.get(pk=1)
p.image.save(...)
I have full control over the CustomImageField widget. The save() method is defined as:
def save(self, name, path, save=True):
Question is, what do I use for the methods parameters?
This question looks like a duplicate of Programmatically saving image to Django ImageField
The parameters of the ImageField.save() method are documented for FileField.save() (of which ImageField is a subclass):
https://docs.djangoproject.com/en/1.9/ref/models/fields/#django.db.models.fields.files.FieldFile.save
Takes two required arguments: name which is the name of the file, and
content which is an object containing the file’s contents. The
optional save argument controls whether or not the model instance is
saved after the file associated with this field has been altered.
Defaults to True.
Here is what is working for us:
class CustomImage(models.Model):
image = models.ImageField(upload_to=get_file_path, max_length=500)
orig_name = models.TextField()
This is the method that adds an image file to the ImageField from an http resource:
from django.core.files.base import ContentFile
def download_photo(amazon_id, url):
img_data = requests.get(url)
img = CustomImage(orig_name=img_data.url)
img.image.save(slugify(img.orig_name), ContentFile(img_data.content), save=True)
It also works without ContentFile:
new_img = File(open(different_obj.image.path), 'r')
img.image.save(different_obj.image.url, new_img, save=True)
See also:
- https://docs.djangoproject.com/en/1.9/topics/files/
- https://djangosnippets.org/snippets/2587/
An option is dirty field checking, either manually (see this SO question) or using a pypi package
Alternatively, if you want to conserve memory, the resizing can be triggered from the field's property setter (assuming you inherit from FileField
class CustomImageField(FileField):
def _set_file(self, file):
has_file_changed = file != self._file
super(CustomImageField, self)._set_file(file)
if has_file_changed:
self.handle_resizing_etc()
# need to redeclare property so it points to the right _set_file
file = property(FileField._get_file, _set_file, FileField._del_file)
disclaimer: I haven't used this approach in production code and I haven't written a proof of concept before posting this answer, so it might not work as desired
I need to generate name of uploaded image with mask
images/{{ uploaded_image.id }}.jpg
I think the best way to do this is FileField.upload_to property, but it is written Here:
In most cases, this object will not have been saved to the database yet, so if it uses the default AutoField, it might not yet have a value for its primary key field.
May be in my case it will be better to rename file after saving object in save() method?
What I do is give upload_to a function, then assign a UUID is the function for it to save to:
import uuid
def get_upload_to(instance, filename):
instance.uuid = uuid.uuid4().hex
return 'blah/%s/%s' % (instance.uuid, filename)
class Photo(models.Model):
file = models.FileField(max_length=200,upload_to=get_upload_to)
uuid = models.CharField(max_length=32)
To avoid concurrency issues, you will need to store your object to the database before you determine the filename (because even if you get the next incremental id, which is possible, it may change while you save the file.) So store the object and then rename the file.
Thanks, Blixt and Todd! I choose better solution for my issue:
import time
class Image(models.Model):
def _get_upload_to(instance, filename):
return 'images/%f.jpg' % time.time()
original = models.ImageField(upload_to=_get_upload_to, ....)
It is guarantees that every filename will be unique in the same directory. In my case I need only jpg files, so in other cases it possible to substitute original extension