I want to create thumbnails for uploaded images. I know there are some libraries available, but I have very basic needs and I would prefer to do this on my own.
My model goes like this:
from image_tools import resize
class Photo(models.Model):
# some fields
image = models.ImageField(
upload_to='images',
height_field='height',
width_field='width'
)
thumb = models.ImageField(
upload_to='thumbs',
editable=False
)
def save(self):
self.thumb = self.image
resize(self.image.path, self.thumb.path, 50, 40)
super(Photo, self).save()
Here resize() is a simple function I have written which accepts as arguments the original file path and the destination path and the dimensions and stores a thumbnail. The only problem is that the file is not yet on the disk at this moment.
Is there a way to force saving the file itself on its location, so that it will be available to my function?
Please, feel free to suggest better ways to handle this matter. Still I would like to know how to handle files which are not yet saved, whatever the best solution turns out to be.
There are a few ways to do this. One you have the raw image data, you can write it to a file pretty easily (look up django file uploading to see how to save an uploaded file to disk). You can read any file your django process (or apache) has access to, not just in the path of your django instance.
Another thing you can do before saving this is to use PIL to thumbnail the image and save it:
http://effbot.org/imagingbook/image.htm
You can also upload the data to s3 using boto s3, if you want the images hosted on a CDN.
Storing the file directly is not a good idea because you ignore the default file storage (i.e. file system, amazon s3, ...)
Better approach is to use django File class
from django.core.files import File
# Create a Python file object using open() and the with statement
>>> with open('/tmp/hello.world', 'w') as f:
... myfile = File(f)
... myfile.write('Hello World')
Also to delete the file use the default storage class
See link below for more details
https://docs.djangoproject.com/en/1.5/topics/files/
Related
In the Flask documentation for file uploads, they recommend use of secure_filename() to sanitize a file's name before storing it.
Here's their example:
uploaded_file = request.files['file']
if uploaded_file:
filename = secure_filename(uploaded_file.filename) # <<<< note the use of secure_filename() here
file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
return redirect(url_for('display_file',
filename=filename))
The documentation says:
Now the problem is that there is that principle called “never trust user
input”. This is also true for the filename of an uploaded file. All
submitted form data can be forged, and filenames can be dangerous. For
the moment just remember: always use that function to secure a
filename before storing it directly on the filesystem.
With offsite storage (S3 or Google Cloud), I will not be using Flask to store the file on the web server. Instead, I'll rename the upload file (with my own UUID), and then upload it elsewhere.
Example:
blob = bucket.blob('prompts/{filename}'.format(filename=uuid.uui4()))
blob.upload_from_string(uploaded_file.read(), content_type=uploaded_file.content_type)
Under this scenario, am I right that you do you not need to invoke secure_filename() first?
It would seem that because I (a) read the contents of the file into a string and then (b) use my own filename, that my use case is not vulnerable to directory traversal or rogue command-type attacks (e.g. "../../../../home/username/.bashrc") but I'm not 100% sure.
You are correct.
You only need to use the secure_filename function if you are using the value of request.files['file'].filename to build a filepath destined for your filesystem - for example as an argument to os.path.join.
As you're using a UUID for the filename, the user input value is disregarded anyway.
Even without S3, it would also be safe NOT to use secure_filename if you used a UUID as the filename segment of the filepath on your local filesystem. For example:
uploaded_file = request.files['file']
if uploaded_file:
file_uuid = uuid.uuid4()
file.save(os.path.join(app.config['UPLOAD_FOLDER'], file_uuid))
# Rest of code
In either scenario you'd then store the UUID somewhere in the database. Whether you store the originally provided request.files['file'].filename value alongside that is your choice.
This might make sense if you want the user to see the original name of the file when they uploaded it. In that case it's definitey wise to run the value through secure_filename anyway, so there's never a situation where the frontend displays a listing to a user which includes a file called ../../../../ohdear.txt
the secure_filename docstring also points out some other functionality:
Pass it a filename and it will return a secure version of it. This
filename can then safely be stored on a regular file system and passed
to :func:os.path.join. The filename returned is an ASCII only string
for maximum portability.
On windows systems the function also makes sure that the file is not
named after one of the special device files.
>>> secure_filename("My cool movie.mov")
'My_cool_movie.mov'
>>> secure_filename("../../../etc/passwd")
'etc_passwd'
>>> secure_filename(u'i contain cool \xfcml\xe4uts.txt')
'i_contain_cool_umlauts.txt'
I'm making a django project that lets specific users to:
* first, upload a file with specified file-type;
* second, use that file to perform some calculations; and
* third, provide a derivative file for download.
The user can then view the list of all the files that was uploaded including the files uploaded by other users and choose which files to download or delete.
I made an app to manage the user's accounts and another app (core) for file uploads and calculations. Now I want to store the uploaded file in a specific directory so that I can easily access the file for data retrieval.
I tried doing this:
core_app/models.py
def file_path_dir(instance, filename):
return 'files/{0}/{1}'.format(instance.id, filename)
class Core(models.Model):
id = models.AutoField(primary_key=True, editable=False)
date_created = models.DateField(auto_now_add=True)
user = models.ForeignKey(get_user_model(), on_delete=models.CASCADE)
csv = models.FileField(upload_to=file_path_dir,
validators=[FileExtensionValidator(allowed_extensions=['csv'])]) # NOT SAFE
Here, I want the file to be uploaded in a directory using the id that will be stored in the database. However, the file was stored in this directory instead - files/None/filename.
I don't know why it says 'None' when I have successfully saved the file in the database (using postgresql). I need help how to solve this or how to properly do this in order for it to work.
Also, I would like to know the safest/ more secure way of validating file extensions.
The file_path_dir is called before an actual save on the database side happens. And the id that you are making use of, is not created yet because the save function isn't called yet.
In order to have the id you need to save the instance and the file in a temporary folder and then move it after you got the id from the database.
But a better way of doing is using something else instead of the id. You can use month, day or even year instead and if you really need something unique for each file directory, you can add a unique field like an extra id or slug and generate it yourself and make use of that.
And for your second question about validating file extensions:
the first step is to check the format which you're already doing and it's usually safe enough depending on the platform. But if you need more security, you can read the file in a safe environment and see if it meets the rules for that type of extension. Or you can use other apps that do that for you and use them in the background.
Class X(models.Model):
zip_file = models.FileField()
Now, this zip_file is a zip having any number of images. I want to be able to extract all the images and save (to DB) them under the same X object i.e. the "primary key" is the same.
I understand that the File is stored in the system and only a reference is stored in the DB and I am fine with it.
I am unsure about what is the best practice to unzip the file at the
server and save them one by one to the DB.
One naive idea I have is using the "validators" to check the files and save them to DB but unsure if that is a best practice.
Any help or suggestion is appreciated. :)
There are a few different ways that you can approach this, based on the requirements and number of images that are in the zip file. I am going to make the following assumptions:
You want to keep the zip file
There are an arbitrary number of image files in each zip file
In this case, then you can format your model as below. Once you finish the file upload process, you can trigger a signal or background task to parse the zip file, and for each image file, create a separate object that is related to the zip file through a foreign key. This would allow you to easily grab all of the files without having to create a number of image fields.
[models.py]
class X(models.Model):
zip_file = models.FileField()
class XImages(models.Model):
image = models.ImageField()
x = models.ForeignKey(X)
In the case where you don't actually care about keeping the zip file, then you can do something similar, but instead of saving the object on the form post, you can simply extract it then, and save the images in the separate class.
If there are a static number of images in each zip file, then you can simply create that specific number of ImageFields in the model itself.
I need to upload an image to a Django ImageField, but due to a restriction based on a third party API, I need the images to be uploaded to the media directory without a subdirectory.
The restriction is due to the fact that the filename stored locally (in the imagefield column) is used to calculate a hash to request that image from a media API, so the field must contain the same name as the filename in the API, unfortunately the API filename cannot contain slashes.
I can't add a field to the db to store the real filename, nor can I control the code that returns the calculated hash.
I tried making the upload_to a callable that returns an empty string, which seems to get around the upload_to required error, but the images are all then saved as just ''(blank) _1, _2 ... etc without the rest of the image name.
I have also tried making the upload_to return '/', '../media', '../media/' all to no avail (a variety of errors or malformed filenames)
So TL;DR how to save images directly to MEDIA_ROOT, without a sub directory?
EDIT, extra info:
So I just tried a little hack that does part of it:
from django.core.files.storage import FileSystemStorage
media_root_store = FileSystemStorage(location='/path/to/media/parent')
class MyModel(models.Model):
img_file = models.ImageField(
max_length=255,
upload_to='media',
storage=media_root_store)
This dirty hack saves the image to the correct location, though it still obviously saves the image path with media/ appended. So I'm unsure if it has actually gotten me any closer.
Turns out there was a pretty big clue in the error from my earlier attempt (the one generating _1, _2 etc)
If I specify an upload_to callable that simply returns the filename argument, then the image will be saved directly in the MEDIA_ROOT, with the correct filename saved in the database (without the directory prefix).
I had been under the impression that upload_to will just provide a path to append to MEDIA_ROOT, which will then have the image filename appended to that. Turns out this is incorrect.
In my app I parse some xml which contains a path to an image file. Now if I am passing the path to the property of my model which is a FileField it's not copying the file using the upload_to settings. I also tried to pass it a stream of that file but that raised an exception.
How do I use the FileField with data that isn't coming from a request?
Assuming the file is in your MEDIA_ROOT (If it's outside of MEDIA_ROOT you'll get SuspiciousOperation errors):
m = YourModel(file='uploads/file.txt')
If you already have the file on your system, it'd surely be easier to just move it to your uploads directory. You could always customize FileField to handle moving the file for you.