I have a Django app where users can upload images and can have a processed version of the images if they want. and the processing function returns the path, so my approach was
model2.processed_image = processingfunction( model1.uploaded_image.path)
and as the processing function returns path here's how it looks in my admin view
not like the normally uploaded images
In my machine it worked correctly and I always get a 404 error for the processed ones while the normally uploaded is shown correctly when I try to change the url of the processed from
myurl.com/media/home/ubuntu/Eyelizer/media/path/to/the/image
to
myurl.com/media/path/to/the/image
so how can I fix this ? is there a better approach to saving the images manually to the database ?
I have the same function but returns a Pil.image.image object and I've tried many methods to save it in a model but I didn't know how so I've made the function return a file path.
I think the problem is from nginx where I define the media path.
should/can I override the url attribute of the processedimage?
making something like
model.processed_image.url = media/somefolder/filename
Instead of using the PIL Image directly, create a django.core.files.File.
Example:
from io import BytesIO
from django.core.files import File
img_io = BytesIO() # create a BytesIO object to temporarily save the file in memory
img = processingfunction( model1.uploaded_image.path)
img.save(img_io, 'PNG') # save the PIL image to the BytesIO object
img_file = File(thumb_io, name='some-name.png') # create the File object
# you can use the `name` from `model1.uploaded_image` and use
# that above
# finally, pass the image file to your model field
model2.processed_image = img_file
To avoid repetition of this code, it would be a good idea to keep this code in processingfunction and return the File object directly from there.
My approach is a bit different from #Xyres's, I thought xyres's would make a duplicate of the existing image and create a new one and when I tried overriding the URL attribute it returned an error of
can't set the attribute
but when I saw this question and this ticket I tried making this and it worked
model2.processed_image = processingfunction(model1.uploaded_image.path)
full_path = model2.processed_image.path
model2.processed_image.name = full_path.split('media')[1]
so that explicitly making the URL media/path/to/image and cut out all of the unneeded parts like home/ubuntu and stuff
Related
I am trying to create a simple blog app using flask that uses flask_flatpages to fill a jinja2 template using the contents of a markdown file for each post.
app = Flask(__name__)
app.config.from_pyfile('settings.py')
pages = FlatPages(app)
#app.route('/<path>/')
def blog_post(path):
post = pages.get_or_404(path)
return render_template('post.html', post=post)
The issue I'm having is that I'm unable to link an image in the markdown file, for example the following example_post.md file returns a 404 error in the rendered HTML for the image.png file (when accessing e.g. http://localhost:5000/example_post/)
# Heading
Here is an example image.
I think this is because accessing the image attempts to find example_post/image.png, due to the route I created, but the image is actually in the same directory as the post.md file (there is no example_post/ directory). The file structure is as follows:
--app.py
--posts/
----example_post.md
----image.png
--templates/
----post.html
Any suggestions for how to correctly reference the image.png file in this case, or how to better structure the app to make this work?
We can use this below as an example to fix your problem.
from flask import send_from_directory
#app.route('<path:filename>')
def serve_static(filename):
root_dir = os.path.dirname(os.getcwd())
return send_from_directory(os.path.join(root_dir, 'md'), filename)
Example :
https://www.programcreek.com/python/example/65747/flask.send_from_directory
I have an API endpoint with Django Rest Framework to upload an image.
class MyImageSerializer(serializers.ModelSerializer):
image = serializers.ImageField(source='image')
I can upload images but they are saved with the filename that is sent from the client which can result to collisions. I would like instead to upload the file to my CDN with a timestamp filename.
Generating the filename is not the problem, just saving the image with it.
Any one knows how to do that?
Thanks.
If your image is of type ImageField from django, then you don't really have to do anything, not even declare it in your serializer like you did. It's enough to add it in the fields attribute and django will handle collisions. This means django will add _index on each new file which might generate a collision, so if you upload a file named 'my_pic.jpg' 5 times, you will actually have files 'my_pic.jpg', 'my_pic_1.jpg', 'my_pic_2.jpg', 'my_pic_3.jpg', 'my_pic_4.jpg' on your server.
Now, this is done using django's implementation for FileSystemStorage (see here), but if you want it to append a timestamp to your filename, all you have to do is write a storage class where you overwrite the get_available_name(name) method. Example:
class MyFileSystemStorage(FileSystemStorage):
def get_available_name(self, name):
''' name is the current file name '''
now = time.time()
stamp = datetime.datetime.fromtimestamp(now).strftime('%Y-%m-%d-%H-%M-%S')
return '{0}_{1}'.format(name, str(stamp))
And the image field in your model:
image = models.ImageField(upload_to='your upload dir', storage= MyFileSystemStorage)
Important update
As of August 20, 2014 this is no longer an issue, since Django found a vulnerability related to this behaviour (thanks #mlissner for pointing it out) . Important excerpt :
We’ve remedied the issue by changing the algorithm for generating file
names if a file with the uploaded name already exists.
Storage.get_available_name() now appends an underscore plus a random 7
character alphanumeric string (e.g. "_x3a1gho"), rather than iterating
through an underscore followed by a number (e.g. "_1", "_2", etc.).
I have a following situation. I created a simple backend in Flask that handles file uploads. With files received, Flask does something (uploads them), and returns the data to the caller. There are two scenarios with the app, to upload one image and multiple images. When uploading one image, I can simply get the response and voila, I'm all set.
However, I am stuck on handling multiple file uploads. I can use the same handler for the actual file upload, but the issue is that all of those files need to be stored into a list or something, then processed, and after doing that, a single link (album) containing all those images, needs to be delivered.
Here is my upload handling code:
#app.route('/uploadv3', methods=['POST'])
def upload():
if request.method == 'POST':
data_file = request.files["file"]
file_name = data_file.filename
path_to_save_to = os.path.join(app.config['UPLOAD_FOLDER'], file_name)
data_file.save(path_to_save_to)
file_url = upload_image_to_image_host(path_to_save_to)
return file_url
I was experimenting with session in flask, but I dont know can I create a list of items under one key, like session['links'], and then get all those, and clear it after doing the work. Or is there some other simpler solution?
I assume that I could probably do this via key for each image, like session["link1"], and so on, but that would impose a limit on the images (depending on how much of those I create), would make the code very ugly, make the iteration over each in order to generate a list that is passed to an album building method problematic, and session clearing would be tedious.
Some code that I wrote for getting the actual link at the end and clearing the session follows (this assume that session['link'] has a list of urls, which I can't really achieve with my knowledge of session management in Flask:
def create_album(images):
session.pop('link', None)
new_album = im.create_album(images)
return new_album.link
#app.route('/get_album_link')
def get_album_link():
return create_album(session['link'])
Thanks in advance for your time!
You can assign anything to a session including individual value or list/dictionary etc. If you know the links, you can store them in the session as follows:
session['links'] = ['link1','link2'...and so on]
This way, you have a list of all the links. You can now access a link by:
if 'links' in session:
for link in session['links']:
print link
Once you are done with them, you can clear the session as:
if 'links' in session:
del session['links']
To clarify what I have done to make this work. At the end, it appeared that the uploading images and adding them to the album anonymously had to be done "reversely", so not adding images to an album object, but uploading an image object to an album id.
I made a method that gets the album link and puts it in the session:
#app.route('/get_album_link')
def get_album_link():
im = pyimgur.Imgur(CLIENT_ID)
new_album = im.create_album()
session.clear()
session['album'] = new_album.deletehash
session['album_link'] = new_album.link
return new_album.link
Later on, when handling uploads, I just add the image to the album and voila, all set :)
uploaded_image = im.upload_image(path_of_saved_image, album=session['album'])
file_url = uploaded_image.link
return file_url
One caveat is that the image should be added to the "deleteahash" value passed as the album value, not the album ID (which is covered by the imgur api documentation).
I am using a file upload in my Django model like this :
def upload_path(self, filename):
return 'upload/actualities/%s/%s' % (self.id, filename)
photo = models.ImageField(upload_to=upload_path)
and my adminModel is :
from actualities.models import *
from django.contrib import admin
class ActualityAdmin(admin.ModelAdmin):
class Media:
js = ('/static/js/tiny_mce/tiny_mce.js', '/static/js/textareas.js')
admin.site.register(Actuality, ActualityAdmin)
Everything works fine except when i edit mu model because it has an id. But when I create it, the file upload happens before the model saving... So i put my file in /media/actualities/None/filename.jpg, and I want /media/2/filename.jpg
How can I force to make the file upload after the model saving?
Thank you!!!
You will probably want to override the Model's save() method, and maybe come up with a custom "don't do anything" UploadHandler, then switch back to the original one and call save again.
https://docs.djangoproject.com/en/dev/topics/http/file-uploads/
https://docs.djangoproject.com/en/dev/topics/db/models/
What I would do in this situation however, is make a custom upload handler that saves the file off into some temp space. Then I'd override the save method (in a mixin or something) that moves the file from temp to wherever you wanted it.
#Tomek's answer is also another way. If you have your model generate it's own id, then you can use that.
A second to last suggestion which is what I do with my photo blog is instead of saving all the images in a directory like media/2/filename.jpg I save the image by date uploaded. 2011/10/2/image.jpg This kind of helps any directory from getting too unwieldy.
Finally, you could hash the file names and store them in directories of hash name to kind of equally spread out the images in a directory.
I've picked the date style because that's meaningful for me with that project. Perhaps there is another way you can name an image for saving that would mean something more than "model with id 2's pics" that you could use for this problem.
Good Luck!
As workaround, try generating UUID for file name (instead of using self.id).
I have a lot of user uploaded content and I want to validate that uploaded image files are not, in fact, malicious scripts. In the Django documentation, it states that ImageField:
"Inherits all attributes and methods from FileField, but also validates that the uploaded object is a valid image."
Is that totally accurate? I've read that compressing or otherwise manipulating an image file is a good validation test. I'm assuming that PIL does something like this....
Will ImageField go a long way toward covering my image upload security?
Django validates the image uploaded via form using PIL.
See https://code.djangoproject.com/browser/django/trunk/django/forms/fields.py#L519
try:
# load() is the only method that can spot a truncated JPEG,
# but it cannot be called sanely after verify()
trial_image = Image.open(file)
trial_image.load()
# Since we're about to use the file again we have to reset the
# file object if possible.
if hasattr(file, 'reset'):
file.reset()
# verify() is the only method that can spot a corrupt PNG,
# but it must be called immediately after the constructor
trial_image = Image.open(file)
trial_image.verify()
...
except Exception: # Python Imaging Library doesn't recognize it as an image
raise ValidationError(self.error_messages['invalid_image'])
PIL documentation states the following about verify():
Attempts to determine if the file is broken, without actually decoding
the image data. If this method finds any problems, it raises suitable
exceptions. This method only works on a newly opened image; if the
image has already been loaded, the result is undefined. Also, if you
need to load the image after using this method, you must reopen the
image file.
You should also note that ImageField is only validated when uploaded using form. If you save the model your self (e.g. using some kind of download script), the validation is not performed.
Another test is with the file command. It checks for the presence of "magic numbers" in the file to determine its type. On my system, the file package includes libmagic as well as a ctypes-based wrapper /usr/lib64/python2.7/site-packages/magic.py. It looks like you use it like:
import magic
ms = magic.open(magic.MAGIC_NONE)
ms.load()
type = ms.file("/path/to/some/file")
print type
f = file("/path/to/some/file", "r")
buffer = f.read(4096)
f.close()
type = ms.buffer(buffer)
print type
ms.close()
(Code from here.)
As to your original question: "Read the Source, Luke."
django/core/files/images.py:
"""
Utility functions for handling images.
Requires PIL, as you might imagine.
"""
from django.core.files import File
class ImageFile(File):
"""
A mixin for use alongside django.core.files.base.File, which provides
additional features for dealing with images.
"""
def _get_width(self):
return self._get_image_dimensions()[0]
width = property(_get_width)
def _get_height(self):
return self._get_image_dimensions()[1]
height = property(_get_height)
def _get_image_dimensions(self):
if not hasattr(self, '_dimensions_cache'):
close = self.closed
self.open()
self._dimensions_cache = get_image_dimensions(self, close=close)
return self._dimensions_cache
def get_image_dimensions(file_or_path, close=False):
"""
Returns the (width, height) of an image, given an open file or a path. Set
'close' to True to close the file at the end if it is initially in an open
state.
"""
# Try to import PIL in either of the two ways it can end up installed.
try:
from PIL import ImageFile as PILImageFile
except ImportError:
import ImageFile as PILImageFile
p = PILImageFile.Parser()
if hasattr(file_or_path, 'read'):
file = file_or_path
file_pos = file.tell()
file.seek(0)
else:
file = open(file_or_path, 'rb')
close = True
try:
while 1:
data = file.read(1024)
if not data:
break
p.feed(data)
if p.image:
return p.image.size
return None
finally:
if close:
file.close()
else:
file.seek(file_pos)
So it looks like it just reads the file 1024 bytes at a time until PIL says it's an image, then stops. This obviously does not integrity-check the entire file, so it really depends on what you mean by "covering my image upload security": illicit data could be appended to an image and passed through your site. Someone could DOS your site by uploading a lot of junk or a really big file. You could be vulnerable to an injection attack if you don't check any uploaded captions or make assumptions about the image's uploaded filename. And so on.