I have an API endpoint with Django Rest Framework to upload an image.
class MyImageSerializer(serializers.ModelSerializer):
image = serializers.ImageField(source='image')
I can upload images but they are saved with the filename that is sent from the client which can result to collisions. I would like instead to upload the file to my CDN with a timestamp filename.
Generating the filename is not the problem, just saving the image with it.
Any one knows how to do that?
Thanks.
If your image is of type ImageField from django, then you don't really have to do anything, not even declare it in your serializer like you did. It's enough to add it in the fields attribute and django will handle collisions. This means django will add _index on each new file which might generate a collision, so if you upload a file named 'my_pic.jpg' 5 times, you will actually have files 'my_pic.jpg', 'my_pic_1.jpg', 'my_pic_2.jpg', 'my_pic_3.jpg', 'my_pic_4.jpg' on your server.
Now, this is done using django's implementation for FileSystemStorage (see here), but if you want it to append a timestamp to your filename, all you have to do is write a storage class where you overwrite the get_available_name(name) method. Example:
class MyFileSystemStorage(FileSystemStorage):
def get_available_name(self, name):
''' name is the current file name '''
now = time.time()
stamp = datetime.datetime.fromtimestamp(now).strftime('%Y-%m-%d-%H-%M-%S')
return '{0}_{1}'.format(name, str(stamp))
And the image field in your model:
image = models.ImageField(upload_to='your upload dir', storage= MyFileSystemStorage)
Important update
As of August 20, 2014 this is no longer an issue, since Django found a vulnerability related to this behaviour (thanks #mlissner for pointing it out) . Important excerpt :
We’ve remedied the issue by changing the algorithm for generating file
names if a file with the uploaded name already exists.
Storage.get_available_name() now appends an underscore plus a random 7
character alphanumeric string (e.g. "_x3a1gho"), rather than iterating
through an underscore followed by a number (e.g. "_1", "_2", etc.).
Related
How can I direct the destination of the output file to my db?
My models.py is structured like so:
class Model(models.Model):
char = models.CharField(max_length=50, null=False, blank=False)
file = models.FileField(upload_to=upload_location, null=True, blank=True)
I have the user enter a value for 'char', and then the value of 'char' is printed on to a file. The process of successfully printing onto the file is working, however, the file is outputting to my source directory.
My goal is to have the output file 'pdf01.pdf' output to my db and be represented as 'file' so that the admin can read it.
Much of the information in the Dango docs has been focussed on directing the path of objects imported by the user directly, not on files that have been created internally. I have been reading mostly from these docs:
Models-Fields
Models
File response objects
Outputting PDFs
I have seen it recommend to write to a buffer, not a file, then save the buffer contents to my db however I haven't been able to find many examples of how to do that relevant to my situation online.
Perhaps there is a relevant gap in my knowledge regarding buffers and BytesIO? Here is the function I have been using to alter the pdf, I have been using BytesIO to temporarily store files throughout the process but have not been able to figure out how to use it to direct the output anywhere specific.
can = canvas.Canvas(BytesIO(), pagesize=letter)
can.drawString(10, 10, char)
can.save()
BytesIO().seek(0)
text_pdf = PdfFileReader(BytesIO())
base_file = PdfFileReader(open("media/01.pdf", "rb"))
page = base_file.getPage(0)
page.mergePage(text_pdf.getPage(0))
PdfFileWriter().addPage(page)
PdfFileWriter().write(open("pdf01.pdf", "wb")
FileField does not store files directly in the database. Files get uploaded in a location on the filesystem determined by the upload_to argument. Only some metadata are stored in the DB, including the path of the file in your filesystem.
If you want to have the contents of the files in the database, you could create a new File model that includes a BinaryField to store the data and a CharField to store the URL from which the file can be fetched. To feed the data of PdfFileWriter to the binary field of Django, perhaps the most appropriate would be to use BytesIO.
I found this workaround to direct the file to a desired location (in this case both my media_cdn folder and also output it to an admin.)
I set up an admin action to perform the function that outputs the file so the admin will have access to both the output version in the form of both an HTTP response and through the media_cdn storage.
Hope this helps anyone who struggles with the same problem.
#admin.py
class edit_and_output():
def output:
author = Account.email
#alter file . . .
with open('media_cdn/account/{0}.pdf'.format(author), 'wb') as out_file:
output.write(out_file)
response = HttpResponse(content_type='application/pdf')
response['Content-Disposition'] = 'attachment;filename="{0}.pdf"'.format(author)
output.write(response)
I have a Django app where users can upload images and can have a processed version of the images if they want. and the processing function returns the path, so my approach was
model2.processed_image = processingfunction( model1.uploaded_image.path)
and as the processing function returns path here's how it looks in my admin view
not like the normally uploaded images
In my machine it worked correctly and I always get a 404 error for the processed ones while the normally uploaded is shown correctly when I try to change the url of the processed from
myurl.com/media/home/ubuntu/Eyelizer/media/path/to/the/image
to
myurl.com/media/path/to/the/image
so how can I fix this ? is there a better approach to saving the images manually to the database ?
I have the same function but returns a Pil.image.image object and I've tried many methods to save it in a model but I didn't know how so I've made the function return a file path.
I think the problem is from nginx where I define the media path.
should/can I override the url attribute of the processedimage?
making something like
model.processed_image.url = media/somefolder/filename
Instead of using the PIL Image directly, create a django.core.files.File.
Example:
from io import BytesIO
from django.core.files import File
img_io = BytesIO() # create a BytesIO object to temporarily save the file in memory
img = processingfunction( model1.uploaded_image.path)
img.save(img_io, 'PNG') # save the PIL image to the BytesIO object
img_file = File(thumb_io, name='some-name.png') # create the File object
# you can use the `name` from `model1.uploaded_image` and use
# that above
# finally, pass the image file to your model field
model2.processed_image = img_file
To avoid repetition of this code, it would be a good idea to keep this code in processingfunction and return the File object directly from there.
My approach is a bit different from #Xyres's, I thought xyres's would make a duplicate of the existing image and create a new one and when I tried overriding the URL attribute it returned an error of
can't set the attribute
but when I saw this question and this ticket I tried making this and it worked
model2.processed_image = processingfunction(model1.uploaded_image.path)
full_path = model2.processed_image.path
model2.processed_image.name = full_path.split('media')[1]
so that explicitly making the URL media/path/to/image and cut out all of the unneeded parts like home/ubuntu and stuff
I am new to Django, and I am looking for the best approach to the following problem.
I have an application that is producing two reports. One is a JSON blob so I store it in psql with data model that uses JSONField.
The second report is a .html file.
The .html file will be generated multiple times a day so the first thing that came to mind was storing it in the db.
I need to be able to pull the report as well so it can be displayed to the user in the UI.
I created a test data model using TextField:
class TestResultsHTML(models.Model):
name = models.CharField(max_length=200)
report = models.TextField()
It makes it into the Db no problem, however when I attempt to retrieve it I can't seem to get the actual report:
In [3]: html_results = TestResultsHTML.objects.get(id=4)
In [4]: html_results.name
Out[4]: 'b0f5c336-867a-44a3-a5ef-6297bf6042cf'
In [5]: html_results.report
Out[5]: "<_io.TextIOWrapper name='report.html' mode='r' encoding='UTF-8'>"
I was expected that .report would return the actual contents of the file. The file itself is 1800+ lines.
Is this a good approach or is this not the intended use of TextField?
A TextField doesn't store the file, Django has a FileField for this (see here). This saves the file to a certain location/folder and the object saved in the DB essentially stores that location which you can then access later. Something like this:
class TestResultsHTML(models.Model):
name = models.CharField(max_length=200)
file_loc = models.FileField(upload_to=upload_location)
Then open the file at a later date with something like this:
with open(html_results.file_loc, 'w'):
I would like to keep the original file name of an UploadedFile in Django that has its location stored in a FileField. Right now I am observing that if two files have the same name, the first file uploaded keeps its original name but the second time a file with that name is uploaded, it has a random string appended to make the file name unique. One solution is to add an additional field to the model: Django: How to save original filename in FileField? or Saving Original File Name in Django with FileField but these solutions seem suboptimal as they require changing the Model fields.
An alternative would be to prepend a random directory path to the front of the file make sure that in a given directory the file name is unique and allowing the basename to remain unchanged. One way to do this would be to pass in a callable upload_to that does just that. Another option would be to subclass FileField and override get_filename to not strip the input filename to the basename allowing the caller to pass in a filename with a prepended path. The latter option is not ideal if I want to use an ImageField as I would have to subclass that as well.
In looking at the code that actually generates the unique filename by appending the random string, it looks like the best solution to this problem might be to subclass the Storage class in-use and override get_available_name method to create unique filenames by prepending a directory rather than post-pending the string to the base name.
Sorry for the quick answere, here is another approach to your question :
The idea here is to create an unique folder for each uploaded file.
# in your settings.py file
MY_FILE_PATH = 'stored_files/'
The path were your files will be stored : /public/media/stored_files
# somewhere in your project create an utils.py file
import random
try:
from hashlib import sha1 as sha_constructor
except ImportError:
from django.utils.hashcompat import sha_constructor
def generate_sha1(string, salt=None):
"""
Generates a sha1 hash for supplied string.
:param string:
The string that needs to be encrypted.
:param salt:
Optionally define your own salt. If none is supplied, will use a random
string of 5 characters.
:return: Tuple containing the salt and hash.
"""
if not isinstance(string, (str, unicode)):
string = str(string)
if isinstance(string, unicode):
string = string.encode("utf-8")
if not salt:
salt = sha_constructor(str(random.random())).hexdigest()[:5]
hash = sha_constructor(salt+string).hexdigest()
return (salt, hash)
In your models.py
from django.conf import settings
from utils.py import generate_sha1
def upload_to_unqiue_folder(instance, filename):
"""
Uploads a file to an unique generated Path to keep the original filename
"""
salt, hash = generate_sha1('{}{}'.format(filename, get_datetime_now().now))
return '%(path)s%(hash_path)s%(filename)s' % {'path': settings.MY_FILE_PATH,
'hash_path': hash[:10],
'filename': filename}
#And then add in your model fileField the uplaod_to function
class MyModel(models.Model):
file = models.FileField(upload_to=upload_to_unique_folder)
The file will be uploaded to this location :
public/media/stored_file_path/unique_hash_folder/my_file.extention
Note : I got the code from Django userena sources, and adapted it to my needs
Note2 : For more informations take a look at this greate post on Django File upload : File upload example
Have a good day.
Edit : Trying to provide a working solution :)
To my understanding, during the form submission/file upload process, you can add form validation functions.
During the validation and cleaning process, you could check that the database does not already have a duplicate name (ie. query to see if that file name exists).
If it is duplicate, you could just rename it xyz_1, xyz_2, etc
I am using a file upload in my Django model like this :
def upload_path(self, filename):
return 'upload/actualities/%s/%s' % (self.id, filename)
photo = models.ImageField(upload_to=upload_path)
and my adminModel is :
from actualities.models import *
from django.contrib import admin
class ActualityAdmin(admin.ModelAdmin):
class Media:
js = ('/static/js/tiny_mce/tiny_mce.js', '/static/js/textareas.js')
admin.site.register(Actuality, ActualityAdmin)
Everything works fine except when i edit mu model because it has an id. But when I create it, the file upload happens before the model saving... So i put my file in /media/actualities/None/filename.jpg, and I want /media/2/filename.jpg
How can I force to make the file upload after the model saving?
Thank you!!!
You will probably want to override the Model's save() method, and maybe come up with a custom "don't do anything" UploadHandler, then switch back to the original one and call save again.
https://docs.djangoproject.com/en/dev/topics/http/file-uploads/
https://docs.djangoproject.com/en/dev/topics/db/models/
What I would do in this situation however, is make a custom upload handler that saves the file off into some temp space. Then I'd override the save method (in a mixin or something) that moves the file from temp to wherever you wanted it.
#Tomek's answer is also another way. If you have your model generate it's own id, then you can use that.
A second to last suggestion which is what I do with my photo blog is instead of saving all the images in a directory like media/2/filename.jpg I save the image by date uploaded. 2011/10/2/image.jpg This kind of helps any directory from getting too unwieldy.
Finally, you could hash the file names and store them in directories of hash name to kind of equally spread out the images in a directory.
I've picked the date style because that's meaningful for me with that project. Perhaps there is another way you can name an image for saving that would mean something more than "model with id 2's pics" that you could use for this problem.
Good Luck!
As workaround, try generating UUID for file name (instead of using self.id).