How does Django rename uploaded files? - django

If upload a file image.png from a web browser, a new file named image.png will appear in the upload directory on the server.
If I then upload another file named image.png (same name), a new file named image_aj642zm.png will appear in the upload directory on the server.
Then, if I upload another file named image.png (again the same name), a new file named image_z6z2BaQ.png will appear in the upload directory on the server.
What method does Django use to rename the uploaded file if a file with that name already exists in the upload directory?
(i.e. where does the extra _aj642zm and _z6z2BaQ come from?)
The usual set-up:
In models.py:
from django.db import models
class Image(models.Model):
image = models.ImageField(upload_to='uploads/')
In forms.py:
from django import forms
from .models import Image
class ImageForm(forms.ModelForm):
class Meta:
model = Image
fields = ['image']
In views.py:
from django.shortcuts import render, redirect
from .forms import ImageForm
def upload_image(request):
if request.method == 'POST':
form = ImageForm(request.POST, request.FILES)
if form.is_valid():
form.save()
return redirect('index')
else:
form = ImageForm()
return render(request, 'upload_file.html', {'form': form})

Django default Storage class method called get_available_name
# If the filename already exists, add an underscore and a random 7
# character alphanumeric string (before the file extension, if one
# exists) to the filename until the generated filename doesn't exist.
# Truncate original name if required, so the new filename does not
# exceed the max_length.
Django by default saves object by its name but if object with that name already exists adds up underscore and 7 random chars as quoted in code comment
Also as addition to this Django Storage class method get_valid_name parses up file name before and replaces all spaces with underscores and removes all chars that are not unicode, alpha, dash, underscore or dot
re.sub(r'(?u)[^-\w.]', '', s)

Related

Django helper function to rename user uploaded image

For a django project I have a model that features an image field. The idea is that the user uploads an image and django renames it according to a chosen pattern before storing it in the media folder.
To achieve that I have created a helper class in a separate file utils.py. The class will callable within the upload_to parameter of the django ImageField model. Images should be renamed by concatenating .name and .id properties of the created item. The problem with my solution is that if the image is uploaded upon creating anew the item object, then there is no .id value to use (just yet). So instead of having let's say: banana_12.jpg I get banana_None.jpg. If instead I upload an image on an already existing item object, then the image is renamed correctly.
Here is my solution. How can I improve the code to make it work on new item object too?
Is there a better way to do this?
# utils.py
from django.utils.deconstruct import deconstructible
import os
#deconstructible
class RenameImage(object):
"""Renames a given image according to pattern"""
def __call__(self, instance, filename):
"""Sets the name of an uploaded image"""
self.name, self.extension = os.path.splitext(filename)
self.folder = instance.__class__.__name__.lower()
image_name = f"{instance}_{instance.id}{self.extension}"
return os.path.join(self.folder, image_name)
rename_image = RenameImage()
# models.py
from .utils import rename_image
class Item(models.Model):
# my model for the Item object
name = models.CharField(max_length=30)
info = models.CharField(max_length=250, blank=True)
image = models.ImageField(upload_to=rename_image, blank=True, null=True) # <-- helper called here
# ...
I created the helper class RenameImage which for the reasons explained above works correctly only if the object already exists.
__EDIT
I have made some improvements based on the post_save advice. The user uploaded image gets resized and renamed as I want but then I get a RecursionError: maximum recursion depth exceeded. It seems that by calling the save() method on the instance I get into a signal loop...
Any idea on how to resolve this?
from django.db.models.signals import post_save
from .models import Item
from PIL import Image
import os
def resize_rename(sender, instance, created, max_height=300, max_width=300, **kwargs):
if instance.image: # only fire if there is an image
with open(instance.image.path, 'r+b') as f:
image = Image.open(f)
# Resize the image if larger than max values
if image.height > max_height or image.width > max_width:
output_size = (max_height, max_width)
image.thumbnail(output_size, Image.ANTIALIAS)
image.save(instance.image.path, quality=100)
# Grab the image extension
_name, extension = os.path.splitext(instance.image.name)
# Use old_name to create the desired new_name while keeping the same dirs
old_name = instance.image.name.split('/')[-1] # e.g. IMG_3402.jpg
new_name = instance.image.name.replace(old_name, f"{instance.name}_{instance.id}{extension}")
# Rename if name is not the right one already
if old_name != new_name:
old_path = instance.image.path
new_path = old_path.replace(old_name, f"{instance.name}_{instance.id}{extension}")
instance.image.name = new_name # Assign the new name
os.replace(old_path, new_path) # Replace with the new file
instance.save(update_fields=['image']) # Save the instance
post_save.connect(resize_rename, sender=Item)
It is not possible because the id is assigned after saving.
You have to use a post_save signal and then change the filename
To do this add signal.py under app
In the apps.py file, override the ready function to add the signals.py import statement
apps.py
from django.apps import AppConfig
class AppNameConfig(AppConfig):
name = 'app_name'
def ready(self):
import app_name.signals
On init.py set default app config
init.py
default_app_config = 'app_name.apps.AppNameConfig'
signals.py
from django.db.models.signals import post_save
from django.dispatch import receiver
from .models import Item
#receiver(post_save, sender=Item)
def post_save(sender, instance: FilettoPasso, **kwargs):
if kwargs['created']:
print(instance.id)
instance.image.path = new path
# rename file in os path
...
instance.save()

Saving a file from FileResponse in Django

How to save file in FileField from FileResponse in Django.
I have the simplest PDF function to create PDF file in Django (according to the documentation).
import io
from django.http import FileResponse
from reportlab.pdfgen import canvas
def some_view(request):
# Create a file-like buffer to receive PDF data.
buffer = io.BytesIO()
# Create the PDF object, using the buffer as its "file."
p = canvas.Canvas(buffer)
# Draw things on the PDF. Here's where the PDF generation happens.
# See the ReportLab documentation for the full list of functionality.
p.drawString(100, 100, "Hello world.")
# Close the PDF object cleanly, and we're done.
p.showPage()
p.save()
# FileResponse sets the Content-Disposition header so that browsers
# present the option to save the file.
buffer.seek(0)
return FileResponse(buffer, as_attachment=True, filename='hello.pdf')
It returns PDF. But how to save this PDF in FileField?
I need something like this:
Models.py
class FileModel(models.Model):
file = models.FileField()
Views.py
def some_view(request):
[...same code as above to line buffer.seek(0)...]
obj = FileModel()
obj.file = FileResponse(buffer, as_attachment=True, filename='hello.pdf')
obj.save()
For this task Django provides the File wrapper object:
from django.core.files import File
def some_view(request):
# ...same code as above to line buffer.seek(0)...
obj = FileModel()
obj.file = File(buffer, name='hello.pdf')
obj.save()

Can't dynamically create path with upload_to

In my user model, I have an ImageField that contains an upload_to attribute that will create the image path dynamically based on the user's id.
avatar = models.ImageField(storage=OverwriteStorage(), upload_to=create_user_image_path)
def create_user_image_path(instance, image_name):
image_name = str(instance.user.id) + image_name[-4:]
return 'users/{0}/avatars/{1}'.format(instance.user.id, image_name)
When I sign up, I get an error that looks like this:
The system cannot find the path specified:
'C:\Users\xx\PycharmProjects\project_name\media\users\150\avatars'
If I remove the id and avatars and just include the image name (with no new directories) it works successfully and writes the image. I have tried doing chmod -R 777 on this directory, but it still doesn't create these new directories dynamically. I'm not sure what I'm doing wrong.
EDIT
class OverwriteStorage(FileSystemStorage):
def get_available_name(self, name, *args, **kwargs):
# Delete all avatars in directory before adding new avatar.
# (Sometimes we have different extension names, so we can't delete by name
file_path = os.path.dirname(name)
shutil.rmtree(os.path.join(settings.MEDIA_ROOT, file_path))
return name
The problem I see is because of my OverwriteStorage function. When I remove it, it saves. How can I fix this function so that it overwrites if it exists?
I think your problem id with instance.user.id
I would suggest you try to import the User as below it may help
from django.contrib.auth.models import User
user = User.objects.get(id=user_id)
def create_user_image_path(instance, image_name):
image_name = str(user) + image_name[-4:]
return 'users/{0}/avatars/{1}'.format(user.id, image_name)
However django recommends to user AUTH_USER_MODEL
implementation
from django.contrib.auth import get_user_model
User = get_user_model()
Update
When you look into override storage, the variable name gets the name of the image you are about to upload. Your file_path = os.path.dirname(name), you are returning a folder with image name. When shutil.rmtree ties to find the folder, it returns error it cannot find the path. Use conditional statement i.e. if or try except block as shown below.
from django.core.files.storage import FileSystemStorage
from django.conf import settings
from django.core.files.storage import default_storage
import os
class OverwriteStorage(FileSystemStorage):
def get_available_name(self, name, max_length=None):
"""Returns a filename that's free on the target storage system, and
available for new content to be written to.
Found at http://djangosnippets.org/snippets/976/
This file storage solves overwrite on upload problem. Another
proposed solution was to override the save method on the model
like so (from https://code.djangoproject.com/ticket/11663):
def save(self, *args, **kwargs):
try:
this = MyModelName.objects.get(id=self.id)
if this.MyImageFieldName != self.MyImageFieldName:
this.MyImageFieldName.delete()
except: pass
super(MyModelName, self).save(*args, **kwargs)
"""
# If the filename already exists, remove it as if it was a true file system
if self.exists(name):
os.remove(os.path.join(settings.MEDIA_ROOT, name))
# default_storage.delete(os.path.join(settings.MEDIA_ROOT, name))
return name
Like in your case
class OverwriteStorage(FileSystemStorage):
def get_available_name(self, name, *args, **kwargs):
# Delete all avatars in directory before adding new avatar.
# (Sometimes we have different extension names, so we can't delete by name
if self.exists(name):
file_path = os.path.dirname(name)
shutil.rmtree(os.path.join(settings.MEDIA_ROOT, file_path))
return name

How to add signal method for my view?

I want to count how may files the user has uploaded.
I have added signals.py
from django.dispatch import Signal
upload_completed = Signal(providing_args=['upload'])
And summary.py
from django.dispatch import receiver
from .signals import upload_completed
#receiver(charge_completed)
def increment_total_uploads(sender, total, **kwargs):
total_u += total
to my project.
My views upload
#login_required
def upload(request):
# Handle file upload
user = request.user
if request.method == 'POST':
form = DocumentForm(request.POST, request.FILES)
if form.is_valid():
newdoc = Document(docfile=request.FILES['docfile'])
newdoc.uploaded_by = request.user.profile
upload_completed.send(sender=self.__class__, 'upload')
#send signal to summary
newdoc.save()
# Redirect to the document list after POST
return HttpResponseRedirect(reverse('upload'))
else:
form = DocumentForm() # A empty, unbound form
# Load documents for the upload page
documents = Document.objects.all()
# Render list page with the documents and the form
return render(request,'upload.html',{'documents': documents, 'form': form})
This effort does not work.I got
upload_completed.send(sender=self.__class__, 'upload')
^
SyntaxError: positional argument follows keyword argument
I found signal example testing-django-signals
from .signals import charge_completed
#classmethod
def process_charge(cls, total):
# Process chargeā€¦
if success:
charge_completed.send_robust(
sender=cls,
total=total,
)
But it seems to me that classmethod would not work in my case
How to fix my method?
You don't need the 'uploads' argument for the send() method.
But a tip, if you're planning on a persistent count of the number of file uploads (which I assume you most likely are), then I think you should create a new model so that you can save it in your database.Then you can update that model every time a Document model is saved.
And I suggest you have a look at post_save. Have a nice day coding!

django form is_valid always fails (extending django-registration form)

I'm trying to extend Django registration to include my own registration form. In principle this is fairly simple. I just have to write my own form (CustomRegistrationForm) which is a child of the original one (RegistrationForm). Then I can process my specific input by using the user_registered signal of django registration.
So here is what I did:
urls.py:
from django.conf.urls import patterns, include, url
from registration.views import register
from forms import CustomRegistrationForm
from django.contrib import admin
import regbackend
admin.autodiscover()
urlpatterns = patterns('',
url(r'^register/$', register, {'backend': 'registration.backends.default.DefaultBackend', 'form_class': CustomRegistrationForm, 'template_name': 'custom_profile/registration_form.html'},
name='registration_register'),
)
regbackend.py:
from django import forms
from models import UserProfile
from forms import CustomRegistrationForm
def user_created(sender, user, request, **kwargs):
form = CustomRegistrationForm(data=request.POST, files=request.FILES)
if form.is_valid(): # HERE: always fails
user_profile = UserProfile()
user_profile.user = user
user_profile.matriculation_number = form.cleaned_data['matriculation_number']
user_profile.save()
from registration.signals import user_registered
user_registered.connect(user_created)
forms.py:
from models import UserProfile
from django import forms
from registration.forms import RegistrationForm
from django.utils.translation import ugettext_lazy as _
attrs_dict = {'class': 'required'}
class CustomRegistrationForm(RegistrationForm):
matriculation_number = forms.CharField(widget=forms.TextInput(attrs=attrs_dict),
label=_("Matriculation number"),
max_length=12,
error_messages={'invalid': _("This value has to be unique and may contain only numbers.")},
initial=108)
def clean_matriculation_number(self):
print "Validating matriculation number."
data = self.cleaned_data['matriculation_number']
if len(data) != 12:
raise forms.ValidationError(_("Matriculation number consists of exactly 12 digits."))
return data
So the problem is the is_valid() function, because it always returns False. Even if there are no errors! So what is wrong? I spent hours on this one and I have no idea anymore :(
Any help is much appreciated!!
The reason form.is_valid() fails is probably because the form's "clean_username" function checks if the username passed to it already exists. Since the signal is sent after the User object is created and added to the database, the form will fail to pass this test every time. My guess would be if you logged form.cleaned_data after is_valid() returns False, you'll get a list of all of the fields except the username.
form.data might not contain changed values for the fields if the form clean_ function makes any changes (I couldn't find much documentation on form.data).
To avoid the potential problems this might cause, I made two classes for my custom registration form:
# I use this one for validating in my create_user_profile function
class MyRegistrationFormInternal(forms.Form):
# Just an example with only one field that holds a name
name = forms.CharField(initial="Your name", max_length=100)
def clean_name(self):
# (Optional) Change the name in some way, but do not check to see if it already exists
return self.cleaned_data['name'] + ' foo '
# This one is actually displayed
class MyRegistrationForm (MyRegistrationFormInternal):
# Here is where we check if the user already exists
def clean_name(self):
modified_name = super(MyRegistrationForm, self).clean_name()
# Check if a user with this name already exists
duplicate = (User.objects.filter(name=modified_name)
if duplicate.exists():
raise forms.ValidationError("A user with that name already exists.")
else:
return modified_name
Then, instead of using the form.data (which may still be "unclean" in some ways), you can run your POST data through MyRegistrationFormInternal, and is_valid() shouldn't always return false.
I realize this isn't the cleanest solution, but it avoids having to use the (possibly raw) form.data.
Ok I think I solved it (more or less).
I'm still not really sure, why the form did not validate. But as I said I was extending django-registration and the 'register' view already called is_valid() of the form, so I can assume that the form is valid when I process the posted data any futher. The view then calls the backend
backend.register(request, **form.cleaned_data)
with the request and the cleaned data (which is just username, email and password). So I can't use it for registration because my additional information is missing. The backend then fires the signal that I am using and what I did is, is that I created the form again with the provided request. This form, however, will NOT validate (and I tried everything!!) I looked it up, I am doing the exact same thing as django-registration, but it's not working in my code.
So I did not really solve the problem, because the form is still not validating. But I found peace with this, when I realized that the form was already validated by the 'register' view. So I am using form.data[..] instead of form.cleaned_data[..] now which shouldn't be a problem...