thanks for tanking the time to look at this query.
I'm setting an ID field within one of my Django models. This is a CharField and looks like the following:
my_id = models.CharField(primary_key=True, max_length=5,
validators=[RegexValidator(
regex=ID_REGEX,
message=ID_ERR_MSG,
code=ID_ERR_CODE
)])
I would like to add a default/blank or null option that calls a global or class function that will cycle through the existing IDs, find the first one that doesn't exist and assign it as the next user ID. However, when I add the call blank=foo() I get an error code that the function doesn't exist.
Best,
pb
Edit1: I also tried using a separate utils file and importing the function, but (unsurprisingly) I get a circular import error as I need the call the class to get the objects.
Edit2 (Reply to Eugene): Tried that, solved the circular import but I'm getting the following error:
TypeError: super(type, obj): obj must be an instance or subtype of type
Previously my override of the save function worked perfectly:
def save(self, *args, **kwargs):
self.full_clean()
super(Staff, self).save(*args, **kwargs)
The custom id function:
def get_id_default():
from .models import MyObj
for temp_id in range(10_000, 100_000):
try:
MyObj.objects.get(my_id=str(temp_id))
except ObjectDoesNotExist:
break # Id doesn't exist
return str(hive_id)
Edit 3 (Reply to PersonPr7): Unfortunately, the kwargs doesn't seem to have my id in it. Actually, after having a print the kwargs dictionary comes back empty.
Save function:
def save(self, *args, **kwargs):
print(kwargs) # --> Returns {}
if kwargs["my_id"] is None:
kwargs["my_id"] = self.get_id_default()
self.full_clean()
super(Staff, self).save(*args, **kwargs)
Where the get_id_default is a class function:
def get_id_default(self):
for temp_id in range(10_000, 100000):
try:
self.objects.get(my_id=str(temp_id))
except ObjectDoesNotExist:
break # Id doesn't exist
return str(temp_id)
Solution1:
For those who are may be struggling with this in the future:
Create a utils/script .py file (or whatever you wanna call it) and create your custom script inside.
from .models import MyModel
def my_custom_default:
# your custom code
return your_value
Inside the main.models.py file.
from django.db import models
from .my_utils import my_custom_default
class MyModel(model.Model):
my_field = models.SomeField(..., default=my_custom_default)
Solution2: Create a static function within your Model class that will create your default value.
#staticmethod
def get_my_default():
# your logic
return your_value
# NOTE: Initially I had the function use self
# to retrieve the objects (self.objects.get(...))
# However, this raised an exception: AttributeError:
# Manager isn't accessible via Sites instances
When setting up your model give your field some kind of default i.e. default=None
Additionally, you need to override the models save function like so:
def save(self, *args, **kwargs):
if self.your_field is None:
self.my_field = self.get_my_default()
self.full_clean()
super().save(*args, **kwargs)
Try overriding the Model's save method and performing the logic there:
def save(self, *args, **kwargs):
#Custom logic
super().save(*args, **kwargs)
Edit:
You don't need to use **kwargs.
You can access your whole model from the save method and loop over objects / ids.
I find myself in an odd situation only when deployed (debug == false):
My model throws a path traversal attempt exception. I want to create a directory for every file uploaded and save the file within the directory (some.zip) used in example. In my dev environment I have no problems and everything works just fine.
models.py:
class Template(models.Model):
def get_folder(self, filename):
filename_PATH = Path(filename)
template_dir = filename_PATH.stem
return Path(settings.TEMPLATES_FOLDER).joinpath(template_dir, filename)
name = models.CharField("template", max_length=32, unique=True)
file = models.FileField("templatefile", upload_to=get_folder, null=True, max_length=260, storage=OverwriteStorage())
class OverwriteStorage(FileSystemStorage): #this is actually above
def get_available_name(self, name, max_length=None):
self.delete(name)
return name
forms.py:
class TemplateAdminForm(forms.ModelForm):
def __init__(self,*args,**kwargs):
super().__init__(*args, **kwargs)
class Meta:
model = Template
fields = ["name", "file", ]
def clean(self):
cleaned_data = super().clean()
upFile = Path(str(cleaned_data["file"]))
if upFile.suffix == ".zip":
path = self.instance.get_folder(cleaned_data["name"])
logging.error(f"{path}")
unpack_zip(path) ## works! the directory is created/filled
else:
raise forms.ValidationError("unknown file type ...")
logging.error("DONE!") # I see this output
return cleaned_data
## signal to see when the error might be happening:
#receiver(post_save, sender = Template)
def testing(sender, **kwargs):
logging.error("we never get here")
settings.py:
TEMPLATES_FOLDER = PATH(MEDIA_ROOT).joinpath("TEMPLATES")
but:
ERROR:django.security.SuspiciousFileOperation:Detected path traversal attempt in '/opt/project/media_root/TEMPLATES/some/some' WARNING:django.request:Bad Request: /admin/appName/template/add/
Edit:
Because of this discussion it might be important, this is happening on django 3.2.8
I get the same error on Django 3.2.6 when opening a file with mode "wb" at an absolute path name, when I'm not using a temporary file which I have read is recommened in order to avoid this problem so I will link this answer in case it helps you deploy it and share my experience.
Here's where it's been advised: answer
One possible solution would be to move that directory under the django project root folder and address it with a relative path. I'd try to use this too in order to understand how you could achieve this:
import os
print("WORKING DIRECTORY: " + os.getcwd())
An article on this topic suggests to use the following code (when dealing with an image file in that case): link
from django.core.files.temp import NamedTemporaryFile
from django.core import files
image_temp_file = NamedTemporaryFile(delete=True)
in_memory_image = open('/path/to/file', 'rb')
# Write the in-memory file to the temporary file
# Read the streamed image in sections
for block in in_memory_image.read(1024 * 8):
# If no more file then stop
if not block:
break # Write image block to temporary file
image_temp_file.write(block)
file_name = 'temp.png' # Choose a unique name for the file
image_temp_file.flush()
temp_file = files.File(image_temp_file, name=file_name)
Lets go through the code:
Create a NamedTemporaryFile instead of TemporaryFile as Django’s ImageField requires file name.
Iterate over your in-memory file and write blocks of data to the NamedTemporaryFile object.
Flush the file to ensure the file is written to the storage.
Change the temporary file to a Django’s File object.
You can assign this file to Django models directly and save it.
>>> from blog.models import Blog
>>> b = Blog.objects.first()
>>> b.image = temp_file
>>> b.save()
I personally solved my SuspiciousFileOperation problems by addressing my directory with "BASE_DIR" from settings.py as the beginning of the path (nothing above that level in the filesystem), using a NamedTemporaryFile and by using the model FileField save() method appropriately like this:
# inside a model class save(self, *args, **kwargs) method
# file_name is the file name alone, no path to the file
self.myfilefield.save(file_name, temporary_file_object, save=False) # and then call the super().save(*args, **kwargs) inside the save() method of your model
I need to rename the file with the name of the variable.
I Have this:
def content_file_name_Left(instance, filename):
return 'user_{0}/Left/{1}'.format(instance.ID, filename)
...
user_ImageLeft = models.FileField(default='', upload_to=content_file_name_Left)
I want that its save in: user_x/Left/user_ImageLeft.[format]
I have 20 images and I don't want make 20 functions for write manually the name of the variable.
Thanks
Just tested this and the best way seems to be by using a deconstructible class (deconstructible is used to prevent migration errors):
#deconstructible
class PathAndUniqueFilename(object):
def __init__(self, sub_path):
self.path = sub_path
def __call__(self, instance, filename):
self.path = self.path.format(instance.user.id)
return os.path.join(self.path, filename)
and then call this in your model like so:
user_ImageLeft = models.FileField(default='', upload_to=PathAndUniqueFilename('user_{0}/Left/'))
What this does is take the parameters of PathAndUniqueFilename('user_{0}/Left/'), and uses format() in the deconstructible in order to add a custom folder name.
I have the following code to delete a file:
from django.db import models
from django import forms
import os
class Document(models.Model):
docfile = models.FileField(upload_to='documents/%Y/%m/%d')
def __unicode__(self):
return '%s' % (self.docfile.name)
def delete(self, *args, **kwargs):
os.rmdir(os.path.join(settings.MEDIA_ROOT, self.docfile.name))
super(Document,self).delete(*args,**kwargs)
It manages to delete the objects I ask it to in my views.py but when I reupload a file of the same name it seems as though the original file still exists since I'll get "output_1.txt" instead of "output.txt".
This is the code I use to delete:
def delete_matrix():
documents = Document.objects.all()
documents.delete()
Am I not deleting the file from the database? Any help would be appreciated.
Your problem is that you are overriding the delete() method on the model but you are calling the delete method on the QuerySet returned by the default manager (Documents.object.all().delete()). These are 2 separate methods so there are 2 ways of fixing this.
1.In the delete method of the model, replace the line
os.rmdir(os.path.join(settings.MEDIA_ROOT, self.docfile.name))
by
os.remove(os.path.join(settings.MEDIA_ROOT, self.docfile.name))
AND, call the delete method for each object separately. Replace
Document.objects.all().delete()
with
documents = Document.objects.all()
for document in documents:
document.delete()
2.Replace the default manager to return a custom QuerySet which overrides the delete() method. This is explained in Overriding QuerySet.delete() in Django
Try this
document = Document.objects.get(pk=pk)
# if `save`=True, changes are saved to the db else only the file is deleted
document.docfile.delete(save=True)
here is another solution
def delete(self, *args, **kwargs):
os.remove(os.path.join(settings.MEDIA_ROOT, self.qr_code.name))
super().delete(*args, **kwargs)
You can use a much simpler code:
def delete(self, *args, **kwargs):
if self.docfile:
self.docfile.delete()
super().delete(*args, **kwargs)
I am using 1.2.5 with a standard ImageField and using the built-in storage backend. Files upload fine but when I remove an entry from admin the actual file on the server does not delete.
You can receive the pre_delete or post_delete signal (see #toto_tico's comment below) and call the delete() method on the FileField object, thus (in models.py):
class MyModel(models.Model):
file = models.FileField()
...
# Receive the pre_delete signal and delete the file associated with the model instance.
from django.db.models.signals import pre_delete
from django.dispatch.dispatcher import receiver
#receiver(pre_delete, sender=MyModel)
def mymodel_delete(sender, instance, **kwargs):
# Pass false so FileField doesn't save the model.
instance.file.delete(False)
Try django-cleanup
pip install django-cleanup
settings.py
INSTALLED_APPS = (
...
'django_cleanup.apps.CleanupConfig',
)
Django 1.5 solution: I use post_delete for various reasons that are internal to my app.
from django.db.models.signals import post_delete
from django.dispatch import receiver
#receiver(post_delete, sender=Photo)
def photo_post_delete_handler(sender, **kwargs):
photo = kwargs['instance']
storage, path = photo.original_image.storage, photo.original_image.path
storage.delete(path)
I stuck this at the bottom of the models.py file.
the original_image field is the ImageField in my Photo model.
This code runs well on Django 1.4 also with the Admin panel.
class ImageModel(models.Model):
image = ImageField(...)
def delete(self, *args, **kwargs):
# You have to prepare what you need before delete the model
storage, path = self.image.storage, self.image.path
# Delete the model before the file
super(ImageModel, self).delete(*args, **kwargs)
# Delete the file after the model
storage.delete(path)
It's important to get the storage and the path before delete the model or the latter will persist void also if deleted.
You need to remove the actual file on both delete and update.
from django.db import models
class MyImageModel(models.Model):
image = models.ImageField(upload_to='images')
def remove_on_image_update(self):
try:
# is the object in the database yet?
obj = MyImageModel.objects.get(id=self.id)
except MyImageModel.DoesNotExist:
# object is not in db, nothing to worry about
return
# is the save due to an update of the actual image file?
if obj.image and self.image and obj.image != self.image:
# delete the old image file from the storage in favor of the new file
obj.image.delete()
def delete(self, *args, **kwargs):
# object is being removed from db, remove the file from storage first
self.image.delete()
return super(MyImageModel, self).delete(*args, **kwargs)
def save(self, *args, **kwargs):
# object is possibly being updated, if so, clean up.
self.remove_on_image_update()
return super(MyImageModel, self).save(*args, **kwargs)
You may consider using a pre_delete or post_delete signal:
https://docs.djangoproject.com/en/dev/topics/signals/
Of course, the same reasons that FileField automatic deletion was removed also apply here. If you delete a file that is referenced somewhere else you will have problems.
In my case this seemed appropriate because I had a dedicated File model to manage all of my files.
Note: For some reason post_delete doesn't seem to work right. The file got deleted, but the database record stayed, which is completely the opposite of what I would expect, even under error conditions. pre_delete works fine though.
Maybe it's a little late. But the easiest way for me is to use a post_save signal. Just to remember that signals are excecuted even during a QuerySet delete process, but the [model].delete() method is not excecuted during the QuerySet delete process, so it's not the best option to override it.
core/models.py:
from django.db import models
from django.db.models.signals import post_delete
from core.signals import delete_image_slide
SLIDE1_IMGS = 'slide1_imgs/'
class Slide1(models.Model):
title = models.CharField(max_length = 200)
description = models.CharField(max_length = 200)
image = models.ImageField(upload_to = SLIDE1_IMGS, null = True, blank = True)
video_embed = models.TextField(null = True, blank = True)
enabled = models.BooleanField(default = True)
"""---------------------------- SLIDE 1 -------------------------------------"""
post_delete.connect(delete_image_slide, Slide1)
"""--------------------------------------------------------------------------"""
core/signals.py
import os
def delete_image_slide(sender, **kwargs):
slide = kwargs.get('instance')
try:
os.remove(slide.image.path)
except:
pass
This functionality will be removed in Django 1.3 so I wouldn't rely on it.
You could override the delete method of the model in question to delete the file before removing the entry from the database completely.
Edit:
Here is a quick example.
class MyModel(models.Model):
self.somefile = models.FileField(...)
def delete(self, *args, **kwargs):
somefile.delete()
super(MyModel, self).delete(*args, **kwargs)
Using the post_delete is for sure the right way to go. Sometimes though things can go wrong, and files don't get deleted. There is of course the case that you have a bunch of old files that weren't deleted before post_delete was used. I created a function that deletes files for objects based on if the file the object references does not exist then delete object, if the file does not have an object, then also delete, also it can delete based on an "active" flag for an object.. Something I added to most of my models. You have to pass it the objects you want to check, the path to the objects files, the file field and a flag to delete inactive objects:
def cleanup_model_objects(m_objects, model_path, file_field='image', clear_inactive=False):
# PART 1 ------------------------- INVALID OBJECTS
#Creates photo_file list based on photo path, takes all files there
model_path_list = os.listdir(model_path)
#Gets photo image path for each photo object
model_files = list()
invalid_files = list()
valid_files = list()
for obj in m_objects:
exec("f = ntpath.basename(obj." + file_field + ".path)") # select the appropriate file/image field
model_files.append(f) # Checks for valid and invalid objects (using file path)
if f not in model_path_list:
invalid_files.append(f)
obj.delete()
else:
valid_files.append(f)
print "Total objects", len(model_files)
print "Valid objects:", len(valid_files)
print "Objects without file deleted:", len(invalid_files)
# PART 2 ------------------------- INVALID FILES
print "Files in model file path:", len(model_path_list)
#Checks for valid and invalid files
invalid_files = list()
valid_files = list()
for f in model_path_list:
if f not in model_files:
invalid_files.append(f)
else:
valid_files.append(f)
print "Valid files:", len(valid_files)
print "Files without model object to delete:", len(invalid_files)
for f in invalid_files:
os.unlink(os.path.join(model_path, f))
# PART 3 ------------------------- INACTIVE PHOTOS
if clear_inactive:
#inactive_photos = Photo.objects.filter(active=False)
inactive_objects = m_objects.filter(active=False)
print "Inactive Objects to Delete:", inactive_objects.count()
for obj in inactive_objects:
obj.delete()
print "Done cleaning model."
This is how you can use this:
photos = Photo.objects.all()
photos_path, tail = ntpath.split(photos[0].image.path) # Gets dir of photos path, this may be different for you
print "Photos -------------->"
cleanup_model_objects(photos, photos_path, file_field='image', clear_inactive=False) # image file is default
make sure you write "self" before the file. so example above should be
def delete(self, *args, **kwargs):
self.somefile.delete()
super(MyModel, self).delete(*args, **kwargs)
I've forgotten the "self" before my file and that didn't work as it was looking in the global namespace.
If you already have number of unused files in your project and want to delete them, you can use django utility django-unused-media
Django 2.x Solution:
There's no need to install any packages! It's very easy to handle in Django 2. I've tried following solution using Django 2 and SFTP Storage (however I think it would work with any storages)
First write a Custom Manager. So if you want to be able to delete files of a model by using objects methods, you must write and use a [Custom Manager][3] (for overriding delete() method of objects):
class CustomManager(models.Manager):
def delete(self):
for obj in self.get_queryset():
obj.delete()
Now you must delete image before deleting deleting the model itself and for assigning the CustomManager to the model, you must initial objects inside your model:
class MyModel(models.Model):
image = models.ImageField(upload_to='/pictures/', blank=True)
objects = CustomManager() # add CustomManager to model
def delete(self, using=None, keep_parents=False):
objects = CustomManager() # just add this line of code inside of your model
def delete(self, using=None, keep_parents=False):
self.image.storage.delete(self.song.name)
super().delete()
I may have a special case since I am using the upload_to option on my file field with dynamic directory names but the solution I found was to use os.rmdir.
In models:
import os
...
class Some_Model(models.Model):
save_path = models.CharField(max_length=50)
...
def delete(self, *args,**kwargs):
os.rmdir(os.path.join(settings.MEDIA_ROOT, self.save_path)
super(Some_Model,self).delete(*args, **kwargs)