I'm building a web app in Django. I have a model that uploads a file, but I can not delete the file. Here is my code:
class Song(models.Model):
name = models.CharField(blank=True, max_length=100)
author = models.ForeignKey(User, to_field='id', related_name="id_user2")
song = models.FileField(upload_to='/songs/')
image = models.ImageField(upload_to='/pictures/', blank=True)
date_upload = models.DateField(auto_now_add=True)
def delete(self, *args, **kwargs):
# You have to prepare what you need before delete the model
storage, path = self.song.storage, self.song.path
# Delete the model before the file
super(Song, self).delete(*args, **kwargs)
# Delete the file after the model
storage.delete(path)
Then, in python manage.py shell I do this:
song = Song.objects.get(pk=1)
song.delete()
It deletes the record from the database but not the file on server.
What else can I try?
Thanks!
Before Django 1.3, the file was deleted from the filesystem automatically when you deleted the corresponding model instance. You are probably using a newer Django version, so you'll have to implement deleting the file from the filesystem yourself.
Simple signal-based sample
My method of choice at the time of writing is a mix of post_delete and pre_save signals, which makes it so that obsolete files are deleted whenever corresponding models are deleted or have their files changed.
Based on a hypothetical MediaFile model:
import os
import uuid
from django.db import models
from django.dispatch import receiver
from django.utils.translation import ugettext_lazy as _
class MediaFile(models.Model):
file = models.FileField(_("file"),
upload_to=lambda instance, filename: str(uuid.uuid4()))
# These two auto-delete files from filesystem when they are unneeded:
#receiver(models.signals.post_delete, sender=MediaFile)
def auto_delete_file_on_delete(sender, instance, **kwargs):
"""
Deletes file from filesystem
when corresponding `MediaFile` object is deleted.
"""
if instance.file:
if os.path.isfile(instance.file.path):
os.remove(instance.file.path)
#receiver(models.signals.pre_save, sender=MediaFile)
def auto_delete_file_on_change(sender, instance, **kwargs):
"""
Deletes old file from filesystem
when corresponding `MediaFile` object is updated
with new file.
"""
if not instance.pk:
return False
try:
old_file = MediaFile.objects.get(pk=instance.pk).file
except MediaFile.DoesNotExist:
return False
new_file = instance.file
if not old_file == new_file:
if os.path.isfile(old_file.path):
os.remove(old_file.path)
I think one of the apps I’ve built a while back used this code in production, but nevertheless use at your own risk.
For example, there’s a possible data loss scenario: your data might end up referencing a nonexistent file if your save() method call happens to be within a transaction that gets rolled back. You could consider wrapping file-removing logic into transaction.on_commit(), along the lines of transaction.on_commit(lambda: os.remove(old_file.path)), as suggested in Mikhail’s comment. django-cleanup library does something along those lines.
Edge case: if your app uploads a new file and points model instance to the new file without calling save() (e.g. by bulk updating a QuerySet), the old file will keep lying around because signals won’t be run. This doesn’t happen if you use conventional file handling methods.
Coding style: this example uses file as field name, which is not a good style because it clashes with the built-in file object identifier.
Addendum: periodic cleanup
Realistically, you may want to also run a periodic task to handle orphan file cleanup in case a runtime failure prevents some file from being removed. With that in mind, you could probably get rid of signal handlers altogether, and make such a task the mechanism for dealing with insensitive data and not-so-large files.
Either way though, if you are handling sensitive data, it’s always better to double- or triple- check that you never fail to timely delete data in production to avoid any associated liabilities.
See also
FieldFile.delete() in Django 1.11 model field reference (note that it describes the FieldFile class, but you’d call .delete() directly on the field: FileField instance proxies to the corresponding FieldFile instance, and you access its methods as if they were field’s)
Note that when a model is deleted, related files are not deleted. If you need to cleanup orphaned files, you’ll need to handle it yourself (for instance, with a custom management command that can be run manually or scheduled to run periodically via e.g. cron).
Why Django doesn’t delete files automatically: entry in release notes for Django 1.3
In earlier Django versions, when a model instance containing a FileField was deleted, FileField took it upon itself to also delete the file from the backend storage. This opened the door to several data-loss scenarios, including rolled-back transactions and fields on different models referencing the same file. In Django 1.3, when a model is deleted the FileField’s delete() method won’t be called. If you need cleanup of orphaned files, you’ll need to handle it yourself (for instance, with a custom management command that can be run manually or scheduled to run periodically via e.g. cron).
Example of using a pre_delete signal only
Try django-cleanup, it automatically invokes delete method on FileField when you remove model.
pip install django-cleanup
settings.py
INSTALLED_APPS = (
...
'django_cleanup.apps.CleanupConfig',
)
You can delete file from filesystem with calling .delete method of file field shown as below with Django >= 1.10:
obj = Song.objects.get(pk=1)
obj.song.delete()
You can also simply overwrite the delete function of the model to check for file if it exists and delete it before calling the super function.
import os
class Excel(models.Model):
upload_file = models.FileField(upload_to='/excels/', blank =True)
uploaded_on = models.DateTimeField(editable=False)
def delete(self,*args,**kwargs):
if os.path.isfile(self.upload_file.path):
os.remove(self.upload_file.path)
super(Excel, self).delete(*args,**kwargs)
Django 2.x Solution:
It's very easy to handle file deletion in Django 2. I've tried following solution using Django 2 and SFTP Storage and also FTP STORAGE, and I'm pretty sure that it'll work with any other storage managers which implemented delete method. (delete method is one of the storage abstract methods which is supposed to delete the file from the storage, physically!)
Override the delete method of the model in a way that the instance deletes its FileFields before deleting itself:
class Song(models.Model):
name = models.CharField(blank=True, max_length=100)
author = models.ForeignKey(User, to_field='id', related_name="id_user2")
song = models.FileField(upload_to='/songs/')
image = models.ImageField(upload_to='/pictures/', blank=True)
date_upload = models.DateField(auto_now_add=True)
def delete(self, using=None, keep_parents=False):
self.song.storage.delete(self.song.name)
self.image.storage.delete(self.image.name)
super().delete()
It works pretty easy for me.
If you want to check if file exists before deletion, you can use storage.exists. e.g. self.song.storage.exists(self.song.name) will return a boolean representing if the song exists. So it will look like this:
def delete(self, using=None, keep_parents=False):
# assuming that you use same storage for all files in this model:
storage = self.song.storage
if storage.exists(self.song.name):
storage.delete(self.song.name)
if storage.exists(self.image.name):
storage.delete(self.image.name)
super().delete()
EDIT (In Addition):
As #HeyMan mentioned, with this solution calling Song.objects.all().delete() does not delete files! This is happening because Song.objects.all().delete() is running delete query of Default Manager. So if you want to be able to delete files of a model by using objects methods, you must write and use a Custom Manager (just for overriding its delete query):
class CustomManager(models.Manager):
def delete(self):
for obj in self.get_queryset():
obj.delete()
and for assigning the CustomManager to the model, you must initial objects inside your model:
class Song(models.Model):
name = models.CharField(blank=True, max_length=100)
author = models.ForeignKey(User, to_field='id', related_name="id_user2")
song = models.FileField(upload_to='/songs/')
image = models.ImageField(upload_to='/pictures/', blank=True)
date_upload = models.DateField(auto_now_add=True)
objects = CustomManager() # just add this line of code inside of your model
def delete(self, using=None, keep_parents=False):
self.song.storage.delete(self.song.name)
self.image.storage.delete(self.image.name)
super().delete()
Now you can use .delete() in the end of any objects sub-queries. I wrote the simplest CustomManager, but you can do it better by returning something about objects you deleted or whatever you want.
Here is an app that will remove old files whenever model is deleted or a new file is uploaded: django-smartfields
from django.db import models
from smartfields import fields
class Song(models.Model):
song = fields.FileField(upload_to='/songs/')
image = fields.ImageField(upload_to='/pictures/', blank=True)
For those who look for an answer in a newer version of Django (currently 3.1).
I found this website and it worked for me without any changes, just add it in your models.py:
from django.db.models.signals import post_delete
from django.dispatch import receiver
from django.db import models
""" Only delete the file if no other instances of that model are using it"""
def delete_file_if_unused(model,instance,field,instance_file_field):
dynamic_field = {}
dynamic_field[field.name] = instance_file_field.name
other_refs_exist = model.objects.filter(**dynamic_field).exclude(pk=instance.pk).exists()
if not other_refs_exist:
instance_file_field.delete(False)
""" Whenever ANY model is deleted, if it has a file field on it, delete the associated file too"""
#receiver(post_delete)
def delete_files_when_row_deleted_from_db(sender, instance, **kwargs):
for field in sender._meta.concrete_fields:
if isinstance(field,models.FileField):
instance_file_field = getattr(instance,field.name)
delete_file_if_unused(sender,instance,field,instance_file_field)
""" Delete the file if something else get uploaded in its place"""
#receiver(pre_save)
def delete_files_when_file_changed(sender,instance, **kwargs):
# Don't run on initial save
if not instance.pk:
return
for field in sender._meta.concrete_fields:
if isinstance(field,models.FileField):
#its got a file field. Let's see if it changed
try:
instance_in_db = sender.objects.get(pk=instance.pk)
except sender.DoesNotExist:
# We are probably in a transaction and the PK is just temporary
# Don't worry about deleting attachments if they aren't actually saved yet.
return
instance_in_db_file_field = getattr(instance_in_db,field.name)
instance_file_field = getattr(instance,field.name)
if instance_in_db_file_field.name != instance_file_field.name:
delete_file_if_unused(sender,instance,field,instance_in_db_file_field)
#Anton Strogonoff
I missing something in the code when a file change, if you create a new file generate an error, becuase is a new file a did not find a path. I modified the code of function and added a try/except sentence and it works well.
#receiver(models.signals.pre_save, sender=MediaFile)
def auto_delete_file_on_change(sender, instance, **kwargs):
"""Deletes file from filesystem
when corresponding `MediaFile` object is changed.
"""
if not instance.pk:
return False
try:
old_file = MediaFile.objects.get(pk=instance.pk).file
except MediaFile.DoesNotExist:
return False
new_file = instance.file
if not old_file == new_file:
try:
if os.path.isfile(old_file.path):
os.remove(old_file.path)
except Exception:
return False
This code will run every time i upload a new image (logo field) and check if a logo already exists if so, close it and remove it from disk. The same procedure could of course be made in receiver function. Hope this helps.
# Returns the file path with a folder named by the company under /media/uploads
def logo_file_path(instance, filename):
company_instance = Company.objects.get(pk=instance.pk)
if company_instance.logo:
logo = company_instance.logo
if logo.file:
if os.path.isfile(logo.path):
logo.file.close()
os.remove(logo.path)
return 'uploads/{0}/{1}'.format(instance.name.lower(), filename)
class Company(models.Model):
name = models.CharField(_("Company"), null=False, blank=False, unique=True, max_length=100)
logo = models.ImageField(upload_to=logo_file_path, default='')
Related
there two basic ways to do something when an instance gets deleted:
Overwrite Model.delete
Signal
I used to reckon both of them serve the same purpose, just provides different ways of writing, but works exactly.
However, in this occasion, I realise I was wrong:
class Human(models.Model):
name = models.CharField(max_length=20)
class Pet(models.Model):
name = models.CharField(max_length=20)
owner = models.ForeignKey(Human, related_name="pet", on_delete=models.CASCADE)
def delete(self, *args, **kwargs):
print('------- Pet.delete is called')
return super().delete(*args, **kwargs)
h = Human(name='jason')
h.save()
p = Pet(name="dog", owner=h)
p.save()
h.delete()
# nothing is shown
Why Pet.delete Is not firing at Human.delete By the foreign cascade? Does I have to apply a signal on this? If so, would it cost more performance?
I am building something very heavy, comment system, filter decent records and delete when the commented target get deleted, the comment model has many null-able foreign key fields, with models.CASCADE Set, only one of them is assigned with value. But in product delete view, I call product.delete Then triggers cascade, but comment.delete Is not firing.
Currently, the project has delete Defined on many models, with assumption that it is always triggered when the instance get removed from database, and it is tremendous work to rewrite it in signal. Is there a way to call delete When at cascading? (I know it is likely impossible since it is a database field specification)
I implement a mix-in for Commendable models with extra methods defined, therefore, I decided to modify delete method to signal to something like this:
from django.db import models
from django.dispatch import receiver
from django.db.models.signals import pre_delete
# Create your models here.
class Base:
def __init_subclass__(cls):
#receiver(pre_delete, sender=cls)
def pet_pre_delete1(sender, instance, **kwargs):
print('pet pre delete 1 is called')
#receiver(pre_delete, sender=cls)
def pet_pre_delete2(sender, instance, **kwargs):
print('pet pre delete 2 is called')
class Human(models.Model):
name = models.CharField(max_length=20)
def __str__(self):
return f'<human>{self.name}'
class Pet(Base, models.Model):
name = models.CharField(max_length=20)
owner = models.ForeignKey(Human, related_name="pet", on_delete=models.CASCADE)
def __str__(self):
return f'<pet>{self.name}'
# ------- Pet.delete is called
# pet pre delete 1 is called
# pet pre delete 2 is called
it works fine in testing, I wonder if there is any risk using this, would it be garbage collected?
Suppose I have a model with a field that restricts another model. For example, a Thing whose name has to be in the current set of AllowedThingNames. The set of allowed_thing_names changes (infrequently) by an external source. I have code to get that external source and create AllowedThingNames as needed.
Here is some code:
models.py:
class AllowedThingName(models.Model):
name = models.CharField(max_length=100, unique=True)
class Thing(models.Model):
name = models.CharField(max_length=100)
def __save__(self, *args, **kwargs):
if AllowedThingName.objects.filter(name=self.name).exists():
return super().save(*args, **kwargs)
return None
tasks.py:
#shared_task
def import_new_names():
response = request.get("some/external/url/where/allowed/names/are/listed")
new_names = clever_way_of_parsing_response(response.content)
for new_name in new_names:
AllowedThingName.objects.get_or_create(name=new_name)
I have created a fixture of AllowedThingNames and I run the loaddata command at startup.
I'm not sure what the best way is to keep the set of AllowedThingNames up-to-date, though.
One solution is to periodically (via a task, or a cronjob, or whatever) call the import_new_names function above and then call dumpdata to overwrite the fixture. This will save it in the db and make sure that it gets re-loaded the next time the project restarts.
Does anybody in StackOverflow-Land have any better ideas?
Have you considered creating a ForeignKey relationship between Thing and AllowedThingName? Something along the lines of
class AllowedThingName(models.Model):
name = models.CharField(max_length=100, unique=True)
class Thing(models.Model):
name = models.ForeignKey(AllowedThingName, on_delete=models.CASCASE)
My Content model has a many-to-many relationship to the Tag model. When I save a Content object, I want to add the relationships dynamically. Im doing this the following way.
# models.py
def tag_content(content_id):
obj = Content.objects.get(pk=content_id)
print obj # Checking
obj.tags = [1, 2, 3] # Adding the relationships using the Tag IDs
class Tag(models.Model):
name = models.CharField(max_length=255)
class Content(models.Model):
title = models.CharField(max_length=255)
is_tagged = models.BooleanField(default=False)
tags = models.ManyToManyField(Tag, blank=True)
def save(self, *args, **kwargs):
super(Content, self).save(*args, **kwargs)
if not self.is_tagged:
tag_content(self.pk) # calling the tagging method
In other words, when a Content object is saved, its tags field is related to 3 different Tag object models. Just to let you know, I do have the Tags with pks = 1, 2, and 3 in database.
However, this simply doesn't work. The save method calls the tag_content method, since the print obj statement works. However, the many-to-many field is not set and remains empty. The funny thing is, If I run the following commands in shell, the tags field is set perfectly.
# python manage.py shell
from myapp.models import *
obj = Content.objects.get(pk=1)
tag_content(obj.pk)
So how come the shell version works, but the other one doesn't? Any help is appreciated.
You can't work on an m2m relationship in a custom save method because of the way Django writes those relationships to the database. When saving a model instance with an m2m relationship, Django first writes the object then goes in again and writes the appropriate m2m relationships. Since the m2m stuff comes "second," trying to work with the relationship in a custom save fails.
The solution is to use a post-save signal. Remove the custom save stuff and add this below your model definitions, making sure to import receiver and post_save:
#receiver(post_save, sender = Content)
def update_m2m_relationships_on_save(sender, **kwargs):
if not kwargs['instance'].is_tagged:
tag_content(kwargs['instance'].pk)
Your tag_content function should probably swap is_tagged to True and then save the instance; if that boolean is never flipped then this could just run in an endless loop. You can also just pass in the object instead of passing in the pk:
def tag_content(thing_to_tag):
thing_to_tag.tags.add([1,2,3])
thing_to_tag.is_tagged = True
thing_to_tag.save()
return thing_to_tag
Note the use of .add(), which is important when adding to an m2m relationship.
I need an admin on django to be able to upload an image that should be saved as /path/{self.id}.jpg
So far I have a database with id,status,title fields and a model with id,status,title,THUMB fields:
class MEL(models.Model):
id = models.AutoField(primary_key=True, editable=False)
status = models.IntegerField(choices=( (0, 'inactive'), (1, 'active') ), default=1)
title = models.TextField(verbose_name='Título')
thumb = models.ImageField(upload_to=upload_path)
class Meta:
db_table = u'MEL'
The problem is that the image is uploaded previously to the model being saved, so I can't save it to "self.id" at this moment.
I think I could save it to /tmp/{uuid} and then renaming it post-save, probably something like:
def upload_path(self, filename):
self.file_uuid = uuid.uuid4()
return '/tmp/' + self.file_uuid
and then a post-save that renames it and delete it from /tmp/
Or I could try overriding this model's save() method to first call super().save(), then process the image upload (which is in a ImageFieldFile object) and rename it to self.id
suggestions?
thanks
You should be able to change the filename using upload_to. I say this because I have done it in projects and the django documentation says so:
https://docs.djangoproject.com/en/dev/ref/models/fields/#django.db.models.FileField.upload_to
The third paragraph down of that link says this: "This may also be a callable, such as a function, which will be called to obtain the upload path, including the filename." [emphasis added]. I'm not trying to be sarcastic I'm just showing that this is and should be possible so something else must be going on.
try something like this. The problem could be that you were leaving off the extension:
def upload_path(self, filename):
basename, extension = os.path.splitext(filename)
return '/'.join(["path",("%s%s" % (self.id, extension))])
When a user uploads a file, it can be used by other users to attach to their project.
class DashFile(models.Model):
dash_version = models.ForeignKey(DashVersion)
title = models.CharField(max_length=255)
file = models.FileField(upload_to=get_fs_upload_path, max_length=255)
display_order = models.IntegerField(default=99)
Basically, I clone the DashFile and so, all the values remain the same with the exception of the dash_version.
Now, if a user wants to delete the file from their project, the file gets deleted. However, that deletes the file for all other users too who have made an association to that file.
So, how can I make it so that when a user 'deletes' the file from their project, the file doesn't actually get deleted?
I also know that the design choice is incorrect. I should have not used a ForeignKey but a many to many field. But I am where I am now.
Thanks
Eric
It depends on Django version that you use. From 1.2.5 version FileField never deletes files from storage backend on model.delete().
If you use Django<1.2.5 you can implement custom FileStorage:
from django.core.files import storage
class NoDeleteStorage(storage.FileSystemStorage):
def delete(self, *a,**kw):
pass
nodelete_storage = NoDeleteStorage()
class DashFile(models.Model):
dash_version = models.ForeignKey(DashVersion)
title = models.CharField(max_length=255)
file = models.FileField(upload_to=get_fs_upload_path,
storage=nodelete_storage,
max_length=255)
display_order = models.IntegerField(default=99)