Concept
First of all i have a pretty komplex model... The idea is of building a Starship as model. So every Ship has a type(ship_type) which is based on a Blueprint of this Shiptype. So when u're creating a Ship u have to decide which ship_type the model should use (foreign key).
Because I want to change a ship (example: buy another software) but not the blueprint itself, every Ship has the same fields in the database as the Blueprint Ship (both inherit from ShipField ). When someone set the ship_type or change it, I want Django to go to the blueprint get all informations and overwrite the information of the ship. So I tried to accomplish this behavior in the save method.
Function and error search
The function I wrote is always triggerd when there was a change in self.ship_type, so far so good. And all "normal" fields are changed too, only the many to many fields don't work.
I dove into it and got confused. Let us assume our ship has no entries in self.software but the new ship_type has 3. If I print out the self.software before the save (1.), I got as expected an empty queryset. When I do it after the super.save (2.) I got a queryset of three elements. So it seems to work everything. But if I take a look at the ship in the admin menu, the ship has no software at all.
Conclusion
So my conclusion is that somewhere after the save method (perhaps in the post_save event) the software get deleted again... At this point I need some help.
Ideas
I hope you guys understand what I'm trying to do here. I am not a database expert and can imagine that there are better ways to achieve this so I'm open for radical changes.
Models (simplified):
class ShipFields(models.Model):
body = models.ForeignKey(to=Body, verbose_name="Body", on_delete=models.SET_NULL,
null=True, blank=False, default=None)
software = models.ManyToManyField(Software, default=None, blank=True)
...
class ShipBlueprints(ShipFields, models.Model):
class Meta:
ordering = ['name']
verbose_name = "Ship Blueprint"
verbose_name_plural = "Ship Blueprints"
class Ship(ShipFields, models.Model):
name = models.CharField(max_length=256, unique=True)
ship_type = models.ForeignKey(to=ShipBlueprints, on_delete=models.SET_NULL, null=True)
...
__original_ship_type = None
def __init__(self, *args, **kwargs):
super(Ship, self).__init__(*args, **kwargs)
self.__original_ship_type = self.ship_type
def save(self, force_insert=False, force_update=False, *args, **kwargs):
# check if the ship type is changed
if self.ship_type != self.__original_ship_type:
...
# copy all fields from the related ShipBlueprints to the fields of Ship
# 1.
print(self.software.all())
self.body = self.ship_type.body
self.software.set(self.ship_type.software.all())
# or
# for soft in self.ship_type.software.all():
# self.software.add(soft)
# 2.
print(self.software.all())
...
super(Ship, self).save(force_insert, force_update, *args, **kwargs)
# 2.
print(self.software.all())
print(Ship.objects.get(name=self.name).software.all())
self.__original_ship_type = self.ship_type
I think i narrowed the problem down. When i change the ship_type via admin side, the many_to_many_fields dont get updated. BUT when i change the ship_type via the django shell it works perfectly!
When im using a form in a view it works, too. So my code works justfine but the admin page is somehow the problem... For me that's ok but it looks like an error in the saving method of admin, perhaps i will report this.
Thank you all for the ideas.
Hmm I believe it's because of __original_ship_type. It needs to be a foreign key to ShipBlueprints too. The __original_ship_type won't save anything inside of each row since it's not a database attribute/column. The __original_ship_type is simply part of the model for a database record.
That's actually one option. The other option is to use Django signals, specifically, pre_save. When you use pre_save, you are getting the NEW instance and you use this NEW instance's ID to do a DB query for the old instance. Afterwards, you save the object like that. The pre_save signal does not necessarily mean you HAVE to save something in there. It's just a signal calling a function.
Can you please tell me the approach to implement a TTL on Django model record?
For Example:Remove all values stored over more than x minutes.
The first thing you would need to do is to add a field to your model that tracks the time the record was created. Passing auto_now_add=True to a DateTimeField means that it will automatically be populated by the time the record was created
class Values(models.Model):
key = models.CharField(max_length=20)
value = models.TextField(blank='False',default='')
created = models.DateTimeField(auto_now_add=True)
def __str__(self):
return self.key
Then you need a function that will delete records that are older than a certain age, a classmethod seems appropriate
#classmethod
def delete_old_records(cls, minutes=5):
cut_off = datetime.datetime.now() - datetime.timedelta(minutes=minutes)
cls.objects.filter(created__lt=cut_off).delete()
Now you need to add a method for calling this function from a script, a custom management command is ideal
class Command(BaseCommand):
help = 'Deletes old Values'
def handle(self, *args, **options):
Values.delete_old_records()
Then you need to schedule this command, something like cron or celerybeat would be good for this
I have this modesl.py below. I need to execute something only when the record is being updated and not created for Target model(e.g. when I update Target.Name via admin). So far the below code executes only when I create new Target record, not update existing one. Spent one day on this...
def create_badge(sender, instance, created, **kwargs):
if not created:
#execute stuff here if record being UPDATED
os.system('touch /tmp/mark')
pass
else:
os.system('touch /tmp/mark2')
class Target(models.Model):
Name = models.CharField(max_length=20)
UID = models.CharField(max_length=15)
SSH = models.CharField(max_length=400)
signals.post_save.connect(create_badge, sender=Target)
Use a custom save() method on the model. Then you can take advantage of the fact that, when the instance is being modified, it will already have a id, but if it's being created it won't yet (because the id is assigned when it's saved for the first time). So set your function to be called only if there's already a id. For example:
class ModelName(models.Model):
...
def save(self):
if self.id:
do_something_here()
super(ModelName, self).save()
I would like to upload images to the media root based on the field values given by the Django admin user. Here is the code that I've written and I know that the upload_to parameter is causing the problem. But I don't know how to make it work.
models.py
class Info(models.Model):
file_no = models.CharField(max_length=100)
date = models.DateField()
area = models.IntegerField(default=0.00)
mouja = models.CharField(max_length=100)
doc_type_choices = (
('deed', 'Deed'),
('khotian', 'Khotian'),
)
doc_type = models.CharField(max_length=50,
choices=doc_type_choices,
default='deed')
doc_no = models.CharField(max_length=50)
def __unicode__(self):
return self.file_no
class Image(models.Model):
info = models.ForeignKey('Info')
content = models.ImageField(upload_to=self.info.mouja/self.info.doc_type)
def __unicode__(self):
return self.info.file_no
Whenever I run python manage.py makemigrations it shows NameError: name 'self' is not defined
Thanks in advance for any help!
In the upload_to keyword you would need to provide a function that you will define, for instance:
def path_file_name(instance, filename):
return '/'.join(filter(None, (instance.info.mouja, instance.info.doc_type, filename)))
class Image(models.Model):
content = models.ImageField(upload_to=path_file_name)
From Django documentation: Model field reference:
This may also be a callable, such as a function, which will be called to obtain the upload path, including the filename. This callable must be able to accept two arguments, and return a Unix-style path (with forward slashes) to be passed along to the storage system.
Within this callable, which in the particular case is path_file_name function, we build a path from the instance field which is the particular record of Image model.
The filter function removes any None items out of the list and the join function constructs the path by joining all list items with /.
Here is the original code that worked. Just in case anyone needs it.
def path_file_name(instance, filename):
return '/'.join(filter(None, (instance.info.mouja, instance.info.doc_type, filename)))
I'm building a web app in Django. I have a model that uploads a file, but I can not delete the file. Here is my code:
class Song(models.Model):
name = models.CharField(blank=True, max_length=100)
author = models.ForeignKey(User, to_field='id', related_name="id_user2")
song = models.FileField(upload_to='/songs/')
image = models.ImageField(upload_to='/pictures/', blank=True)
date_upload = models.DateField(auto_now_add=True)
def delete(self, *args, **kwargs):
# You have to prepare what you need before delete the model
storage, path = self.song.storage, self.song.path
# Delete the model before the file
super(Song, self).delete(*args, **kwargs)
# Delete the file after the model
storage.delete(path)
Then, in python manage.py shell I do this:
song = Song.objects.get(pk=1)
song.delete()
It deletes the record from the database but not the file on server.
What else can I try?
Thanks!
Before Django 1.3, the file was deleted from the filesystem automatically when you deleted the corresponding model instance. You are probably using a newer Django version, so you'll have to implement deleting the file from the filesystem yourself.
Simple signal-based sample
My method of choice at the time of writing is a mix of post_delete and pre_save signals, which makes it so that obsolete files are deleted whenever corresponding models are deleted or have their files changed.
Based on a hypothetical MediaFile model:
import os
import uuid
from django.db import models
from django.dispatch import receiver
from django.utils.translation import ugettext_lazy as _
class MediaFile(models.Model):
file = models.FileField(_("file"),
upload_to=lambda instance, filename: str(uuid.uuid4()))
# These two auto-delete files from filesystem when they are unneeded:
#receiver(models.signals.post_delete, sender=MediaFile)
def auto_delete_file_on_delete(sender, instance, **kwargs):
"""
Deletes file from filesystem
when corresponding `MediaFile` object is deleted.
"""
if instance.file:
if os.path.isfile(instance.file.path):
os.remove(instance.file.path)
#receiver(models.signals.pre_save, sender=MediaFile)
def auto_delete_file_on_change(sender, instance, **kwargs):
"""
Deletes old file from filesystem
when corresponding `MediaFile` object is updated
with new file.
"""
if not instance.pk:
return False
try:
old_file = MediaFile.objects.get(pk=instance.pk).file
except MediaFile.DoesNotExist:
return False
new_file = instance.file
if not old_file == new_file:
if os.path.isfile(old_file.path):
os.remove(old_file.path)
I think one of the apps I’ve built a while back used this code in production, but nevertheless use at your own risk.
For example, there’s a possible data loss scenario: your data might end up referencing a nonexistent file if your save() method call happens to be within a transaction that gets rolled back. You could consider wrapping file-removing logic into transaction.on_commit(), along the lines of transaction.on_commit(lambda: os.remove(old_file.path)), as suggested in Mikhail’s comment. django-cleanup library does something along those lines.
Edge case: if your app uploads a new file and points model instance to the new file without calling save() (e.g. by bulk updating a QuerySet), the old file will keep lying around because signals won’t be run. This doesn’t happen if you use conventional file handling methods.
Coding style: this example uses file as field name, which is not a good style because it clashes with the built-in file object identifier.
Addendum: periodic cleanup
Realistically, you may want to also run a periodic task to handle orphan file cleanup in case a runtime failure prevents some file from being removed. With that in mind, you could probably get rid of signal handlers altogether, and make such a task the mechanism for dealing with insensitive data and not-so-large files.
Either way though, if you are handling sensitive data, it’s always better to double- or triple- check that you never fail to timely delete data in production to avoid any associated liabilities.
See also
FieldFile.delete() in Django 1.11 model field reference (note that it describes the FieldFile class, but you’d call .delete() directly on the field: FileField instance proxies to the corresponding FieldFile instance, and you access its methods as if they were field’s)
Note that when a model is deleted, related files are not deleted. If you need to cleanup orphaned files, you’ll need to handle it yourself (for instance, with a custom management command that can be run manually or scheduled to run periodically via e.g. cron).
Why Django doesn’t delete files automatically: entry in release notes for Django 1.3
In earlier Django versions, when a model instance containing a FileField was deleted, FileField took it upon itself to also delete the file from the backend storage. This opened the door to several data-loss scenarios, including rolled-back transactions and fields on different models referencing the same file. In Django 1.3, when a model is deleted the FileField’s delete() method won’t be called. If you need cleanup of orphaned files, you’ll need to handle it yourself (for instance, with a custom management command that can be run manually or scheduled to run periodically via e.g. cron).
Example of using a pre_delete signal only
Try django-cleanup, it automatically invokes delete method on FileField when you remove model.
pip install django-cleanup
settings.py
INSTALLED_APPS = (
...
'django_cleanup.apps.CleanupConfig',
)
You can delete file from filesystem with calling .delete method of file field shown as below with Django >= 1.10:
obj = Song.objects.get(pk=1)
obj.song.delete()
You can also simply overwrite the delete function of the model to check for file if it exists and delete it before calling the super function.
import os
class Excel(models.Model):
upload_file = models.FileField(upload_to='/excels/', blank =True)
uploaded_on = models.DateTimeField(editable=False)
def delete(self,*args,**kwargs):
if os.path.isfile(self.upload_file.path):
os.remove(self.upload_file.path)
super(Excel, self).delete(*args,**kwargs)
Django 2.x Solution:
It's very easy to handle file deletion in Django 2. I've tried following solution using Django 2 and SFTP Storage and also FTP STORAGE, and I'm pretty sure that it'll work with any other storage managers which implemented delete method. (delete method is one of the storage abstract methods which is supposed to delete the file from the storage, physically!)
Override the delete method of the model in a way that the instance deletes its FileFields before deleting itself:
class Song(models.Model):
name = models.CharField(blank=True, max_length=100)
author = models.ForeignKey(User, to_field='id', related_name="id_user2")
song = models.FileField(upload_to='/songs/')
image = models.ImageField(upload_to='/pictures/', blank=True)
date_upload = models.DateField(auto_now_add=True)
def delete(self, using=None, keep_parents=False):
self.song.storage.delete(self.song.name)
self.image.storage.delete(self.image.name)
super().delete()
It works pretty easy for me.
If you want to check if file exists before deletion, you can use storage.exists. e.g. self.song.storage.exists(self.song.name) will return a boolean representing if the song exists. So it will look like this:
def delete(self, using=None, keep_parents=False):
# assuming that you use same storage for all files in this model:
storage = self.song.storage
if storage.exists(self.song.name):
storage.delete(self.song.name)
if storage.exists(self.image.name):
storage.delete(self.image.name)
super().delete()
EDIT (In Addition):
As #HeyMan mentioned, with this solution calling Song.objects.all().delete() does not delete files! This is happening because Song.objects.all().delete() is running delete query of Default Manager. So if you want to be able to delete files of a model by using objects methods, you must write and use a Custom Manager (just for overriding its delete query):
class CustomManager(models.Manager):
def delete(self):
for obj in self.get_queryset():
obj.delete()
and for assigning the CustomManager to the model, you must initial objects inside your model:
class Song(models.Model):
name = models.CharField(blank=True, max_length=100)
author = models.ForeignKey(User, to_field='id', related_name="id_user2")
song = models.FileField(upload_to='/songs/')
image = models.ImageField(upload_to='/pictures/', blank=True)
date_upload = models.DateField(auto_now_add=True)
objects = CustomManager() # just add this line of code inside of your model
def delete(self, using=None, keep_parents=False):
self.song.storage.delete(self.song.name)
self.image.storage.delete(self.image.name)
super().delete()
Now you can use .delete() in the end of any objects sub-queries. I wrote the simplest CustomManager, but you can do it better by returning something about objects you deleted or whatever you want.
Here is an app that will remove old files whenever model is deleted or a new file is uploaded: django-smartfields
from django.db import models
from smartfields import fields
class Song(models.Model):
song = fields.FileField(upload_to='/songs/')
image = fields.ImageField(upload_to='/pictures/', blank=True)
For those who look for an answer in a newer version of Django (currently 3.1).
I found this website and it worked for me without any changes, just add it in your models.py:
from django.db.models.signals import post_delete
from django.dispatch import receiver
from django.db import models
""" Only delete the file if no other instances of that model are using it"""
def delete_file_if_unused(model,instance,field,instance_file_field):
dynamic_field = {}
dynamic_field[field.name] = instance_file_field.name
other_refs_exist = model.objects.filter(**dynamic_field).exclude(pk=instance.pk).exists()
if not other_refs_exist:
instance_file_field.delete(False)
""" Whenever ANY model is deleted, if it has a file field on it, delete the associated file too"""
#receiver(post_delete)
def delete_files_when_row_deleted_from_db(sender, instance, **kwargs):
for field in sender._meta.concrete_fields:
if isinstance(field,models.FileField):
instance_file_field = getattr(instance,field.name)
delete_file_if_unused(sender,instance,field,instance_file_field)
""" Delete the file if something else get uploaded in its place"""
#receiver(pre_save)
def delete_files_when_file_changed(sender,instance, **kwargs):
# Don't run on initial save
if not instance.pk:
return
for field in sender._meta.concrete_fields:
if isinstance(field,models.FileField):
#its got a file field. Let's see if it changed
try:
instance_in_db = sender.objects.get(pk=instance.pk)
except sender.DoesNotExist:
# We are probably in a transaction and the PK is just temporary
# Don't worry about deleting attachments if they aren't actually saved yet.
return
instance_in_db_file_field = getattr(instance_in_db,field.name)
instance_file_field = getattr(instance,field.name)
if instance_in_db_file_field.name != instance_file_field.name:
delete_file_if_unused(sender,instance,field,instance_in_db_file_field)
#Anton Strogonoff
I missing something in the code when a file change, if you create a new file generate an error, becuase is a new file a did not find a path. I modified the code of function and added a try/except sentence and it works well.
#receiver(models.signals.pre_save, sender=MediaFile)
def auto_delete_file_on_change(sender, instance, **kwargs):
"""Deletes file from filesystem
when corresponding `MediaFile` object is changed.
"""
if not instance.pk:
return False
try:
old_file = MediaFile.objects.get(pk=instance.pk).file
except MediaFile.DoesNotExist:
return False
new_file = instance.file
if not old_file == new_file:
try:
if os.path.isfile(old_file.path):
os.remove(old_file.path)
except Exception:
return False
This code will run every time i upload a new image (logo field) and check if a logo already exists if so, close it and remove it from disk. The same procedure could of course be made in receiver function. Hope this helps.
# Returns the file path with a folder named by the company under /media/uploads
def logo_file_path(instance, filename):
company_instance = Company.objects.get(pk=instance.pk)
if company_instance.logo:
logo = company_instance.logo
if logo.file:
if os.path.isfile(logo.path):
logo.file.close()
os.remove(logo.path)
return 'uploads/{0}/{1}'.format(instance.name.lower(), filename)
class Company(models.Model):
name = models.CharField(_("Company"), null=False, blank=False, unique=True, max_length=100)
logo = models.ImageField(upload_to=logo_file_path, default='')