How to track changes when using update() in Django models - django

I'm trying to keep track of the changes whenever a field is changed.
I can see the changes in Django Admin History whenever I use the .save() method, but whenever I use the .update() method it does not record whatever I changed in my object.
I want to use update() because it can change multiple fields at the same time. It makes the code cleaner and more efficient (one query, one line...)
Right now I'm using this:
u = Userlist.objects.filter(username=user['username']).update(**user)
I can see all the changes when I do
u = Userlist.objects.get(username=user['username'])
u.lastname=lastname
u.save()
I'm also using django-simple-history to see the changes.setup.

From the docs:
Finally, realize that update() does an update at the SQL level and,
thus, does not call any save() methods on your models, nor does it
emit the pre_save or post_save signals (which are a consequence of
calling Model.save())
update() works at the DB level, so Django admin cannot track changes when updates are applied via .update(...).
If you still want to track the changes on updates, you can use:
for user in Userlist.objects.filter(age__gt=40):
user.lastname = 'new name'
user.save()
This is however more expensive and is not advisable if the only benefit is tracking changes via the admin history.

Here's how I've handled this and it's worked well so far:
# get current model instance to update
instance = UserList.objects.get(username=username)
# use model_to_dict to convert object to dict (imported from django.forms.models import model_to_dict)
obj_dict = model_to_dict(instance)
# create instance of the model with this old data but do not save it
old_instance = UserList(**obj_dict)
# update the model instance (there are multiple ways to do this)
UserList.objects.filter(username=username).update(**user)
# get the updated object
updated_object = UserList.objects.get(id=id)
# get list of fields in the model class
my_model_fields = [field.name for field in cls._meta.get_fields()]
# get list of fields if they are different
differences = list(filter(lambda field: getattr(updated_object, field, None)!= getattr(old_instance, field, None), my_model_fields))
The differences variable will give you the list of fields that are different between the two instances. I also found it helpful to add which model fields I don't want to check for differences (e.g. we know the updated_date will always be changed, so we don't need to keep track of it).
skip_diff_fields = ['updated_date']
my_model_fields = []
for field in cls._meta.get_fields():
if field.name not in skip_diff_fields:
my_model_fields.append(field.name)

Related

How to get a model's last access date in Django?

I'm building a Django application, and in it I would like to track whenever a particular model was last accessed.
I'm opting for this in order to build a user activity history.
I know Django provides auto_now and auto_now_add, but these do not do what I want them to do. The latter tracks when a model was created, and the former tracks when it was last modified, which is different from when it was last accessed, mind you.
I've tried adding another datetime field to my model's specification:
accessed_on = models.DateTimeField()
Then I try to update the model's access manually by calling the following after each access:
model.accessed_on = datetime.utcnow()
model.save()
But it still won't work.
I've gone through the django documentation for an answer, but couldn't find one.
Help would be much appreciated.
What about creating a model with a field that contains the last save-date. Plus saving the object every time is translated from the DB representation to the python representation?
class YourModel(models.Model):
date_accessed = models.DateTimeField(auto_now=True)
#classmethod
def from_db(cls, db, field_names, values):
obj = super().from_db(db, field_names, values)
obj.save()
return obj

Django - copy and insert queryset clone using bulk_create

My goal is to create a clone of a queryset and then insert it into the database.
Following the suggestions of this post, I have the following code:
qs_new = copy.copy(qs)
MyModel.objects.bulk_create(qs_new)
However, with this code I run into duplicate primary key error. As for now, I only can come up with the following work-around:
qs_new = copy.copy(qs)
for x in qs_new:
x.id = None
MyModel.objects.bulk_create(qs_new)
Question: Can I implement this code snippet without going through loop ?
Can't think of a way without loop, but just a suggestion:
# add all fields here except 'id'
qs = qs.values('field1', 'field2', 'field3')
new_qs = [MyModel(**i) for i in qs]
MyModel.objects.bulk_create(new_qs)
Note that bulk_create behaves differently depending on the underlying database. With Postgres you get the new primary keys set:
Support for setting primary keys on objects created using
bulk_create() when using PostgreSQL was added.
https://docs.djangoproject.com/en/1.10/ref/models/querysets/#django.db.models.query.QuerySet.bulk_create
You should, however make sure that the objects you are creating either have no primary keys or only keys that are not taken yet. In the latter case you should run the code that sets the PKs as well as the bulk_create inside transaction.atomic().
Fetching the values explicitly as suggested by Shang Wang might be faster because only the given values are retrieved from the DB instead of fetching everything. If you have foreign key relations or m2m relations you might want to avoid simply throwing the complex instances into bulk_create but instead explicitly naming all attributes that are required when constructing a new MyModel instance.
Here an example:
class MyModel(Model):
name = TextField(...)
related = ForeignKeyField(...)
my_m2m = ManyToManyField(...)
In case of MyModel above, you would want to preserve the ForeignKey relations by specifying related_id and the PK of the related object in the constructor of MyModel, avoiding specifying related.
With m2m relations, you might end up skipping bulk_create altogether because you need each specific new PK, the corresponding original PK (from the instance that was copied) and the m2m relations of that original instance. Then you would have to create new m2m relations with the new PK and these mappings.
# add all fields here except 'id'
qs = qs.values('name', 'related_id')
MyModel.objects.bulk_create([MyModel(**i) for i in qs])
Note for completeness:
If you have overriden save() on your model (or if you are inheriting from 3rd party with custom save methods), it won't be executed and neither will any post_save handlers (yours or 3rd party).
I tried and you need a loop to set the id to None, then it works. so finally it may be like this:
qs_new = copy.copy(qs)
for q in qs_new:
q.id = None
# also, you can set other fields if you need
MyModel.objects.bulk_create(qs_new)
This works for me.

Concise way of getting or creating an object with given field values

Suppose I have:
from django.db import models
class MyContentClass(models.Model):
content = models.TextField()
another_field = models.TextField()
x = MyContentClass(content="Hello, world!", another_field="More Info")
Is there a more concise way to perform the following logic?
existing = MyContentClass.objects.filter(content=x.content, another_field=x.another_field)
if existing:
x = existing[0]
else:
x.save()
# x now points to an object which is saved to the DB,
# either one we've just saved there or one that already existed
# with the same field values we're interested in.
Specifically:
Is there a way to query for both (all) fields without specifying
each one separately?
Is there a better idiom for either getting the old object or saving the new one? Something like get_or_create, but which accepts an object as a parameter?
Assume the code which does the saving is separate from the code which generates the initial MyContentClass instance which we need to compare to. This is typical of a case where you have a function which returns a model object without also saving it.
You could convert x to a dictionary with
x_data = x.__dict__
Then that could be passed into the object's get_or_create method.
MyContentClass.objects.get_or_create(**x_data)
The problem with this is that there are a few fields that will cause this to error out (eg the unique ID, or the _state Django modelstate field). However, if you pop() those out of the dictionary beforehand, then you'd probably be good to go :)
cleaned_dict = remove_unneeded_fields(x_data)
MyContentClass.objects.get_or_create(**cleaned_dict)
def remove_unneeded_fields(x_data):
unneeded_fields = [
'_state',
'id',
# Whatever other fields you don't want the new obj to have
# eg any field marked as 'unique'
]
for field in unneeded_fields:
del x_data[field]
return x_data
EDIT
To avoid issues associated with having to maintain a whitelist/blacklist of fields you, could do something like this:
def remove_unneeded_fields(x_data, MyObjModel):
cleaned_data = {}
for field in MyObjModel._meta.fields:
if not field.unique:
cleaned_data[field.name] = x_data[field.name]
return cleaned_Data
There would probably have to be more validation than simply checking that the field is not unique, but this might offer some flexibility when it comes to minor model field changes.
I would suggest to create a custom manager for those models and add the functions you want to do with the models (like a custom get_or_create function).
https://docs.djangoproject.com/en/1.10/topics/db/managers/#custom-managers
This would be the cleanest way and involves no hacking. :)
You can create specific managers for specific models or create a superclass with functions you want for all models.
If you just want to add a second manager with a different name, beware that it will become the default manager if you don't set the objects manager first (https://docs.djangoproject.com/en/1.10/topics/db/managers/#default-managers)

Django form save with post_save signal causing conflict

I have a Physical_therapy_order model and an Event model (an event has foreignkey to Physical_therapy_order). I have a view which allows a user to create a new event. It also has a form with 3 fields from the Physical_therapy_order model.
def PTEventCreateView(request, pt_pk):
#get the pt order and create an a form for that order
pt_order = get_object_or_404(Physical_therapy_order, pk=pt_pk)
ptform = PT_schedule_form(instance=pt_order)
if request.POST:
eventform = PTEventForm(data=request.POST)
ptform = PT_schedule_form(data=request.POST, instance=pt_order)
if eventform.is_valid() and ptform.is_valid():
#I do some checks here that compare data across the two forms.
# if everything looks good i mark keep_saving=True so I can
# continue to save all the data provided in the two forms
if keep_saving:
ptform.save()
eventform.save()
#...send user to succss page
This works just FINE EXCEPT: my PTEvent model has a function attached to its post_save signal. This function pulls the event's related pt_order and makes some modifications to it. Now, if i save the eventform first then the changes from the signal don't happen. if i save the ptform first the ptform changes get discarded and the changes from the signal happen.
THIS IS IMPORTANT: The ptform is editing three entirely different fields than the post_save signal. So its not like they're modifying the same data, only the same model instance. I thought a form only saves the fields in its meta.fields attribute. Why would this be happening? Also, if i save the ptform first, then when eventsform is saved shouldn't the signal use the updated physical_therapy_order? I'm not sure if I'm even on the right track?
I think this is because of cached objects.
What I would suggest is
Save eventform first
Get new instance of pt_order either querying db or through saved instance of eventform
And then re-create form and save.
Sample code change:
# your code
if keep_saving:
evt = eventform.save()
# I'm not sure exact name of your field name for pt_order in Event model, change appropriately
newptform = PT_schedule_form(data=request.POST, instance= evt.pt_order)
newpt = newptform.save()

Django pre_save signal called when commit=False

I am writing a Django application that will track changes to the models, in a similar way to the admin interface. For example, I will be able to display a list of changes to a model, that look something like Changed Status from Open to Closed.
I am using the pre_save signal to do this, comparing the relevant fields between the existing item in the database, and the "instance" which is being saved. To get the existing item, I have to do sender._default_manager.get(pk=sender.pk) which seems a bit messy, but that part works.
The problem is, the view for changing this model calls the save() method on the form twice (first with commit=False) - this means that 2 changes get logged in the database, as the pre_save signal is emitted twice.
Is there any way I can accomplish this? Maybe in a different way altogether, though I remember reading that the Django admin app uses signals to track changes that users make.
Looking through the Django source, it seems that pre_save signals are sent on every call to save, even if commit is false. I would suggest inserting on the first pre_save, but add a flag column to the changes table, e.g.
class FooChanges(models.Model):
foo = models.ForeignKey(Foo)
dt = models.DateTimeField(default=datetime.now)
field = models.CharField(max_length=50)
value = models.CharField(max_length=50) # Or whatever is appropriate here
finished = models.BooleanField(default=False)
Then, your presave can be:
def pre_save_handler(sender, instance):
foo_changes, created = FooChanges.objects.get_or_create(foo=instance, finished=False, field='Status', value=instance.status)
if not created:
foo_changes.finished = True
foo_changes.save()
So on the first pre_save, you actually insert the change. On the second pass, you retrieve it from the database, and set the flag to false to make sure you don't pick it up the next time Foo's status changes.
use dispatch_uid:
http://docs.djangoproject.com/en/1.2/topics/signals/#preventing-duplicate-signals
Django Audit Log
django-audit-log is a pluggable app that does exactly what you want with little effort. I've used it in a project and I'll surely use it in many more now that I know it.