Django - How to save m2m data via post_save signal? - django

(Django 1.1) I have a Project model that keeps track of its members using a m2m field. It looks like this:
class Project(models.Model):
members = models.ManyToManyField(User)
sales_rep = models.ForeignKey(User)
sales_mgr = models.ForeignKey(User)
project_mgr = models.ForeignKey(User)
... (more FK user fields) ...
When the project is created, the selected sales_rep, sales_mgr, project_mgr, etc Users are added to members to make it easier to keep track of project permissions. This approach has worked very well so far.
The issue I am dealing with now is how to update the project's membership when one of the User FK fields is updated via the admin. I've tried various solutions to this problem, but the cleanest approach seemed to be a post_save signal like the following:
def update_members(instance, created, **kwargs):
"""
Signal to update project members
"""
if not created: #Created projects are handled differently
instance.members.clear()
members_list = []
if instance.sales_rep:
members_list.append(instance.sales_rep)
if instance.sales_mgr:
members_list.append(instance.sales_mgr)
if instance.project_mgr:
members_list.append(instance.project_mgr)
for m in members_list:
instance.members.add(m)
signals.post_save.connect(update_members, sender=Project)
However, the Project still has the same members even if I change one of the fields via the admin! I have had success updating members m2m fields using my own views in other projects, but I never had to make it play nice with the admin as well.
Is there another approach I should take other than a post_save signal to update membership? Thanks in advance for your help!
UPDATE:
Just to clarify, the post_save signal works correctly when I save my own form in the front end (old members are removed, and new ones added). However, the post_save signal does NOT work correctly when I save the project via the admin (members stay the same).
I think Peter Rowell's diagnosis is correct in this situation. If I remove the "members" field from the admin form the post_save signal works correctly. When the field is included, it saves the old members based on the values present in the form at the time of the save. No matter what changes I make to the members m2m field when project is saved (whether it be a signal or custom save method), it will always be overwritten by the members that were present in the form prior to the save. Thanks for pointing that out!

Having had the same problem, my solution is to use the m2m_changed signal. You can use it in two places, as in the following example.
The admin upon saving will proceed to:
save the model fields
emit the post_save signal
for each m2m:
emit pre_clear
clear the relation
emit post_clear
emit pre_add
populate again
emit post_add
Here you have a simple example that changes the content of the saved data before actually saving it.
class MyModel(models.Model):
m2mfield = ManyToManyField(OtherModel)
#staticmethod
def met(sender, instance, action, reverse, model, pk_set, **kwargs):
if action == 'pre_add':
# here you can modify things, for instance
pk_set.intersection_update([1,2,3])
# only save relations to objects 1, 2 and 3, ignoring the others
elif action == 'post_add':
print pk_set
# should contain at most 1, 2 and 3
m2m_changed.connect(receiver=MyModel.met, sender=MyModel.m2mfield.through)
You can also listen to pre_remove, post_remove, pre_clear and post_clear. In my case I am using them to filter one list ('active things') within the contents of another ('enabled things') independent of the order in which lists are saved:
def clean_services(sender, instance, action, reverse, model, pk_set, **kwargs):
""" Ensures that the active services are a subset of the enabled ones.
"""
if action == 'pre_add' and sender == Account.active_services.through:
# remove from the selection the disabled ones
pk_set.intersection_update(instance.enabled_services.values_list('id', flat=True))
elif action == 'pre_clear' and sender == Account.enabled_services.through:
# clear everything
instance._cache_active_services = list(instance.active_services.values_list('id', flat=True))
instance.active_services.clear()
elif action == 'post_add' and sender == Account.enabled_services.through:
_cache_active_services = getattr(instance, '_cache_active_services', None)
if _cache_active_services:
instance.active_services.add(*list(instance.enabled_services.filter(id__in=_cache_active_services)))
delattr(instance, '_cache_active_services')
elif action == 'pre_remove' and sender == Account.enabled_services.through:
# de-default any service we are disabling
instance.active_services.remove(*list(instance.active_services.filter(id__in=pk_set)))
If the "enabled" ones are updated (cleared/removed + added back, like in admin) then the "active" ones are cached and cleared in the first pass ('pre_clear') and then added back from the cache after the second pass ('post_add').
The trick was to update one list on the m2m_changed signals of the other.

I can't see anything wrong with your code, but I'm confused as to why you think the admin should work any different from any other app.
However, I must say I think your model structure is wrong. I think you need to get rid of all those ForeignKey fields, and just have a ManyToMany - but use a through table to keep track of the roles.
class Project(models.Model):
members = models.ManyToManyField(User, through='ProjectRole')
class ProjectRole(models.Model):
ROLES = (
('SR', 'Sales Rep'),
('SM', 'Sales Manager'),
('PM', 'Project Manager'),
)
project = models.ForeignKey(Project)
user = models.ForeignKey(User)
role = models.CharField(max_length=2, choices=ROLES)

I've stuck on situation, when I needed to find latest item from set of items, that connected to model via m2m_field.
Following Saverio's answer, following code solved my issue:
def update_item(sender, instance, action, **kwargs):
if action == 'post_add':
instance.related_field = instance.m2m_field.all().order_by('-datetime')[0]
instance.save()
m2m_changed.connect(update_item, sender=MyCoolModel.m2m_field.through)

Related

M2M relationship validation in Django

I have these two models:
class Test(models.Model):
problems = models.ManyToManyField('Problem')
...
class Problem(models.Model):
type = models.CharField(max_length=3, choices=SOME_CHOICES)
...
Now, while adding Problems to a Test, I need to limit the number of particular type of problems in the Test. E.g. a Test can contain only 3 Problems of type A, and so on.
The only way to validate this seems to be by using m2m_changed signal on Test.problems.through table. However, to do the validation, I need to access the current Problem being added AND the existing Problems - which doesn't seem to be possible somehow.
What is the correct way to do something like this? M2M validation seems to be a topic untouched in the docs. What am I missing?
You are right on the part that you have to register an m2m_changed signal function like the following:
def my_callback(sender, instance, action, reverse, model, pk_set, **kwargs)
If you read the documentation you 'll see that sender is the object-model that triggers the change and model is the object-model that will change. pk_set will give you the pkeys that will be the new reference for your model. So in your Test model you have to do something like this:
#receiver(m2m_changed)
def my_callback(sender, instance, action, reverse, model, pk_set, **kwargs):
if action == "pre_add":
problem_types = [x.type for x in model.objects.filter(id__in=pk_set)]
if problem_types.count("A") > some_number:
raise SomeException
Mind though that an Exception at that level will not be caught if you're entering fields from Django admin site. To be able to provide user friendly errors for django admin data entry, you'll have to register your own form as admin form. In your case, you need to do the following:
class ProblemTypeValidatorForm(ModelForm):
def clean(self):
super(ProblemTypeValidatorForm, self).clean()
problem_types = [x.type for x in self.cleaned_data.get("problems") if x]
if problem_types.count("A") > some_number:
raise ValidationError("Cannot have more than {0} problems of type {1}"
.format(len(problem_types), "A")
then in your admin.py
#admin.register(Test)
class TestAdmin(admin.ModelAdmin):
form = ProblemTypeValidatorForm
Now keep in mind that these are two different level implementations. None will protect you from someone doing manually this:
one_test_object.problems.add(*Problem.objects.all())
one_test_object.save()
Personal opinion:
So keeping in mind the above, I suggest you go with the ModelForm & ModelAdmin approach and if you're providing an API for CRUD operations, make your validations there as well. Nothing can protect you from someone entering stuff in your db through django shell. If you want such solution types you should go directly to your db and write some kind of magic trigger script. But keep in mind that your db is actually data. Your backend is the one with the business logic. So you shouldn't really try to impose business rules down to the db level. Keep the rules in your backend by validating your data at the spots where create/update happens.
You can't override save for a M2M I'm afraid, but you can achieve what you want.
Use the m2m_changed signal where the action is pre_add.
The 'instance' kwarg will be the Test model the problem is being added to.
The 'pk_id' kwarg will be the primary key of the Problems being added (1 or more).
The validation logic will be something like this:
p_type = Problem.objects.get(id=kwargs['pk_id']).type
type_count = kwargs['instance'].problems.filter(type=p_type).count()
if p_type == 'A' and type_count == 3:
raise Exception("cannot have more than 3 Problems of type A")
[sorry don't have django on hand to verify the query]

How to delete an object in response to a m2m_changed signal in Django?

I have two models Image and Category related via a m2m relation (defined in Category). Images may be under several categories. The API allows to remove an image from a category. In response to that I need to remove the image when it has no categories.
I have the following:
#receiver(m2m_changed, sender=Category.images.through)
def delete_image_if_has_no_categories(sender, instance, action, reverse,
model, pk_set, **kwargs):
# we only watch reverse signal, because in other cases the images are
# being moved or copied, so don't have to be deleted.
if reverse and action == 'post_remove':
if not instance.categories.exists():
instance.delete()
I have placed several debug logs to check the code is being run. And it runs. But the images remain in the DB after the instance.delete().
I have the remove_from_category view inside a transaction.atomic, but it does not help.
Any ideas?
Update
The view call this method in our Image model:
def remove_from_category(self, category_id):
self.categories.remove(category_id)
The view is called via a REST API like this DELETE /category/<catid>/<image-id>.
The images field in the Category model is defined like this:
class Category(MPTTModel):
images = models.ManyToManyField(
'Image',
related_name='categories',
null=True, blank=True,
)
Would the MPTTModel be the culprit? I'm using django-mptt==0.6.0.
I guess u can call ur method to remove when the selected item that u want delete appeas on pre_clear method (django always save the last change in ur m2m field in the pre_clear, so if the attribute is there but didnt on ur post_add this obj is being deleted, so u can force trigger ur function there
check this answer mine
https://stackoverflow.com/a/39331735/6373505

Django form save with post_save signal causing conflict

I have a Physical_therapy_order model and an Event model (an event has foreignkey to Physical_therapy_order). I have a view which allows a user to create a new event. It also has a form with 3 fields from the Physical_therapy_order model.
def PTEventCreateView(request, pt_pk):
#get the pt order and create an a form for that order
pt_order = get_object_or_404(Physical_therapy_order, pk=pt_pk)
ptform = PT_schedule_form(instance=pt_order)
if request.POST:
eventform = PTEventForm(data=request.POST)
ptform = PT_schedule_form(data=request.POST, instance=pt_order)
if eventform.is_valid() and ptform.is_valid():
#I do some checks here that compare data across the two forms.
# if everything looks good i mark keep_saving=True so I can
# continue to save all the data provided in the two forms
if keep_saving:
ptform.save()
eventform.save()
#...send user to succss page
This works just FINE EXCEPT: my PTEvent model has a function attached to its post_save signal. This function pulls the event's related pt_order and makes some modifications to it. Now, if i save the eventform first then the changes from the signal don't happen. if i save the ptform first the ptform changes get discarded and the changes from the signal happen.
THIS IS IMPORTANT: The ptform is editing three entirely different fields than the post_save signal. So its not like they're modifying the same data, only the same model instance. I thought a form only saves the fields in its meta.fields attribute. Why would this be happening? Also, if i save the ptform first, then when eventsform is saved shouldn't the signal use the updated physical_therapy_order? I'm not sure if I'm even on the right track?
I think this is because of cached objects.
What I would suggest is
Save eventform first
Get new instance of pt_order either querying db or through saved instance of eventform
And then re-create form and save.
Sample code change:
# your code
if keep_saving:
evt = eventform.save()
# I'm not sure exact name of your field name for pt_order in Event model, change appropriately
newptform = PT_schedule_form(data=request.POST, instance= evt.pt_order)
newpt = newptform.save()

Django - How to compare pre / post save state of a model including all relations

I am trying to maintain a few sets in redis that track models in django
class Campaign(models.Model):
advertiser = models.ForeignKey(User)
name = models.CharField(max_length=200)
carriers = models.ManyToManyField(Carrier)
countries = models.ManyToManyField(Country)
#receiver(pre_save, sender=Campaign)
def adserver_clear_cache(sender, **kwargs):
campaign = kwargs['instance']
for con in campaign.countries.all():
r.srem("con:" + str(con.id), campaign.id)
for car in campaign.carriers.all():
r.srem("car:" + str(car.id), campaign.id)
#receiver(post_save, sender=Campaign)
def adserver_save_cache(sender, **kwargs):
campaign = kwargs['instance']
for con in campaign.countries.all():
r.sadd("con:" + str(con.id), campaign.id)
for car in campaign.carriers.all():
r.sadd("car:" + str(car.id), campaign.id)
The issue is, the campaign is fully saved, before each of the carriers, campaigns and such are saved. So I am getting the same data in pre + post_save. Is there a way to call a function when everything, including relations are finished saving?
thanks!
The problem is that the m2m relationship is saved after your model is saved, in other words "post_save() is still too early" for what you're after.
You can either use the m2m_changed signal as jpic points out, or use Django 1.4's new "save_related" - https://docs.djangoproject.com/en/dev/ref/contrib/admin/#django.contrib.admin.ModelAdmin.save_related which closed ticket 16115 https://code.djangoproject.com/ticket/16115
See also https://stackoverflow.com/a/8462541/640759
ManyToManyField is special. It does not represent in the database by a column of its model's table. In fact, a ManyToManyField represents as another table that links the models at both sides of the n:m relation.
It does not make sense to emit a model signal such as post_save for an arbitrary side of the n:m relation. When a ManyToManyField changes, m2m_changed signal is emited:
Sent when a ManyToManyField is changed on a model instance. Strictly speaking, this is not a model signal since it is sent by the ManyToManyField, but since it complements the pre_save/post_save and pre_delete/post_delete when it comes to tracking changes to models, it is included here.

In Django, how do I get a signal for when a Group has a User added or removed?

In the Django admin I sometimes add or delete users to or from (existing) groups. When this happens I'd like to be able to run a function.
I'm just using the standard User and Group models.
I have looked at doing it with signals, through m2m_changed, but it seems to need a Through class - and I don't think there is one in this case.
From the django doc:
sender - The intermediate model class describing the ManyToManyField. This class is automatically created when a many-to-many field is defined; you can access it using the through attribute on the many-to-many field.
When subscribing to m2m_changed like so:
#receiver(m2m_changed)
def my_receiver(**kwargs):
from pprint import pprint
pprint(kwargs)
You will receive a bunch of signals like this (shortened):
{'sender': <class 'django.contrib.auth.models.User_groups'>,
'action': 'post_add',
'instance': <User: bouke>,
'model': <class 'django.contrib.auth.models.Group'>,
'pk_set': set([1]),
'reverse': False,
'signal': <django.dispatch.dispatcher.Signal object at 0x101840210>,
'using': 'default'}
So the user bouke has been added to pk_set groups: [1]. However I noted that the admin layout clears all groups and then adds the selected groups back in. The signals you will receive are pre_clear, post_clear, pre_add, post_add. Using a combination of these signals you could store the pre and post groups. Doing a diff over these lists, you have the deleted and added groups for the user.
Note that the signals are the other way around (pk_set and instance) when editing a group instead of a user.
You'll see in the Django documentation (v1.11) that your desired sender should be the intermediate through field belonging to the ManyToMany field, wherever that's defined. If you register that as your sender, then you'll be listening to eg Users adding Groups to themselves, as well as Groups adding Users to themselves.
self.walrus.groups.remove(self.peon_group)
#receiver(signal=m2m_changed, sender=User.groups.through)
def adjust_group_notifications(instance, action, reverse, model, pk_set, using, *args, **kwargs):
if model == Group and not reverse:
logger.info("User %s deleted their relation to groups «%s»", instance.username, pk_set)
if action == 'post_remove':
# The *modification* happening is a deletion of the link
…
elif action == 'post_add':
logger.info("User %s created a relation to groups «%s»", instance.username, ", ".join(pk_set))
…
else:
logger.info("Group %s is modifying its relation to users «%s»", instance, pk_set)
…
return
You need to create a signal using m2m_changed as a receiver. According to the official Django documentation:
A signal is sent when a ManyToManyField is changed on a model instance. Strictly speaking, this is not a model signal since it is sent by the ManyToManyField.
So, the simplest implementation is as follows:
#receiver(m2m_changed)
def signal_handler(**kwargs):
from pprint import pprint
pprint(kwargs)
In your case, you want to perform something when a user is added or removed from a group, so you can take advantage of the action parameter when it takes the values 'pre_add', 'post_add', 'pre_remove', and 'post_remove'. You can also take advantage of pk_set parameter which contains primary key values that have been added to or removed from the relation.
#receiver(m2m_changed)
def signal_handler_when_user_is_added_or_removed_from_group(action, instance, pk_set, model, **kwargs):
if model == Group:
if action == 'pre_add':
# TODO: add logic
pass
elif action == 'post_add':
# TODO: add logic
pass
# handle as many actions as one needs
# The following for loop prints every group that were
# added/removed.
for pk in pk_set:
group = Group.objects.get(id=pk)
print(group)
It might be better to try and achieve this with django-celery, that way you can write custom tasks, and based on a certain criteria (such as removal or addition) you can fire of a certain task.
Celery
RabbitMQ