I have these two models:
class Test(models.Model):
problems = models.ManyToManyField('Problem')
...
class Problem(models.Model):
type = models.CharField(max_length=3, choices=SOME_CHOICES)
...
Now, while adding Problems to a Test, I need to limit the number of particular type of problems in the Test. E.g. a Test can contain only 3 Problems of type A, and so on.
The only way to validate this seems to be by using m2m_changed signal on Test.problems.through table. However, to do the validation, I need to access the current Problem being added AND the existing Problems - which doesn't seem to be possible somehow.
What is the correct way to do something like this? M2M validation seems to be a topic untouched in the docs. What am I missing?
You are right on the part that you have to register an m2m_changed signal function like the following:
def my_callback(sender, instance, action, reverse, model, pk_set, **kwargs)
If you read the documentation you 'll see that sender is the object-model that triggers the change and model is the object-model that will change. pk_set will give you the pkeys that will be the new reference for your model. So in your Test model you have to do something like this:
#receiver(m2m_changed)
def my_callback(sender, instance, action, reverse, model, pk_set, **kwargs):
if action == "pre_add":
problem_types = [x.type for x in model.objects.filter(id__in=pk_set)]
if problem_types.count("A") > some_number:
raise SomeException
Mind though that an Exception at that level will not be caught if you're entering fields from Django admin site. To be able to provide user friendly errors for django admin data entry, you'll have to register your own form as admin form. In your case, you need to do the following:
class ProblemTypeValidatorForm(ModelForm):
def clean(self):
super(ProblemTypeValidatorForm, self).clean()
problem_types = [x.type for x in self.cleaned_data.get("problems") if x]
if problem_types.count("A") > some_number:
raise ValidationError("Cannot have more than {0} problems of type {1}"
.format(len(problem_types), "A")
then in your admin.py
#admin.register(Test)
class TestAdmin(admin.ModelAdmin):
form = ProblemTypeValidatorForm
Now keep in mind that these are two different level implementations. None will protect you from someone doing manually this:
one_test_object.problems.add(*Problem.objects.all())
one_test_object.save()
Personal opinion:
So keeping in mind the above, I suggest you go with the ModelForm & ModelAdmin approach and if you're providing an API for CRUD operations, make your validations there as well. Nothing can protect you from someone entering stuff in your db through django shell. If you want such solution types you should go directly to your db and write some kind of magic trigger script. But keep in mind that your db is actually data. Your backend is the one with the business logic. So you shouldn't really try to impose business rules down to the db level. Keep the rules in your backend by validating your data at the spots where create/update happens.
You can't override save for a M2M I'm afraid, but you can achieve what you want.
Use the m2m_changed signal where the action is pre_add.
The 'instance' kwarg will be the Test model the problem is being added to.
The 'pk_id' kwarg will be the primary key of the Problems being added (1 or more).
The validation logic will be something like this:
p_type = Problem.objects.get(id=kwargs['pk_id']).type
type_count = kwargs['instance'].problems.filter(type=p_type).count()
if p_type == 'A' and type_count == 3:
raise Exception("cannot have more than 3 Problems of type A")
[sorry don't have django on hand to verify the query]
Related
I am struggling to understand how permissioning in DRF is meant to work. Particularly when/why should a permission be used versus when the queryset should be filtered and the difference between has_object_permission() & has_permission() and finally, where does the serializer come in.
For example, with models:
class Patient(models.Model):
user = models.OneToOneField(User, related_name='patient')
class Appointment(models.Model):
patient = models.ForeignKey(Patient, related_name='appointment')
To ensure that patients can only see/change their own appointments, you might check in a permission:
class IsRelevantPatient(BasePermission):
def has_object_permission(self, request, view, obj):
if self.request.user.patient == obj.appointment.patient:
return True
else:
return False
But, modifying the queryset also makes sense:
class AppointmentViewSet(ModelViewSet):
...
def get_queryset(self):
if self.request.user.is_authenticated:
return Appointment.objects.filter(patient=self.request.user.patient)
What's confusing me is, why have both? Filtering the queryset does the job - a GET (retrieve and list) only returns that patient's appointments and, a POST or PATCH (create or update) only works for that patient's appointments.
Aside from this seemingly redundant permission - what is the difference between has_object_permission() & has_permission(), from my research, it sounds like has_permission() is for get:list and post:create whereas has_object_permission() is for get:retrieve and patch:update. But, I feel like that is probably an oversimplification.
Lastly - where does validation in the serializer come in? For example, rather than a permission to check if the user is allowed to patch:update an object, You can effectively check permissions by overriding the update() method of the serializer and checking there.
Apologies for the rambling post but I have read the docs and a few other question threads and am at the point where I am probably just confusing myself more. Would really appreciate a clear explanation.
Thanks very much.
First, difference between has_object_permission() and has_permission() :
has_permission() tells if the user has the permission to use the view or the viewset without dealing with any object in the database
has_object_permission() tells if the user has the permission to use the view or the viewset based on a specific object in the database.
The important note thaw is that DRF wont perform the test itself in the case of object level permission, but you have to do it explicitly by calling check_object_permission() somewhere in your view (doc here).
The second important note is that DRF will not filter the result of the query based on object permission. If you want the query to be filtered, then you have to do it yourself (by overriding get_queryset() like you did or using a filter backend), that's the difference.
The serializer has nothing to do with permission neither with filtering. It handles objects one by one, applying validation (not permission) on each field of each objects.
I have a simple Model that stores the user that created it with a ForeignKey. The model has a corresponding ModelSerializer and ModelViewSet.
The problem is that when the user submits a POST to create a new record, the user should be set by the backend. I tried overriding perform_create on the ModelViewSet to set the user, but it actually still fails during the validation step (which makes sense). It comes back saying the user field is required.
I'm thinking about overriding the user field on the ModelSerializer to be optional, but I feel like there's probably a cleaner and more efficient way to do this. Any ideas?
I came across this answer while looking for a way to update values before the control goes to the validator.
This might be useful for someone else - here's how I finally did it (DRF 3) without rewriting the whole validator.
class MyModelSerializer(serializers.ModelSerializer):
def to_internal_value(self, data):
data['user'] = '<Set Value Here>'
return super(MyModelSerializer, self).to_internal_value(data)
For those who're curious, I used this to round decimal values to precision defined in the model so that the validator doesn't throw errors.
You can make the user field as read_only.
This will ensure that the field is used when serializing a representation, but is not used when creating or updating an instance during deserialization.
In your serializers, you can do something like:
class MyModelSerializer(serializers.ModelSerializer):
class Meta:
model = MyModel
extra_kwargs = {
'user' : {'read_only' : True} # define the 'user' field as 'read-only'
}
You can then override the perform_create() and set the user as per your requirements.
Old topic but it could be useful for someone.
If you want to alter your data before validation of serializer:
serializer.initial_data["your_key"] = "your data"
serializer.is_valid(raise_exception=True)
I am very new to django and python in general, and I was trying to learn rest_framework to create RESTful APIs.
So i have a model like this:
class Listing(models.Model):
listingid = models.BigIntegerField(primary_key=True)
sellerid = models.IntegerField()
createdon = models.DateTimeField(auto_now_add=True, editable=False)
expirydate = models.DateTimeField(null=True)
validationstatus = models.SmallIntegerField(default=0)
listingstatus = models.SmallIntegerField(
choices=((0, 'Active'),
(1, 'Hidden'),
(2, 'Suspended'),
(4, 'Expired'),
(5, 'Deleted'),
),
default=0)
Now i need to validate that the expirydate is always greater than the createdon date.
I know i can do this in the views, I guess that would not be a good idea, since now the validation only exists in the views.
So that leaves me with the serializers and the model.
I know I can override the save method to do check this like so:
class MasterListing(models.Model):
# fields here..
def save(self, *args, **kwargs):
if self.expirydate > self.createdon:
super().save(*args, **kwargs)
return ValidationError("Expiry date cannot be greater than created date ("++")")
but I dont know if this would be a good idea, since now I am raising an error which the programmer may forget to catch. I am also not sure if the fields would be populated when this method would run.
Another way I read about in the docs is the clean method which i couldn't really understand so well.
Can anyone guide me on how to handle situations like this when you are working with the rest_framework?
Some of the things I have read about validation till now:
Serializer Validation
Field level validation
Validators
Model Validation
override clean method
override save method
Just do it manually in the views
There seem to be so many options, and I might have even left a few, I could not clearly get an idea of when to use where.
I am sorry if this is a little on the beginner level, but i am new to frameworks and django seems to be very different from what i was doing in PHP. Any advice is welcome!
Edit: I will be using django for the rest_framework only and nothing else, since we only want to build RESTful APIs.
Django REST framework used to call Model.clean, which was previously the recommended place for putting validation logic that needed to be used in Django forms and DRF serializers. As of DRF 3.0, this is no longer the case and Model.clean will no longer be called during the validation cycle. With that change, there are now two possible places to put in custom validation logic that works on multiple fields.
If you are only using Django REST framework for validation, and you don't have any other areas where data needs to be manually validated (like a ModelForm, or in the Django admin), then you should look into Django REST framework's validation framework.
class MySerializer(serializers.ModelSerializer):
# ...
def validate(self, data):
# The keys can be missing in partial updates
if "expirydate" in data and "createdon" in data:
if data["expirydate"] < data["createdon"]:
raise serializers.ValidationError({
"expirydata": "Expiry date cannot be greater than created date",
})
return super(MySerializer, self).validate(data)
If you need to use Django REST framework in combination with a Django component that uses model-level validation (like the Django admin), you have two options.
Duplicate your logic in both Model.clean and Serializer.validate, violating the DRY principle and opening yourself up to future issues.
Do your validation in Model.save and hope that nothing strange happens later.
but I dont know if this would be a good idea, since now I am raising an error which the programmer may forget to catch.
I would venture to say that it would be better for the error to be raised than for the saved data to possibly become invalid on purpose. Once you start allowing invalid data, you have to put in checks anywhere the data is used to fix it. If you don't allow it to go into an invalid state, you don't run into that issue.
I am also not sure if the fields would be populated when this method would run.
You should be able to assume that if an object is going to be saved, the fields have already been populated with their values.
If you would like to both Model Validation and Serializer validation using Django REST Framework 3.0, you can force your serializer to use the Model validation like this (so you don't repeat yourself):
import rest_framework, django
from rest_framework import serializers
class MySerializer(serializers.ModelSerializer):
def validate(self, data):
for key, val in data.iteritems():
setattr(self.instance, key, val)
try:
self.instance.clean()
except django.core.exceptions.ValidationError as e:
raise rest_framework.exceptions.ValidationError(e.message_dict)
return data
I thought about generating a new function from my model's clean() function's code, and have it either spit out django.core.exceptions.ValidationError or rest_framework.exceptions.ValidationError, based on a parameter source (or something) to the function. Then I would call it from the model, and from the serializer. But that hardly seemed better to me.
If you want to make sure that your data is valid on the lowest level, use Model Validation (it should be run by the serializer class as well as by (model)form classes (eg. admin)).
If you want the validation to happen only in your API/forms put it in a serializer/form class. So the best place to put your validation should be Model.clean().
Validation should never actually happen in views, as they shouldn't get too bloated and the real business logic should be encapsulated in either models or forms.
(Django 1.1) I have a Project model that keeps track of its members using a m2m field. It looks like this:
class Project(models.Model):
members = models.ManyToManyField(User)
sales_rep = models.ForeignKey(User)
sales_mgr = models.ForeignKey(User)
project_mgr = models.ForeignKey(User)
... (more FK user fields) ...
When the project is created, the selected sales_rep, sales_mgr, project_mgr, etc Users are added to members to make it easier to keep track of project permissions. This approach has worked very well so far.
The issue I am dealing with now is how to update the project's membership when one of the User FK fields is updated via the admin. I've tried various solutions to this problem, but the cleanest approach seemed to be a post_save signal like the following:
def update_members(instance, created, **kwargs):
"""
Signal to update project members
"""
if not created: #Created projects are handled differently
instance.members.clear()
members_list = []
if instance.sales_rep:
members_list.append(instance.sales_rep)
if instance.sales_mgr:
members_list.append(instance.sales_mgr)
if instance.project_mgr:
members_list.append(instance.project_mgr)
for m in members_list:
instance.members.add(m)
signals.post_save.connect(update_members, sender=Project)
However, the Project still has the same members even if I change one of the fields via the admin! I have had success updating members m2m fields using my own views in other projects, but I never had to make it play nice with the admin as well.
Is there another approach I should take other than a post_save signal to update membership? Thanks in advance for your help!
UPDATE:
Just to clarify, the post_save signal works correctly when I save my own form in the front end (old members are removed, and new ones added). However, the post_save signal does NOT work correctly when I save the project via the admin (members stay the same).
I think Peter Rowell's diagnosis is correct in this situation. If I remove the "members" field from the admin form the post_save signal works correctly. When the field is included, it saves the old members based on the values present in the form at the time of the save. No matter what changes I make to the members m2m field when project is saved (whether it be a signal or custom save method), it will always be overwritten by the members that were present in the form prior to the save. Thanks for pointing that out!
Having had the same problem, my solution is to use the m2m_changed signal. You can use it in two places, as in the following example.
The admin upon saving will proceed to:
save the model fields
emit the post_save signal
for each m2m:
emit pre_clear
clear the relation
emit post_clear
emit pre_add
populate again
emit post_add
Here you have a simple example that changes the content of the saved data before actually saving it.
class MyModel(models.Model):
m2mfield = ManyToManyField(OtherModel)
#staticmethod
def met(sender, instance, action, reverse, model, pk_set, **kwargs):
if action == 'pre_add':
# here you can modify things, for instance
pk_set.intersection_update([1,2,3])
# only save relations to objects 1, 2 and 3, ignoring the others
elif action == 'post_add':
print pk_set
# should contain at most 1, 2 and 3
m2m_changed.connect(receiver=MyModel.met, sender=MyModel.m2mfield.through)
You can also listen to pre_remove, post_remove, pre_clear and post_clear. In my case I am using them to filter one list ('active things') within the contents of another ('enabled things') independent of the order in which lists are saved:
def clean_services(sender, instance, action, reverse, model, pk_set, **kwargs):
""" Ensures that the active services are a subset of the enabled ones.
"""
if action == 'pre_add' and sender == Account.active_services.through:
# remove from the selection the disabled ones
pk_set.intersection_update(instance.enabled_services.values_list('id', flat=True))
elif action == 'pre_clear' and sender == Account.enabled_services.through:
# clear everything
instance._cache_active_services = list(instance.active_services.values_list('id', flat=True))
instance.active_services.clear()
elif action == 'post_add' and sender == Account.enabled_services.through:
_cache_active_services = getattr(instance, '_cache_active_services', None)
if _cache_active_services:
instance.active_services.add(*list(instance.enabled_services.filter(id__in=_cache_active_services)))
delattr(instance, '_cache_active_services')
elif action == 'pre_remove' and sender == Account.enabled_services.through:
# de-default any service we are disabling
instance.active_services.remove(*list(instance.active_services.filter(id__in=pk_set)))
If the "enabled" ones are updated (cleared/removed + added back, like in admin) then the "active" ones are cached and cleared in the first pass ('pre_clear') and then added back from the cache after the second pass ('post_add').
The trick was to update one list on the m2m_changed signals of the other.
I can't see anything wrong with your code, but I'm confused as to why you think the admin should work any different from any other app.
However, I must say I think your model structure is wrong. I think you need to get rid of all those ForeignKey fields, and just have a ManyToMany - but use a through table to keep track of the roles.
class Project(models.Model):
members = models.ManyToManyField(User, through='ProjectRole')
class ProjectRole(models.Model):
ROLES = (
('SR', 'Sales Rep'),
('SM', 'Sales Manager'),
('PM', 'Project Manager'),
)
project = models.ForeignKey(Project)
user = models.ForeignKey(User)
role = models.CharField(max_length=2, choices=ROLES)
I've stuck on situation, when I needed to find latest item from set of items, that connected to model via m2m_field.
Following Saverio's answer, following code solved my issue:
def update_item(sender, instance, action, **kwargs):
if action == 'post_add':
instance.related_field = instance.m2m_field.all().order_by('-datetime')[0]
instance.save()
m2m_changed.connect(update_item, sender=MyCoolModel.m2m_field.through)
I have the following models (simplified example):
class Book(models.Model):
users = models.ManyToManyField(User, through=Permission)
class Permission(models.Model):
user = models.ForeignKey(User)
role = models.ForeignKey(Group)
active = models.BooleanField()
book = models.ForeignKey(Book)
What I need is that for a Book instance there cannot be more than one User of with the same Role and Active.
So this is allowed:
Alice, Admin, False (not active), BookA
Dick, Admin, True (active), BookA
Chris, Editor, False (not active), BookA
Matt, Editor, False (not active), BookA
But this is not allowed:
Alice, Admin, True (active), BookA
Dick, Admin, True (active), BookA
Now this cannot be done with unique_together, because it only counts when active is True. I've tried to write a custom clean method (like how I have done here). But it seems that when you save a Book and it runs the validation on each Permission, the already validated Permission instances aren't saved until they've all been validated. This makes sense, because you don't want them to be saved in case something doesn't validate.
Could anyone tell me if there is a way to perform the validation described above?
P.S. I could imagine using the savepoint feature (http://docs.djangoproject.com/en/1.2/topics/db/transactions/), but I only really want to consider that as a last resort.
Maybe you can do something like: unique_together = [[book, role, active=1],] ?
Edit Sep. 23, 2010 14:00 Response to Manoj Govindan:
My admin.py (simplified version for clarity):
class BookAdmin(admin.ModelAdmin):
inlines = (PermissionInline,)
class PermissionInline(admin.TabularInline):
model = Permission
In the shell your validation would work. Because you first have to create the book instance and then you create all the Permission instances one by one: http://docs.djangoproject.com/en/1.2/topics/db/models/#extra-fields-on-many-to-many-relationships. So in the shell if you add 2 Permission instances the 1st Permission instance has been saved by the time 2nd is being validated, and so the validation works.
However when you use the Admin interface and you add all the book.users instances at the same time via the book users inline, I believe it does all the validation on all the book.users instances first, before it saves them.
When I tried it, the validation didn't work, it just succeeded without an error when there should have been a ValidationError.
You can use signals to prevent the saving of data that is invalid: I'm still working on a nice solution on how to get the validation to bubble up in a nice way in the admin.
#receiver(models.signals.m2m_changed, sender=Book.users.through)
def prevent_duplicate_active_user(sender, instance, action, reverse, model, pk_set, **kwargs):
if action != "pre_add":
return
if reverse:
# Editing the Permission, not the Book.
pass
else:
# At this point, look for already saved Users with the book/active.
if instance.permissions.filter(active=True).exists():
raise forms.ValidationError(...)
Note that this is not a complete solution, but is a pointer how I am doing something similar.
One way to do this is to use the newfangled model validation. Specifically, you can add a custom validate_unique method to Permission models to achieve this effect. For e.g.
from django.core.exceptions import ValidationError, NON_FIELD_ERRORS
class Permission(models.Model):
...
def validate_unique(self, exclude = None):
options = dict(book = self.book, role = self.role, active = True)
if Permission.objects.filter(**options).count() != 0:
template = """There cannot be more than one User of with the
same Role and Active (book: {0})"""
message = template.format(self.book)
raise ValidationError({NON_FIELD_ERRORS: [message]})
I did some rudimentary testing using one of my projects' Admin app and it seemed to work.
Now this cannot be done with unique_together, because it only counts when active is True.
Simplest way, imo, is to change type of active from BooleanField to CharField. Store 'Y' and 'N' in active.
That way you can use built-in unique_together = [[book, role, active],]