Correct Way to Validate Django Model Objects? - django

I'm still trying to understand the correct way to validate a Django model object using a custom validator at the model level. I know that validation is usually done within a form or model form. However, I want to ensure the integrity of my data at the model level if I'm interacting with it via the ORM in the Python shell. Here's my current approach:
from django.db import models
from django.core import validators
from django.core exceptions import ValidationError
def validate_gender(value):
""" Custom validator """
if not value in ('m', 'f', 'M', 'F'):
raise ValidationError(u'%s is not a valid value for gender.' % value)
class Person(models.Model):
name = models.CharField(max_length=128)
age = models.IntegerField()
gender = models.CharField(maxlength=1, validators=[validate_gender])
def save(self, *args, **kwargs):
""" Override Person's save """
self.full_clean(exclude=None)
super(Person, self).save(*args, **kwargs)
Here are my questions:
Should I create a custom validation function, designate it as a validator, and then override the Person's save() function as I've done above? (By the way, I know I could validate my gender choices using the 'choices' field option but I created 'validate_gender' for the purpose of illustration).
If I really want to ensure the integrity of my data, should I not only write Django unit tests for testing at the model layer but also equivalent database-level unit tests using Python/Psycopg? I've noticed that Django unit tests, which raise ValidationErrors, only test the model's understanding of the database schema using a copy of the database. Even if I were to use South for migrations, any database-level constraints are limited to what Django can understand and translate into a Postgres constraint. If I need a custom constraint that Django can't replicate, I could potentially enter data into my database that violates that constraint if I'm interacting with the database directly via the psql terminal.
Thanks!

I had a similar misunderstanding of the ORM when I first started with Django.
No, don't put self.full_clean() inside of save. Either
A) use a ModelForm (which will cause all the same validation to occur - note: ModelForm.is_valid() won't call Model.full_clean explicitly, but will perform the exact same checks as Model.full_clean). Example:
class PersonForm(forms.ModelForm):
class Meta:
model = Person
def add_person(request):
if request.method == 'POST':
form = PersonForm(request.POST, request.FILES)
if form.is_valid(): # Performs your validation, including ``validate_gender``
person = form.save()
return redirect('some-other-view')
else:
form = PersonForm()
# ... return response with ``form`` in the context for rendering in a template
Also note, forms aren't for use only in views that render them in templates - they're great for any sort of use, including an API, etc. After running form.is_valid() and getting errors, you'll have form.errors which is a dictionary containing all the errors in the form, including a key called '__all__' which will contain non-field errors.
B) Simply use model_instance.full_clean() in your view (or other logical application layer), instead of using a form, but forms are a nice abstraction for this.
I don't really have a solution to, but I've never run into such a problem, even in large projects (the current project I work with my company on has 146 tables) and I don't suspect it'll be a concern in your case either.

Related

How to call my field validators in my save method in Django?

I have a model with a field that requires alphanumeric characters. I force this constraint using a validator. Something like this:
from django.core.validators import RegexValidator
validate_alphanumeric = RegexValidator(r'^[a-zA-Z0-9]*$', 'Only alphanumeric characters are allowed.')
class MyModel(models.Model):
my_field = models.CharField(max_length=255, validators=[validate_alphanumeric,]
def save(self, *args, **kwargs):
# self.call_validators() or whatever
super(MyModel, self).save(*args, **kwargs)
Now, this works automatically in my admin site, and model forms. However, when I create objects from the shell, or lets say a manual API endpoint, then the validator is not enforced.
Is there a built-in function in django like the one in my comments that I can just call in my save method? Or do I have to manually validate my field again in the save method? Thanks.
Here's the validating objects documentation. Basically, if you call an object's full_clean() method, you'll run all validations on the object. You can run only the individual field validators by calling self.clean_fields().
But in general, it's not good practice to add validation in the save() method. The reason is that in most Django apps, you'd create a form (a ModelForm) which would call the validation methods and be able to return something meaningful to the user when validation fails.
When the model's save() method is called it's too late to show something to the user, so you can only raise an exception at that point (and crash).
The normal procedure (which the admin forms use) is: validate the form by calling form.is_valid() (which calls full_clean() on the model), then if and only if the form is valid, save the model.
The shell is not the regular interaction method and should be used only very carefully as it bypasses the normal flow of the application.

Model datetime field validation for fields with auto_now

I am very new to django and python in general, and I was trying to learn rest_framework to create RESTful APIs.
So i have a model like this:
class Listing(models.Model):
listingid = models.BigIntegerField(primary_key=True)
sellerid = models.IntegerField()
createdon = models.DateTimeField(auto_now_add=True, editable=False)
expirydate = models.DateTimeField(null=True)
validationstatus = models.SmallIntegerField(default=0)
listingstatus = models.SmallIntegerField(
choices=((0, 'Active'),
(1, 'Hidden'),
(2, 'Suspended'),
(4, 'Expired'),
(5, 'Deleted'),
),
default=0)
Now i need to validate that the expirydate is always greater than the createdon date.
I know i can do this in the views, I guess that would not be a good idea, since now the validation only exists in the views.
So that leaves me with the serializers and the model.
I know I can override the save method to do check this like so:
class MasterListing(models.Model):
# fields here..
def save(self, *args, **kwargs):
if self.expirydate > self.createdon:
super().save(*args, **kwargs)
return ValidationError("Expiry date cannot be greater than created date ("++")")
but I dont know if this would be a good idea, since now I am raising an error which the programmer may forget to catch. I am also not sure if the fields would be populated when this method would run.
Another way I read about in the docs is the clean method which i couldn't really understand so well.
Can anyone guide me on how to handle situations like this when you are working with the rest_framework?
Some of the things I have read about validation till now:
Serializer Validation
Field level validation
Validators
Model Validation
override clean method
override save method
Just do it manually in the views
There seem to be so many options, and I might have even left a few, I could not clearly get an idea of when to use where.
I am sorry if this is a little on the beginner level, but i am new to frameworks and django seems to be very different from what i was doing in PHP. Any advice is welcome!
Edit: I will be using django for the rest_framework only and nothing else, since we only want to build RESTful APIs.
Django REST framework used to call Model.clean, which was previously the recommended place for putting validation logic that needed to be used in Django forms and DRF serializers. As of DRF 3.0, this is no longer the case and Model.clean will no longer be called during the validation cycle. With that change, there are now two possible places to put in custom validation logic that works on multiple fields.
If you are only using Django REST framework for validation, and you don't have any other areas where data needs to be manually validated (like a ModelForm, or in the Django admin), then you should look into Django REST framework's validation framework.
class MySerializer(serializers.ModelSerializer):
# ...
def validate(self, data):
# The keys can be missing in partial updates
if "expirydate" in data and "createdon" in data:
if data["expirydate"] < data["createdon"]:
raise serializers.ValidationError({
"expirydata": "Expiry date cannot be greater than created date",
})
return super(MySerializer, self).validate(data)
If you need to use Django REST framework in combination with a Django component that uses model-level validation (like the Django admin), you have two options.
Duplicate your logic in both Model.clean and Serializer.validate, violating the DRY principle and opening yourself up to future issues.
Do your validation in Model.save and hope that nothing strange happens later.
but I dont know if this would be a good idea, since now I am raising an error which the programmer may forget to catch.
I would venture to say that it would be better for the error to be raised than for the saved data to possibly become invalid on purpose. Once you start allowing invalid data, you have to put in checks anywhere the data is used to fix it. If you don't allow it to go into an invalid state, you don't run into that issue.
I am also not sure if the fields would be populated when this method would run.
You should be able to assume that if an object is going to be saved, the fields have already been populated with their values.
If you would like to both Model Validation and Serializer validation using Django REST Framework 3.0, you can force your serializer to use the Model validation like this (so you don't repeat yourself):
import rest_framework, django
from rest_framework import serializers
class MySerializer(serializers.ModelSerializer):
def validate(self, data):
for key, val in data.iteritems():
setattr(self.instance, key, val)
try:
self.instance.clean()
except django.core.exceptions.ValidationError as e:
raise rest_framework.exceptions.ValidationError(e.message_dict)
return data
I thought about generating a new function from my model's clean() function's code, and have it either spit out django.core.exceptions.ValidationError or rest_framework.exceptions.ValidationError, based on a parameter source (or something) to the function. Then I would call it from the model, and from the serializer. But that hardly seemed better to me.
If you want to make sure that your data is valid on the lowest level, use Model Validation (it should be run by the serializer class as well as by (model)form classes (eg. admin)).
If you want the validation to happen only in your API/forms put it in a serializer/form class. So the best place to put your validation should be Model.clean().
Validation should never actually happen in views, as they shouldn't get too bloated and the real business logic should be encapsulated in either models or forms.

Proper Model Field For One To One Database Relationships In Django

My issue is that my app is not allowing me to update a OneToOneField field. Here's my explanation of what I'm trying to do.
I am building an inventory app that keeps track of instruments that have been loaned to students. There will always be a one-to-one database relationship between students and instruments. So an individual student can't ever have more than one instrument and vice versa.
I therefore created an Intrument model that looks like this:
class Instrument(models.Model):
instrument_type = models.CharField(max_length=100)
needs_repairs = models.BooleanField()
inventory_id = models.CharField(max_length=100)
student = models.OneToOneField(Student, null=True, blank=True, default = None)
I have created a form that allows me to update existing students, and I'm trying to use as much built-in stuff as possible so that I don't need to re-write validation code or HTML. So I'm using a ModelForm object and validating my input using the is_valid() method.
Here's an example of a POST request to update an instrument:
csrfmiddlewaretoken=xyUBhVuQZus6XmeV2DhCmpJHwIXVmdHm&instrument_type=Viola&inventory_id=abcde&student=3
Please note that the only field with a uniqueness constraint is student.
So finally, here's the problem: when I call the is_valid() method it always fails with an error saying that the student has already been assigned to an instrument.
My first thought was to use the framework to add some pre-validation code that didn't error if the student pkey didn't change. This certainly seems easy enough, but it seems to be a bit hacky to me. I assumed that one-to-one relationships would "just work" like all of the other Model fields and that no special validation would be required.
But then I read the API docs for the OneToOneField class and it doesn't seem to address one-to-one database relationships - it seems to address one-to-one OO relationships. So I may be using the wrong Model field type all together. And since this is such a simple app, I'm not performing a ton of OO modeling - I'm just worried about proper data modeling :-)
So am I using the wrong field, or is the "proper" way to fix this to add pre-validation code to my Student model?
Updates From Comments
Here's the closest thing that I have to a stack trace:
>>> data = {'instrument_type': 'Viola', 'inventory_id': 'abcde', 'student': 3, 'repairer': 1}
>>> form = InstrumentForm(data)
>>> form.is_bound
True
>>> form.is_valid()
False
>>> form.errors
{'student': [u'Instrument with this Student already exists.']}
I use a single view method to display Instrument detail and update a single Instrument. Here's that:
def instrument_detail(request, instrument_id):
try:
instrument = Instrument.objects.get(pk=instrument_id)
except Instrument.DoesNotExist:
raise Http404
# Default if not a POST
form = InstrumentForm(instance=instrument)
if request.method == 'POST':
form = InstrumentForm(request.POST)
if form.is_valid():
form.save()
return HttpResponseRedirect(reverse('instruments.views.instruments_index'))
# otherwise...
t = loader.get_template('instruments/details.html')
c = RequestContext(request, {
'instrument': instrument,
'form': form,
})
return HttpResponse(t.render(c))
You're not passing the instance when instantiating the form on POST.
if request.method == 'POST':
form = InstrumentForm(request.POST, instance=instrument)

django: use a queryset as modelform initial data

I'm making a settings interface which works by scanning for a settings folder in the installed applications, scanning for settings files, and finally scanning for ModelForms.
I'm at the last step now. The forms are properly found and loaded, but I now need to provide the initial data. The initial data is to be pulled from the database, and, as you can imagine, it must be limited to the authenticated user (via request.user.id).
Keep in mind, this is all done dynamically. None of the names for anything, nor their structure is known in advanced (I really don't want to maintain a boring settings interface).
Here is an example settings form. I just pick the model and which fields the user can edit (this is the extent to which I want to maintain a settings interface).
class Set_Personal_Info(forms.ModelForm):
class Meta:
model = UserProfile
fields = ('nick_name', 'url')
I've looked at modelformset_factory which almost does what I want to do, but it only seems to work with results of two or more. (Here, obj is one of the settings forms)
Formset = modelformset_factory(obj.Meta.model, form=obj)
Formset(queryset=obj.Meta.model.objects.filter(id=request.user.id))
I can't filter the data, I have to get one, and only one result. Unfortunately I can't use get()
Formset = modelformset_factory(obj.Meta.model, form=obj)
Formset(queryset=obj.Meta.model.objects.get(id=request.user.id))
'User' object has no attribute 'ordered'
Providing the query result as initial data also doesn't work as it's not a list.
Formset = modelformset_factory(obj.Meta.model, form=obj)
Formset(initial=obj.Meta.model.objects.get(id=request.user.id))
'User' object does not support indexing
I have a feeling that the answer is right in front of me. How can I pull database from the database and shove it into the form as initial values?
I'm not really sure I understand what you're trying to do - if you're just interested in a single form, I don't know why you're getting involved in formsets at all.
To populate a modelform with initial data from the database, you just pass the instance argument:
my_form = Set_Personal_Info(instance=UserProfile.objects.get(id=request.user.id))
Don't forget to also pass the instance argument when you're instantiating the form on POST, so that Django updates the existing instance rather than creating a new one.
(Note you might want to think about giving better names to your objects. obj usually describes a model instance, rather than a form, for which form would be a better name. And form classes should follow PEP8, and probably include the word 'form' - so PersonalInfoForm would be a good name.)
Based on what I've understand ... if you want to generate a form with dynamic fields you can use this:
class MyModelForm(forms.ModelForm):
def __init__(self, dynamic_fields, *args, **kwargs):
super(MyModelForm, self).__init__(*args, **kwargs)
self.fields = fields_for_model(self._meta.model, dynamic_fields, self._meta.exclude, self._meta.widgets)
class Meta:
model = MyModel
Where dynamic_fields is a tuple.
More on dynamic forms:
http://www.rossp.org/blog/2008/dec/15/modelforms/
http://jacobian.org/writing/dynamic-form-generation/
http://dougalmatthews.com/articles/2009/dec/16/nicer-dynamic-forms-django/
Also Daniel's approach is valid and clean ... Based on your different ids/types etc you can you use different Form objects
forms.py
class MyModelFormA(forms.ModelForm):
class Meta:
model = MyModel
fields = ('field_a','field_b','field_c')
class MyModelFormB(forms.ModelForm):
class Meta:
model = MyModel
fields = ('field_d','field_e','field_f')
views.py
if request.method == 'POST':
if id == 1:
form = MyModelFormA(data=request.POST)
elif id == 2:
form = MyModelFormB(data=request.POST)
else:
form = MyModelFormN(data=request.POST)
if form.is_valid():
form.save() else:
if id == 1:
form = MyModelFormA()
elif id == 2:
form = MyModelFormB()
else:
form = MyModelFormN()

Building Select Options in Django

I'd like to create a form that when viewed, the user's favorite fruits are queried from the database and displayed as follows:
<select size="4">
<option selected>Apples</option>
<option>Bananas</option>
<option>Oranges</option>
<option>Watermelon</option>
</select>
The view that uses the form will:
Get the user object.
Query the database for the user's favorite fruits. (Each is a separate object of the Fruit model.)
Load the form with the fruit choices collected in (2).
I was considering using the ChoiceField, but it looks like you cannot load the list of choices into the form dynamically, at least in a straightforward manner. Am I better off skipping the form and generating the code directly at the template? Or is there a way to load the form's ChoiceField with the user items at the view?
Also, are there any general rules of thumb that dictate where it's easier to build a form using the django form fields vs generating the form code at the template?
I found the answer in this stack overflow topic. The trick is to override the form __init__() so that it accepts a new keyword argument, which in this case is the user.
views.py snippet
context = RequestContext(request)
user = User.objects.get(username=context['user'])
form = forms.FruitForm(user=user)
forms.py snippet
from django import forms
class FruitForm(forms.Form):
fruits = forms.ModelChoiceField(queryset=Fruit.objects.all())
def __init__(self, *args, **kwargs):
user = kwargs.pop('user', None)
super(FruitForm, self).__init__(*args, **kwargs)
if user:
self.fields['fruits'].queryset = Fruit.objects.filter(user=user)
It's not that difficult. You can accomplish this easily using a modelform.
See: https://docs.djangoproject.com/en/dev/topics/forms/modelforms/
One of the strengths of the Django framework is it's form handling and validation methods. So if possible, it always better for you to use Django forms or model forms.
Create a Form or a ModelForm that will be used in you view. The differnce between the two classes is the the ModelForm is built to closely resemble a database model defined in your models.py file where a Form can have custom attributes.
from django.forms import ModelForm
class FruitForm(ModelForm):
class Meta:
model = User
fields = ('favorite-fruits', )