I needed to validate a model field to be unique as lower case, but without actually saving the model field in lower case; e.g. if someone has already taken the username 'david', then the username 'David' would not be available. I tried various things, and eventually ended up doing the following:
def _perform_unique_checks(self, unique_checks):
errors = {}
for model_class, unique_check in unique_checks:
lookup_kwargs = {}
for field_name in unique_check:
f = self._meta.get_field(field_name)
lookup_value = getattr(self, f.attname)
if lookup_value is None:
continue
if f.primary_key and not self._state.adding:
continue
lookup_kwargs[str(field_name)] = lookup_value
if len(unique_check) != len(lookup_kwargs):
continue
if 'username' in lookup_kwargs:
lookup_kwargs['username'] = lookup_kwargs['username'].lower()
qs = model_class._default_manager.filter(**lookup_kwargs)
model_class_pk = self._get_pk_val(model_class._meta)
if not self._state.adding and model_class_pk is not None:
qs = qs.exclude(pk=model_class_pk)
if qs.exists():
if len(unique_check) == 1:
key = unique_check[0]
else:
key = NON_FIELD_ERRORS
errors.setdefault(key, []).append(
self.unique_error_message(model_class, unique_check))
... which works, but feels a little convoluted to me. I wondered whether any there's a more succinct way of achieving this?
Have you tried filtering on field_name__iexact, to do a case insensitive match?
The iregex, icontains and iexact filters should be able to do what you need.
First off, you can user the clean for this functionality (triggered when trying to validate a form):
def clean__field_name(self):
Also, you can add a unique=True constraint to the field in question. Then do a Try/Except during save to catch any IntegrityError which will tell you that the field is not unique. The database and model should do this work for you instead of trying to code around it.
Also see Case insensitive unique model fields in Django?
With more detail (database type, copy of model, are you using a Form?), I can expand on this answer.
Related
I was surprised that this was difficult to do. However I came up with this, which seems to work at least for my simple case. Can anyone recommend a better approach?
def field_changed(self, fieldname):
"""Tests if the value of the field changed from the original data"""
orig_value = self.fields[fieldname].initial or getattr(self.instance, field, None)
orig_value = getattr(orig_value, 'pk', orig_value)
if type(orig_value) is bool:
# because None and False can be interchangeable
return bool(self.data.get(fieldname)) != bool(orig_value)
else:
return unicode(self.data.get(fieldname)) != unicode(orig_value)
Form contains a property changed_data which holds a list of all the fields whose values have changed.
Try:
'fieldname' in myforminstance.changed_data
It seems that you have reinvented .has_changed() method.
Extending on Airs's answer, for multiple fields:
In a scenario where you'd like to track changes for a list of fields ['field_a', 'field_b', 'field_c']
If you'd like to check if any of those fields has changed:
any(x in myforminstance.changed_data for x in ['field_a', 'field_b', 'field_c'])
If you'd like to check if all of those fields have changed:
all(x in myforminstance.changed_data for x in ['field_a', 'field_b', 'field_c'])
You must override the method post_save. Overriding methods is a good practice in Django: https://docs.djangoproject.com/en/dev/ref/signals/#post-save
I suggest you adding something like this into your model class:
def post_save(self, sender, instance, created, raw, using, update_fields):
if 'the_field' in update_fields:
# Do whatever...
I'm using Django 1.8.4 in my dev machine using Sqlite and I have these models:
class ModelA(Model):
field_a = CharField(verbose_name='a', max_length=20)
field_b = CharField(verbose_name='b', max_length=20)
class Meta:
unique_together = ('field_a', 'field_b',)
class ModelB(Model):
field_c = CharField(verbose_name='c', max_length=20)
field_d = ForeignKey(ModelA, verbose_name='d', null=True, blank=True)
class Meta:
unique_together = ('field_c', 'field_d',)
I've run proper migration and registered them in the Django Admin. So, using the Admin I've done this tests:
I'm able to create ModelA records and Django prohibits me from creating duplicate records - as expected!
I'm not able to create identical ModelB records when field_b is not empty
But, I'm able to create identical ModelB records, when using field_d as empty
My question is: How do I apply unique_together for nullable ForeignKey?
The most recent answer I found for this problem has 5 year... I do think Django have evolved and the issue may not be the same.
Django 2.2 added a new constraints API which makes addressing this case much easier within the database.
You will need two constraints:
The existing tuple constraint; and
The remaining keys minus the nullable key, with a condition
If you have multiple nullable fields, I guess you will need to handle the permutations.
Here's an example with a thruple of fields that must be all unique, where only one NULL is permitted:
from django.db import models
from django.db.models import Q
from django.db.models.constraints import UniqueConstraint
class Badger(models.Model):
required = models.ForeignKey(Required, ...)
optional = models.ForeignKey(Optional, null=True, ...)
key = models.CharField(db_index=True, ...)
class Meta:
constraints = [
UniqueConstraint(fields=['required', 'optional', 'key'],
name='unique_with_optional'),
UniqueConstraint(fields=['required', 'key'],
condition=Q(optional=None),
name='unique_without_optional'),
]
UPDATE: previous version of my answer was functional but had bad design, this one takes in account some of the comments and other answers.
In SQL NULL does not equal NULL. This means if you have two objects where field_d == None and field_c == "somestring" they are not equal, so you can create both.
You can override Model.clean to add your check:
class ModelB(Model):
#...
def validate_unique(self, exclude=None):
if ModelB.objects.exclude(id=self.id).filter(field_c=self.field_c, \
field_d__isnull=True).exists():
raise ValidationError("Duplicate ModelB")
super(ModelB, self).validate_unique(exclude)
If used outside of forms you have to call full_clean or validate_unique.
Take care to handle the race condition though.
#ivan, I don't think that there's a simple way for django to manage this situation. You need to think of all creation and update operations that don't always come from a form. Also, you should think of race conditions...
And because you don't force this logic on DB level, it's possible that there actually will be doubled records and you should check it while querying results.
And about your solution, it can be good for form, but I don't expect that save method can raise ValidationError.
If it's possible then it's better to delegate this logic to DB. In this particular case, you can use two partial indexes. There's a similar question on StackOverflow - Create unique constraint with null columns
So you can create Django migration, that adds two partial indexes to your DB
Example:
# Assume that app name is just `example`
CREATE_TWO_PARTIAL_INDEX = """
CREATE UNIQUE INDEX model_b_2col_uni_idx ON example_model_b (field_c, field_d)
WHERE field_d IS NOT NULL;
CREATE UNIQUE INDEX model_b_1col_uni_idx ON example_model_b (field_c)
WHERE field_d IS NULL;
"""
DROP_TWO_PARTIAL_INDEX = """
DROP INDEX model_b_2col_uni_idx;
DROP INDEX model_b_1col_uni_idx;
"""
class Migration(migrations.Migration):
dependencies = [
('example', 'PREVIOUS MIGRATION NAME'),
]
operations = [
migrations.RunSQL(CREATE_TWO_PARTIAL_INDEX, DROP_TWO_PARTIAL_INDEX)
]
Add a clean method to your model - see below:
def clean(self):
if Variants.objects.filter("""Your filter """).exclude(pk=self.pk).exists():
raise ValidationError("This variation is duplicated.")
I think this is more clear way to do that for Django 1.2+
In forms it will be raised as non_field_error with no 500 error, in other cases, like DRF you have to check this case manual, because it will be 500 error.
But it will always check for unique_together!
class BaseModelExt(models.Model):
is_cleaned = False
def clean(self):
for field_tuple in self._meta.unique_together[:]:
unique_filter = {}
unique_fields = []
null_found = False
for field_name in field_tuple:
field_value = getattr(self, field_name)
if getattr(self, field_name) is None:
unique_filter['%s__isnull' % field_name] = True
null_found = True
else:
unique_filter['%s' % field_name] = field_value
unique_fields.append(field_name)
if null_found:
unique_queryset = self.__class__.objects.filter(**unique_filter)
if self.pk:
unique_queryset = unique_queryset.exclude(pk=self.pk)
if unique_queryset.exists():
msg = self.unique_error_message(self.__class__, tuple(unique_fields))
raise ValidationError(msg)
self.is_cleaned = True
def save(self, *args, **kwargs):
if not self.is_cleaned:
self.clean()
super().save(*args, **kwargs)
One possible workaround not mentioned yet is to create a dummy ModelA object to serve as your NULL value. Then you can rely on the database to enforce the uniqueness constraint.
I have a very large database (6 GB) that I would like to use Django-REST-Framework with. In particular, I have a model that has a ForeignKey relationship to the django.contrib.auth.models.User table (not so big) and a Foreign Key to a BIG table (lets call it Products). The model can be seen below:
class ShoppingBag(models.Model):
user = models.ForeignKey('auth.User', related_name='+')
product = models.ForeignKey('myapp.Product', related_name='+')
quantity = models.SmallIntegerField(default=1)
Again, there are 6GB of Products.
The serializer is as follows:
class ShoppingBagSerializer(serializers.ModelSerializer):
product = serializers.RelatedField(many=False)
user = serializers.RelatedField(many=False)
class Meta:
model = ShoppingBag
fields = ('product', 'user', 'quantity')
So far this is great- I can do a GET on the list and individual shopping bags, and everything is fine. For reference the queries (using a query logger) look something like this:
SELECT * FROM myapp_product WHERE product_id=1254
SELECT * FROM auth_user WHERE user_id=12
SELECT * FROM myapp_product WHERE product_id=1404
SELECT * FROM auth_user WHERE user_id=12
...
For as many shopping bags are getting returned.
But I would like to be able to POST to create new shopping bags, but serializers.RelatedField is read-only. Let's make it read-write:
class ShoppingBagSerializer(serializers.ModelSerializer):
product = serializers.PrimaryKeyRelatedField(many=False)
user = serializers.PrimaryKeyRelatedField(many=False)
...
Now things get bad... GET requests to the list action take > 5 minutes and I noticed that my server's memory jumps up to ~6GB; why?! Well, back to the SQL queries and now I see:
SELECT * FROM myapp_products;
SELECT * FROM auth_user;
Ok, so that's not good. Clearly we're doing "prefetch related" or "select_related" or something like that in order to get access to all the products; but this table is HUGE.
Further inspection reveals where this happens on Line 68 of relations.py in DRF:
def initialize(self, parent, field_name):
super(RelatedField, self).initialize(parent, field_name)
if self.queryset is None and not self.read_only:
manager = getattr(self.parent.opts.model, self.source or field_name)
if hasattr(manager, 'related'): # Forward
self.queryset = manager.related.model._default_manager.all()
else: # Reverse
self.queryset = manager.field.rel.to._default_manager.all()
If not readonly, self.queryset = ALL!!
So, I'm pretty sure that this is where my problem is; and I need to say, don't select_related here, but I'm not 100% if this is the issue or where to deal with this. It seems like all should be memory safe with pagination, but this is simply not the case. I'd appreciate any advice.
In the end, we had to simply create our own PrimaryKeyRelatedField class to override the default behavior in Django-Rest-Framework. Basically we ensured that the queryset was None until we wanted to lookup the object, then we performed the lookup. This was extremely annoying, and I hope the Django-Rest-Framework guys take note of this!
Our final solution:
class ProductField(serializers.PrimaryKeyRelatedField):
many = False
def __init__(self, *args, **kwargs):
kwarsgs['queryset'] = Product.objects.none() # Hack to ensure ALL products are not loaded
super(ProductField, self).__init__(*args, **kwargs)
def field_to_native(self, obj, field_name):
return unicode(obj)
def from_native(self, data):
"""
Perform query lookup here.
"""
try:
return Product.objects.get(pk=data)
except Product.ObjectDoesNotExist:
msg = self.error_messages['does_not_exist'] % smart_text(data)
raise ValidationError(msg)
except (TypeError, ValueError):
msg = self.error_messages['incorrect_type'] % type(data)
raise ValidationError(msg)
And then our serializer is as follows:
class ShoppingBagSerializer(serializers.ModelSerializer):
product = ProductField()
...
This hack ensures the entire database isn't loaded into memory, but rather performs one-off selects based on the data. It's not as efficient computationally, but it also doesn't blast our server with 5 second database queries loaded into memory!
Firstly, I did my homework and looked around before posting! My question seems like a very basic thing that must’ve been covered before.
I'm now looking at Django-filter as a potential solution, but would like some advice on if this is the right way to go and if there any other solutions.
I have a Django app wit 10 models, each model has a few fields. Most fields are ChoiceField that users populate using forms with the default select widget. There is a separate form for each model.
I want to create a separate form for each model (in separate views) that users will use to search the database. The search form will contain only drop-down boxes (the select widgets) with the same choices as the forms used to populate the database with the addition of the “any” option.
I know how to use .object.filter(), however the “any” option would correspond to not include specific fields in the filter and I'm not sure how to add model fields to the filter based on users’ selection
I briefly looked at Haystack as an option but it seems to be made for full text search rather than “model filed search” I'm after.
Sample model (simplified):
class Property():
TYPE_CHOICES = (‘apartment’, ‘house’, ‘flat’)
type = charfield(choices=TYPE_CHOICES)
LOC_CHOICES = (‘Brussels’, ‘London’, ‘Dublin’, ‘Paris’)
location = charfield(choices=LOC_CHOICES)
price = PostivieInteger()
Users can select only “type”, only “location” or both (not making selection is equal to ANY) in which case I end up with 3 different filters:
Property.objects.filter(type=’apartment’)
Property.objects.filter(location=’Dublin’)
Property.objects.filter(type=’apartment’, location=’Dublin’)
The main question: django-filter the best option?
Question 1: what’s the best option of accomplishing this overall?
Question 2: how do I add model fields to the filter based on user’s form selection?
Question 3: how do I do the filter based on user selection? (I know how to use .filter(price_lt=).exclude(price_gt=) but again how do I do it dynamically based on selection as “ANY” would mean this is not included in the query)
I had a similar case like yours (real estate project), I ended up with the following approach, you can refine this to your needs...I removed select_related and prefetch_related models for easier reading
properties/forms.py:
class SearchPropertyForm(forms.Form):
property_type = forms.ModelChoiceField(label=_("Property Type"), queryset=HouseType.objects.all(),widget=forms.Select(attrs={'class':'form-control input-sm'}))
location = forms.ModelChoiceField(label=_('Location'), queryset=HouseLocation.objects.all(), widget=forms.Select(attrs={'class':'form-control input-sm'}))
Then in the properties/views.py
# Create a Mixin to inject the search form in our context
class SeachPropertyMixin(object):
def get_context_data(self, **kwargs):
context = super(SeachPropertyMixin, self).get_context_data(**kwargs)
context['search_property_form'] = SearchPropertyForm()
return context
In your actual view (I apply the search form as a sidebar element in my detailview only:
# Use Class Based views, saves you a great deal of repeating code...
class PropertyView(SeachPropertyMixin,DetailView):
template_name = 'properties/view.html'
context_object_name = 'house'
...
queryset = HouseModel.objects.select_related(...).prefetch_related(...).filter(flag_active=True, flag_status='a')
Finally your search result view (this is performed as GET request, since we are not altering any data in our DB, we stick to the GET method):
# Search results should return a ListView, here is how we implement it:
class PropertySearchResultView(ListView):
template_name = "properties/propertysearchresults.html"
context_object_name = 'houses'
paginate_by = 6
queryset = HouseModel.objects.select_related(...).prefetch_related(...).order_by('-sale_price').filter(flag_active=True, flag_status='a')
def get_queryset(self):
qs = super(PropertySearchResultView,self).get_queryset()
property_type = self.request.GET.get('property_type')
location = self.request.GET.get('location')
'''
Start Chaining the filters based on the input, this way if the user has not
selected a filter it wont be used.
'''
if property_type != '' and property_type is not None:
qs = qs.filter(housetype=property_type)
if location != '' and location is not None:
qs = qs.filter(location=location)
return qs
def get_context_data(self, **kwargs):
context = super(PropertySearchResultView, self).get_context_data()
'''
Add the current request to the context
'''
context['current_request'] = self.request.META['QUERY_STRING']
return context
Your solution works. I've modified it and I'm not using ModelChoiceField but the standard form.ChoiceField. The reason for that is that I wanted to add option "Any". My "if" statements look like:
if locality != 'Any Locality':
qs = qs.filter(locality=locality)
if property_type != 'Any Type':
qs = qs.filter(property_type=property_type)
if int(price_min) != 0:
qs = qs.filter(price__gte=price_min)
if int(price_max) != 0:
qs = qs.filter(price__lte=price_max)
if bedrooms != 'Any Number':
qs = qs.filter(bedrooms=bedrooms)
And so on....
This does the job, however it seems like an ugly and hacky solution to a simple problem. I would think is a common use case. I feel there should be a cleaner solution...
I've tried the django-filter. It is close to doing what I want but I couldn't add the "Any" choice and it filters inline rather than returning. It should do with some modifications.
Cheers
I'm trying to write an internal API in my application without necessarily coupling it with the database.
class Product(models.Model):
name=models.CharField(max_length=4000)
price=models.IntegerField(default=-1)
currency=models.CharField(max_length=3, default='INR')
class Image(models.Model):
# NOTE -- Have changed the table name to products_images
width=models.IntegerField(default=-1)
height=models.IntegerField(default=-1)
url=models.URLField(max_length=1000, verify_exists=False)
product=models.ForeignKey(Product)
def create_product:
p=Product()
i=Image(height=100, widght=100, url='http://something/something')
p.image_set.add(i)
return p
Now, when I call create_product() Django throws up an error:
IntegrityError: products_images.product_id may not be NULL
However, if I call p.save() & i.save() before calling p.image_set.add(i) it works. Is there any way that I can add objects to a related object set without saving both to the DB first?
def create_product():
product_obj = Product.objects.create(name='Foobar')
image_obj = Image.objects.create(height=100, widght=100, url='http://something/something', product=product_obj)
return product_obj
Explanation:
Product object has to be created first and then assign it to the Image object because id and name here is required field.
I am wondering why wouldn't you not require to make a product entry in DB in first case? If there is any specific reason then i may suggest you some work around?
EDIT: Okay! i think i got you, you don't want to assign a product to an image object initially. How about creating a product field as null is equal to true.
product = models.ForeignKey(Product, null=True)
Now, your function becomes something like this:
def create_product():
image_obj = Image.objects.create(height=100, widght=100, url='http://something/something')
return image_obj
Hope it helps you?
I got same issue with #Saurabh Nanda
I am using Django 1.4.2. When I read in django, i see that
# file django/db/models/fields/related.py
def get_query_set(self):
try:
return self.instance._prefetched_objects_cache[rel_field.related_query_name()]
except (AttributeError, KeyError):
db = self._db or router.db_for_read(self.model, instance=self.instance)
return super(RelatedManager,self).get_query_set().using(db).filter(**self.core_filters)
# file django/db/models/query.py
qs = getattr(obj, attname).all()
qs._result_cache = vals
# We don't want the individual qs doing prefetch_related now, since we
# have merged this into the current work.
qs._prefetch_done = True
obj._prefetched_objects_cache[cache_name] = qs
That 's make sese, we only need to set property _prefetched_objects_cache for the object.
p = Product()
image_cached = []
for i in xrange(100):
image=Image(height=100, widght=100, url='http://something/something')
image_cached.append(image)
qs = p.images.all()
qs._result_cache = image_cached
qs._prefetch_done = True
p._prefetched_objects_cache = {'images': qs}
Your problem is that the id isn't set by django, but by the database (it's represented in the database by an auto-incremented field), so until it's saved there's no id. More about this in the documentation.
I can think of three possible solutions:
Set a different field of your Image model as the primary key (documented here).
Set a different field of your Production model as the foreign key (documented here).
Use django's database transactions API (documented here).