After creating a clean() method to avoid overlapping date ranges in an admin form, I added an ExclusionContraint to ensure integrity at the DB level, too:
class DateRangeFunc(models.Func):
function = 'daterange'
output_field = DateRangeField()
class Occupancy(models.Model):
unit = models.ForeignKey(Unit, on_delete=models.CASCADE)
number_of = models.IntegerField()
begin = models.DateField()
end = models.DateField(default=datetime.date(9999,12,31))
class Meta:
constraints = [
ExclusionConstraint(
name="exclude_overlapping_occupancies",
expressions=(
(
DateRangeFunc(
"begin", "end", RangeBoundary(inclusive_lower=True, inclusive_upper=True)
),
RangeOperators.OVERLAPS,
),
("unit", RangeOperators.EQUAL),
),
),
]
This constraint works as expected, but it seems to precede clean(), because any overlap raises an IntegrityError for the admin form. I would have expected that clean() is called first.
I have two questions (related, but not identical to this question):
Is there any way to change the order of evaluation (clean() → ExclusionConstraint)?
Which method (save()?) would I need to override to catch the IntegrityError raised by the constraint?
[Django 4.1.5/Python 3.11.1/PostgreSQL 14.6]
Related
I am using Django 3.2.
Backend database used for testing: sqlite3
I have a model Foo:
class Foo(models.Model):
# some fields ...
some_count = models.IntegerField()
class Meta:
models.constraints = [
models.CheckConstraint(
check = ~models.Q(some_count=0),
name = 'check_some_count',
),
]
I also have a unit test like this:
def test_Foo_some_count_field_zero_value(self):
# Wrap up the db error in a transaction, so it doesn't get percolated up the stack
with transaction.atomic():
with self.assertRaises(IntegrityError) as context:
baker.make(Foo, some_count=0)
When my unit tests are run, it fails at the test above, with the error message:
baker.make(Foo, some_count=0)
AssertionError: IntegrityError not raised
I then changed the CheckConstraint attribute above to:
class Meta:
models.constraints = [
models.CheckConstraint(
check = models.Q(some_count__lt=0) | models.Q(some_count__gt=0),
name = 'check_some_count',
),
]
The test still failed, with the same error message. I then tried this to check if constraints were being enforced at all:
def test_Foo_some_count_field_zero_value(self):
foo = baker.make(Foo, some_count=0)
self.assertEqual(foo.some_count, 0)
To my utter dismay, the test passed - clearly showing that the constraint check was being ignored. I've done a quick lookup online to see if this is a known issue with sqlite3, but I haven't picked up anything on my radar yet - so why is the constraint check being ignored - and how do I fix it (without overriding models.Model.clean() ) ?
I'm having some trouble generating an annotation for the following models:
class ResultCode(GenericSteamDataModel):
id = models.IntegerField(db_column='PID')
result_code = models.IntegerField(db_column='resultcode', primary_key=True)
campaign = models.OneToOneField(SteamCampaign, db_column='campagnePID', on_delete=models.CASCADE)
sale = models.BooleanField(db_column='ishit')
factor = models.DecimalField(db_column='factor', max_digits=5, decimal_places=2)
class Meta:
managed = False
constraints = [
models.UniqueConstraint(fields=['result_code', 'campaign'], name='result_code per campaign unique')
]
class CallStatistics(GenericShardedDataModel, GenericSteamDataModel):
objects = CallStatisticsManager()
project = models.OneToOneField(SteamProject, primary_key=True, db_column='projectpid', on_delete=models.CASCADE)
result_code = models.ForeignKey(ResultCode, db_column='resultcode', on_delete=models.CASCADE)
class Meta:
managed = False
The goal is to find the sum of factors based on the result_code field in the ResultCode and CallStatistics model, when sale=True.
Note that:
Result codes are not unique by themselves (described in model). A Project has a relation to a Campaign
The following annotation generates the result that is desired (possible solution):
result = CallStatistics.objects.all().values('project').annotate(
sales_factored=models.Sum(
models.Case(
models.When(
models.Q(sale=True) & models.Q(project__campaign=models.F('result_code__campaign')),
then=models.F('result_code__factor')
)
)
)
)
The problem is that the generated query performs a Inner Join on result_code between the 2 models.
Trying to add another field in the same annotation (that should not be joined with Resultcode), for example:
sales=models.Sum(Cast('sale', models.IntegerField())),
results in a wrong summation.
The Questions is if there is an alternative to the automatic Inner Join that Django generates. So that it is possible to retrieve the following fields (and others similar) in 1 annotation:
...
sales=models.Sum(Cast('sale', models.IntegerField())),
sales_factored= [sum of factores, without Inner Join]
...
Thanks in advance for taking your time for this.
I have a Django model where each instance requires a unique identifier that is derived from three fields:
class Example(Model):
type = CharField(blank=False, null=False) # either 'A' or 'B'
timestamp = DateTimeField(default=timezone.now)
number = models.IntegerField(null=True) # a sequential number
This produces a label of the form [type][timestamp YEAR][number], which must be unique unless number is null.
I thought I might be able to use a couple of annotations:
uid_expr = Case(
When(
number=None,
then=Value(None),
),
default=Concat(
'type', ExtractYear('timestamp'), 'number',
output_field=models.CharField()
),
output_field=models.CharField()
)
uid_count_expr = Count('uid', distinct=True)
I overrode the model's manager's get_queryset to apply the annotations by default and then tried to use CheckConstraint:
class Example(Model):
...
class Meta:
constraints = [
models.CheckConstraint(check=Q(uid_cnt=1), name='unique_uid')
]
This fails because it's unable to find a field on the instance called uid_cnt, however I thought annotations were accessible to Q objects. It looks like CheckConstraint queries against the model directly rather than using the queryset returned by the manager:
class CheckConstraint(BaseConstraint):
...
def _get_check_sql(self, model, schema_editor):
query = Query(model=model)
...
Is there a way to apply a constraint to an annotation? Or is there a better approach?
I'd really like to enforce this at the db layer.
Thanks.
This is pseudo-code, but try:
class Example(Model):
...
class Meta:
constraints = [
models.UniqueConstraint(
fields=['type', 'timestamp__year', 'number'],
condition=Q(number__isnull=False),
name='unique_uid'
)
]
Maybe I misunderstand the purpose of Django's update_or_create Model method.
Here is my Model:
from django.db import models
import datetime
from vc.models import Cluster
class Vmt(models.Model):
added = models.DateField(default=datetime.date.today, blank=True, null=True)
creation_time = models.TextField(blank=True, null=True)
current_pm_active = models.TextField(blank=True, null=True)
current_pm_total = models.TextField(blank=True, null=True)
... more simple fields ...
cluster = models.ForeignKey(Cluster, null=True)
class Meta:
unique_together = (("cluster", "added"),)
Here is my test:
from django.test import TestCase
from .models import *
from vc.models import Cluster
from django.db import transaction
# Create your tests here.
class VmtModelTests(TestCase):
def test_insert_into_VmtModel(self):
count = Vmt.objects.count()
self.assertEqual(count, 0)
# create a Cluster
c = Cluster.objects.create(name='test-cluster')
Vmt.objects.create(
cluster=c,
creation_time='test creaetion time',
current_pm_active=5,
current_pm_total=5,
... more simple fields ...
)
count = Vmt.objects.count()
self.assertEqual(count, 1)
self.assertEqual('5', c.vmt_set.all()[0].current_pm_active)
# let's test that we cannot add that same record again
try:
with transaction.atomic():
Vmt.objects.create(
cluster=c,
creation_time='test creaetion time',
current_pm_active=5,
current_pm_total=5,
... more simple fields ...
)
self.fail(msg="Should violated integrity constraint!")
except Exception as ex:
template = "An exception of type {0} occurred. Arguments:\n{1!r}"
message = template.format(type(ex).__name__, ex.args)
self.assertEqual("An exception of type IntegrityError occurred.", message[:45])
Vmt.objects.update_or_create(
cluster=c,
creation_time='test creaetion time',
# notice we are updating current_pm_active to 6
current_pm_active=6,
current_pm_total=5,
... more simple fields ...
)
count = Vmt.objects.count()
self.assertEqual(count, 1)
On the last update_or_create call I get this error:
IntegrityError: duplicate key value violates unique constraint "vmt_vmt_cluster_id_added_c2052322_uniq"
DETAIL: Key (cluster_id, added)=(1, 2018-06-18) already exists.
Why didn't wasn't the model updated? Why did Django try to create a new record that violated the unique constraint?
The update_or_create(defaults=None, **kwargs) has basically two parts:
the **kwargs which specify the "filter" criteria to determine if such object is already present; and
the defaults which is a dictionary that contains the fields mapped to values that should be used when we create a new row (in case the filtering fails to find a row), or which values should be updated (in case we find such row).
The problem here is that you make your filters too restrictive: you add several filters, and as a result the database does not find such row. So what happens? The database then aims to create the row with these filter values (and since defaults is missing, no extra values are added). But then it turns out that we create a row, and that the combination of the cluster and added already exists. Hence the database refuses to add this row.
So this line:
Model.objects.update_or_create(field1=val1,
field2=val2,
defaults={
'field3': val3,
'field4': val4
})
Is to semantically approximately equal to:
try:
item = Model.objects.get(field1=val1, field2=val2)
except Model.DoesNotExist:
Model.objects.create(field1=val1, field2=val2, field3=val3, field4=val4)
else:
item = Model.objects.filter(
field1=val1,
field2=val2,
).update(
field3 = val3
field4 = val4
)
(but the original call is typically done in a single query).
You probably thus should write:
Vmt.objects.update_or_create(
cluster=c,
creation_time='test creaetion time',
defaults = {
'current_pm_active': 6,
'current_pm_total': 5,
}
)
(or something similar)
You should separate your field:
Fields that should be searched for
Fields that should be updated
for example:
If I have the model:
class User(models.Model):
username = models.CharField(max_length=200)
nickname = models.CharField(max_length=200)
And I want to search for username = 'Nikolas' and update this instance nickname to 'Nik'(if no User with username 'Nikolas' I need to create it) I should write this code:
User.objects.update_or_create(
username='Nikolas',
defaults={'nickname': 'Nik'},
)
see in https://docs.djangoproject.com/en/3.1/ref/models/querysets/
This is already answered well in the above.
To be more clear the update_or_create() method should have **kwargs as those parameters on which you want to check if that data already exists in DB by filtering.
select some_column from table_name where column1='' and column2='';
Filtering by **kwargs will give you objects. Now if you wish to update any data/column of those filtered objects, you should pass them in defaults param in update_or_create() method.
so lets say you found an object based on a filter now the default param values are expected to be picked and updated.
and if there's no matching object found based on the filter then it goes ahead and creates an entry with filters and the default param passed.
I need to skip validation of the OrderAmount field but for it to still save the invalidated data. Is there a way this can be done? I know django allows you to make your own validation, but I don't know how to make it completely skip just one field's validation.
model:
class LiquorOrder(models.Model):
pack_size = (
('1', '1'),
('2', '2'),
)
LiquorOrderID = models.AutoField(primary_key=True)
storeliquorID = models.ForeignKey(StoreLiquor)
orderID = models.ForeignKey(Order)
OrderAmount = models.PositiveSmallIntegerField('Order Amount', max_length=3, choices=pack_size)
TotalPrice = models.DecimalField('Total Price', max_digits=5, decimal_places=2)
Form:
class AddToOrderForm(forms.ModelForm):
class Meta:
model = LiquorOrder
fields = ('OrderAmount',)
For a PositiveSmallIntegerField the only validation Django does is ensure the value is indeed a positive integer within the appropriate range. If you were to skip this, you would run into problems when Django tries to write the value to your database. If you were to, say, try to write the value "marshmallows" to a DB column that's expecting an integer, the DB will throw errors and Django will turn around and throw you an IntegrityError.
If you really wanted to try, you could override the field to be CharField with required set to False (basically allowing any keyboard input):
class AddToOrderForm(forms.ModelForm):
OrderAmount = forms.CharField(required=False)
Django would then return True when you run is_valid(), but throw errors when you try to call save().
It sounds like your real issue is with your model not matching your current project requirements. If that is the case, look into migrating to a new model. The Python library South is brilliant tool for this purpose and is used heavily by the Django community. I would read up on DB migrations and see if you can come up with a solution that way.