unique_together doesn't work, it only set the unique constraints on the first field and ignore the second field. Is there any way to enforce unique constraints?
class BaseModel(models.Model):
id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
deleted = models.DateTimeField(db_index=True, null=True, blank=True)
last_modified_at = models.DateTimeField(auto_now=True)
class Meta:
abstract = True
class Book(BaseModel):
first_form_number = models.CharField(max_length=8)
class Meta:
unique_together = (("first_form_number", "deleted"),)
Your models work correctly in that extent that the right unique index is created:
$ python manage.py sqlmigrate app 0001_initial
...
CREATE UNIQUE INDEX "app_base_slug_version_a455c5b7_uniq" ON "app_base" ("slug", "version");
...
(expected like the name of your application is "app")
I must roughly agree with user3541631's answer. It depends on the database in general, but all four db engines supported directly by Django are similar. They expect that "nulls are distinct in a UNIQUE column" (see NULL Handling in SQLite Versus Other Database Engines)
I verified your problem with and without null:
class Test(TestCase):
def test_without_null(self):
timestamp = datetime.datetime(2017, 8, 25, tzinfo=pytz.UTC)
book_1 = Book.objects.create(deleted=timestamp, first_form_number='a')
with self.assertRaises(django.db.utils.IntegrityError):
Book.objects.create(deleted=timestamp, first_form_number='a')
def test_with_null(self):
# this test fails !!! (and a duplicate is created)
book_1 = Book.objects.create(first_form_number='a')
with self.assertRaises(django.db.utils.IntegrityError):
Book.objects.create(first_form_number='a')
A solution is possible for PostgreSQL if you are willing to manually write a migration to create two special partial unique indexes:
CREATE UNIQUE INDEX book_2col_uni_idx ON app_book (first_form_number, deleted)
WHERE deleted IS NOT NULL;
CREATE UNIQUE INDEX book_1col_uni_idx ON app_book (first_form_number)
WHERE deleted IS NULL;
See:
Answer for Create unique constraint with null columns
Django docs Writing database migrations
Django docs migrations.RunSQL(sql)
depending on your database, it is possible that NULL isn't equal to any other NULL.
Therefore the rows you create are not the same, if one of the values is NULL, will be unique only by the non null field, in your case 'first_form_number'.
Also take in consideration that is case sensitive so "char" and "Char" are not the same.
I had a similar situation and I did my own check by overriding the save method on the model.
You check if exist in the database, but also exclude the current instance, in case of updating, not to compare with itself..
if not deleted:
exists = model.objects.exclude(pk=instance.pk).filter(first_form_number__iexact=first_form_number).exists()
Make sure you actually extend the inherited Meta class, rather than defining your own Meta class (which is ignored by Django):
class Meta(BaseModel.Meta):
unique_together = (("first_form_number", "deleted"),)
Related
I have models with this layout:
class SafeDeleteModel(models.Model):
.....
deleted = models.DateTimeField(editable=False, null=True)
......
class MyModel(SafeDeleteModel):
safedelete_policy = SOFT_DELETE
field1 = models.CharField(max_length=200)
field2 = models.CharField(max_length=200)
field3 = models.ForeignKey(MyModel3)
field4 = models.ForeignKey(MyModel4)
field5 = models.ForeignKey(MyModel5)
class Meta:
unique_together = [['field2', 'field3', 'field4', 'deleted'],]
The scenario here is that I never want users to delete data. Instead a delete will just hide records. However, I still want all non-soft-deleted records to respect unique key constraints. Basically, I want to have as many duplicated deleted records, but only a single unique un-deleted record can exist. So I was thinking to include "deleted" field (provided by django-safedelete library), but the issue becomes that Django's unique checks fail with "psycopg2.IntegrityError: duplicate key value violates unique constraint" for ['field2', 'field3', 'field4', 'deleted'] because NULL is not "equal to" NULL and it yields false in PostgreSQL.
Is there a way to enforce a unique_together constraint with the Django model layout as mine? Or is there a better idea to physically delete the record, then move it to an archive database, and if the user wants the record back, then software will look for the record in the archive and recreate it?
Yes, as of Django version 2.2 it is possible to use a UniqueConstraint with a condition.
Have a look at the documentation in this link: https://docs.djangoproject.com/en/2.2/ref/models/constraints/#uniqueconstraint
So your model would be something like this:
class MyModel(SafeDeleteModel):
safedelete_policy = SOFT_DELETE
field1 = models.CharField(max_length=200)
field2 = models.CharField(max_length=200)
field3 = models.ForeignKey(MyModel3)
field4 = models.ForeignKey(MyModel4)
field5 = models.ForeignKey(MyModel5)
class Meta:
constraints = [
models.UniqueConstraint(
fields=['field2', 'field3', 'field4'],
condition=Q(deleted=False),
name='unique_if_not_deleted')
]
If you are using an older version of Django that doesn't have this feature available, you can create a migration with a partial unique index (have a look at this question here: Postgresql: Conditionally unique constraint).
As for your second question (would it be better to physically delete the record and move it elsewhere), it really depends on the characteristics of your application. If these soft-deletes don't happen very often and your table is still on the small side, I would keep the records in the same table for simplicity's sake, but if the number of records in the table starts growing fast and they affect the performance of the queries on this table then you should move the records elsewhere. You have to evaluate the trade-off between complexity and performance.
Maybe I misunderstand the purpose of Django's update_or_create Model method.
Here is my Model:
from django.db import models
import datetime
from vc.models import Cluster
class Vmt(models.Model):
added = models.DateField(default=datetime.date.today, blank=True, null=True)
creation_time = models.TextField(blank=True, null=True)
current_pm_active = models.TextField(blank=True, null=True)
current_pm_total = models.TextField(blank=True, null=True)
... more simple fields ...
cluster = models.ForeignKey(Cluster, null=True)
class Meta:
unique_together = (("cluster", "added"),)
Here is my test:
from django.test import TestCase
from .models import *
from vc.models import Cluster
from django.db import transaction
# Create your tests here.
class VmtModelTests(TestCase):
def test_insert_into_VmtModel(self):
count = Vmt.objects.count()
self.assertEqual(count, 0)
# create a Cluster
c = Cluster.objects.create(name='test-cluster')
Vmt.objects.create(
cluster=c,
creation_time='test creaetion time',
current_pm_active=5,
current_pm_total=5,
... more simple fields ...
)
count = Vmt.objects.count()
self.assertEqual(count, 1)
self.assertEqual('5', c.vmt_set.all()[0].current_pm_active)
# let's test that we cannot add that same record again
try:
with transaction.atomic():
Vmt.objects.create(
cluster=c,
creation_time='test creaetion time',
current_pm_active=5,
current_pm_total=5,
... more simple fields ...
)
self.fail(msg="Should violated integrity constraint!")
except Exception as ex:
template = "An exception of type {0} occurred. Arguments:\n{1!r}"
message = template.format(type(ex).__name__, ex.args)
self.assertEqual("An exception of type IntegrityError occurred.", message[:45])
Vmt.objects.update_or_create(
cluster=c,
creation_time='test creaetion time',
# notice we are updating current_pm_active to 6
current_pm_active=6,
current_pm_total=5,
... more simple fields ...
)
count = Vmt.objects.count()
self.assertEqual(count, 1)
On the last update_or_create call I get this error:
IntegrityError: duplicate key value violates unique constraint "vmt_vmt_cluster_id_added_c2052322_uniq"
DETAIL: Key (cluster_id, added)=(1, 2018-06-18) already exists.
Why didn't wasn't the model updated? Why did Django try to create a new record that violated the unique constraint?
The update_or_create(defaults=None, **kwargs) has basically two parts:
the **kwargs which specify the "filter" criteria to determine if such object is already present; and
the defaults which is a dictionary that contains the fields mapped to values that should be used when we create a new row (in case the filtering fails to find a row), or which values should be updated (in case we find such row).
The problem here is that you make your filters too restrictive: you add several filters, and as a result the database does not find such row. So what happens? The database then aims to create the row with these filter values (and since defaults is missing, no extra values are added). But then it turns out that we create a row, and that the combination of the cluster and added already exists. Hence the database refuses to add this row.
So this line:
Model.objects.update_or_create(field1=val1,
field2=val2,
defaults={
'field3': val3,
'field4': val4
})
Is to semantically approximately equal to:
try:
item = Model.objects.get(field1=val1, field2=val2)
except Model.DoesNotExist:
Model.objects.create(field1=val1, field2=val2, field3=val3, field4=val4)
else:
item = Model.objects.filter(
field1=val1,
field2=val2,
).update(
field3 = val3
field4 = val4
)
(but the original call is typically done in a single query).
You probably thus should write:
Vmt.objects.update_or_create(
cluster=c,
creation_time='test creaetion time',
defaults = {
'current_pm_active': 6,
'current_pm_total': 5,
}
)
(or something similar)
You should separate your field:
Fields that should be searched for
Fields that should be updated
for example:
If I have the model:
class User(models.Model):
username = models.CharField(max_length=200)
nickname = models.CharField(max_length=200)
And I want to search for username = 'Nikolas' and update this instance nickname to 'Nik'(if no User with username 'Nikolas' I need to create it) I should write this code:
User.objects.update_or_create(
username='Nikolas',
defaults={'nickname': 'Nik'},
)
see in https://docs.djangoproject.com/en/3.1/ref/models/querysets/
This is already answered well in the above.
To be more clear the update_or_create() method should have **kwargs as those parameters on which you want to check if that data already exists in DB by filtering.
select some_column from table_name where column1='' and column2='';
Filtering by **kwargs will give you objects. Now if you wish to update any data/column of those filtered objects, you should pass them in defaults param in update_or_create() method.
so lets say you found an object based on a filter now the default param values are expected to be picked and updated.
and if there's no matching object found based on the filter then it goes ahead and creates an entry with filters and the default param passed.
I'm new to Django and I have some issues with a ManyToMany relationship.
I work on a blastn automatisation and here are my classes:
class Annotation(models.Model):
sequence = models.IntegerField()
annotation = models.TextField()
start = models.IntegerField()
end = models.IntegerField()
class Blast(models.Model):
sequence = models.ManyToManyField(Annotation, through="AnnotBlast")
expectValue = models.IntegerField()
class AnnotBlast(models.Model):
id_blast = models.ForeignKey(Blast, to_field="id")
id_annot = models.ForeignKey(Annotation, to_field="id")
class Hit(models.Model):
id_hit = models.ForeignKey(Blast, to_field="id")
length = models.IntegerField()
evalue = models.IntegerField()
start_seq = models.IntegerField()
end_seq = models.IntegerField()
In a view, I want to access to Annotation's data from the rest of the model via this many to many field and then apply filters based on a form. But when I do a syncdb , the "sequence" field of the Blast class disappear :
In Sqlite3 :
.schema myApp_blast
CREATE TABLE "myApp_blast" (
"id" integer not null primary key,
"expectValue" integer not null
);
So I can't load data in this table as I want. I don't understand why this field disappear during the syncdb. How can I do to link the first class to the others (and then be able to merge data in a template) ?
A ManyToManyField isn't itself a column in the database. It's represented only by an element in the joining table, which you have here defined explicitly as AnnotBlast (note that since you're not defining any extra fields on the relationship, you didn't actually need to define a through table - Django would have done it automatically if you hadn't).
So to add data to your models, you add data to the AnnotBlast table pointing at the relevant Blast and Annotation rows.
For a many-to-many relationship an intermediate join table is created, see documentation here: https://docs.djangoproject.com/en/1.2/ref/models/fields/#id1
I am converting a web project that currently uses the Propel ORM, to a django project.
My first task is to 'port' the model schema to django's.
I have read the django docs, but they do not appear to be in enough detail. Case in point, how may I 'port' a (contrived) table defined in the Propel YML schema as follows:
demo_ref_country:
code: { type: varchar(4), required: true, index: unique }
name: { type: varchar(64), required: true, index: unique }
geog_region_id: { type: integer, foreignTable: demo_ref_geographic_region, foreignReference: id, required: true, onUpdate: cascade, onDelete: restrict }
ccy_id: { type: integer, foreignTable: demo_ref_currency_def, foreignReference: id, required: true, onUpdate: cascade, onDelete: restrict }
flag_image_path: { type: varchar(64), required: true, default: ''}
created_at: ~
_indexes:
idx_f1: [geog_region_id, ccy_id, created_at]
_uniques:
idxu_f1_key: [code, geog_region_id, ccy_id]
Here is my (feeble) attempt so far:
class Country(models.Model):
code = models.CharField(max_length=4) # Erm, no index on this column .....
name = models.CharField(max_length=64) # Erm, no index on this column .....
geog_region_id = models.ForeignKey(GeogRegion) # Is this correct ? (how about ref integrity constraints ?
ccy_id = models.ForeignKey(Currency) # Is this correct?
flag_image_path = models.CharField(max_length=64) # How to set default on this col?
created_at = models.DateTimeField() # Will this default to now() ?
# Don't know how to specify indexes and unique indexes ....
[Edit]
To all those suggesting that I RTFM, I understand your frustration. Its just that the documentation is not very clear to me. It is probably a Pythonic way of documentation - but coming from a C++ background, I feel the documentation could be improved to make it more accesible for people coming from different languages.
Case in point: the documentation merely states the class name and an **options parameter in the ctor, but doesn't tell you what the possible options are.
For example class CharField(max_length=None,[**options])
There is a line further up in the documentation that gives a list of permissible options, which are applicable to all field types.
However, the options are provided in the form:
Field.optionname
The (apparently implicit) link between a class property and a constructor argument was not clear to me. It appears that if a class has a property foo, then it means that you can pass an argument named foo to its constructor. Does that observation hold true for all Python classes?
The indexes are automatically generated for your references to other models (i.e. your foreign keys). In other words: your geog_region_id is correct (but it would be better style to call it geog_region).
You can set default values using the default field option.
import datetime
class Country(models.Model):
code = models.CharField(max_length=4, unique=True)
name = models.CharField(max_length=64)
geog_region = models.ForeignKey(GeogRegion)
ccy = models.ForeignKey(Currency, unique=True)
flag_image_path = models.CharField(max_length=64, default='')
created_at = models.DateTimeField(default=datetime.now())
(I'm no expert on propel's orm)
Django always tries to imitate the "cascade on delete" behaviour, so no need to specify that somewhere. By default all fields are required, unless specified differently.
For the datetime field see some more options here. All general field options here.
code = models.CharField(max_length=4) # Erm, no index on this column .....
name = models.CharField(max_length=64) # Erm, no index on this column .....
You can pass the unique = True keyword argument and value for both of the above.
geog_region_id = models.ForeignKey(GeogRegion) # Is this correct ? (how about ref integrity constraints ?
ccy_id = models.ForeignKey(Currency) # Is this correct?
The above lines are correct if GeogRegion and Currency are defined before this model. Otherwise put quotes around the model names. For e.g. models.ForeignKey("GeogRegion"). See documentation.
flag_image_path = models.CharField(max_length=64) # How to set default on this col?
Easy. Use the default = "/foo/bar" keyword argument and value.
created_at = models.DateTimeField() # Will this default to now() ?
Not automatically. You can do default = datetime.now (remember to first from datetime import datetime). Alternately you can specify auto_now_add = True.
# Don't know how to specify indexes and unique indexes ....
Take a look at unique_together.
You'll see that the document I have linked to is the same pointed out by others. I strongly urge you to read the docs and work through the tutorial.
I'm sorry, you haven't read the docs. A simple search for index, unique or default on the field reference page reveals exactly how to set those options.
Edit after comment I don't understand what you mean about multiple lines. Python doesn't care how many lines you use within brackets - so this:
name = models.CharField(unique=True, db_index=True)
is exactly the same as this:
name = models.CharField(
unique=True,
db_index=True
)
Django doesn't support multi-column primary keys, but if you just want a multi-column unique constraint, see unique_together.
Class demo_ref_country(models.Model)
code= models.CharField(max_length=4, db_index=True, null=False)
name= models.CharField(max_length=64, db_index=True, null=False)
geog_region = models.ForeignKey(geographic_region, null=False)
ccy = models.ForeignKey(Currency_def, null=False)
flag = models.ImageField(upload_to='path to directory', null=False, default="home")
created_at = models.DateTimeField(auto_now_add=True, db_index=True)
class Meta:
unique_together = (code, geog_region, ccy)
You can set default values,, db_index paramaeter creates indexes for related fields. You can use unique=True for seperate fields, but tahat unique together will check uniqueness in columns together.
UPDATE: First of all, i advice you to read documentatin carefully, since django gives you a lot of opportunuties, some of them have some restrictions... Such as, unique_together option is used just for django admin. It means if you create a new record or edit it via admin interface, it will be used. If you will alsa insert data with other ways (like a DataModel.objects.create statement) its better you use uniaue=True in field definition like:
code= models.CharField(max_length=4, db_index=True, null=False, unique=True)
ForeignKey fields are unique as default, so you do not need to define uniqueness for them.
Django supports method override, so you can override Model save and delete methods as you like.
check it here. Django also allows you to write raw sql queries you can check it here
As i explained, unique together is a django admin feature. So dont forget to add unique=True to required fields.
Unique together also allows you to define diffrent unique pairs, such as;
unique_together = (('id','code'),('code','ccy','geog_region'))
That means, id and code must be unique together and code, ccy and geog_region must be unique together
UPDATE 2: Prior to your question update...
It is better yo start from tutorials. It defines basics with good examples.
As for doc style, let me give you an example, but if you start from tutors, it will be easier for you...
There are from model structure... Doc here
BooleanField
class BooleanField(**options)
that defines, the basic structure of a database field, () is used, and it has some parameters taken as options. that is the part:
models.BooleansField()
Since this is a field struvture, available options are defines as:
unique
Field.unique
So,
models.BooleansField(unique=True)
That is the general usage. Since uniqu is a basic option available to all field types, it classified as field.unique. There are some options available to a single field type, like symmetrical which is a ManyToMany field option, is classified as ManyToMany.Symmetrical
For the queryset
class QuerySet([model=None])
That is used as you use a function, but you use it to filter a model, with other words, write a filter query to execute... It has some methods, like filter...
filter(**kwargs)
Since this takes some kwargs, and as i told before, this is used to filter your query results, so kwargs must be your model fields (database table fields) Like:
MyModel.objects.filter(id=15)
what object is defines in the doc, but it is a manager that helps you get related objects.
Doc contains good examples, but you have to start from tutors, that is what i can advice you...
this is a model of the view table.
class QryDescChar(models.Model):
iid_id = models.IntegerField()
cid_id = models.IntegerField()
cs = models.CharField(max_length=10)
cid = models.IntegerField()
charname = models.CharField(max_length=50)
class Meta:
db_table = u'qry_desc_char'
this is the SQL i use to create the table
CREATE VIEW qry_desc_char as
SELECT
tbl_desc.iid_id,
tbl_desc.cid_id,
tbl_desc.cs,
tbl_char.cid,
tbl_char.charname
FROM tbl_desC,tbl_char
WHERE tbl_desc.cid_id = tbl_char.cid;
i dont know if i need a function in models or views or both. i want to get a list of objects from that database to display it. This might be easy but im new at Django and python so i having some problems
Django 1.1 brought in a new feature that you might find useful. You should be able to do something like:
class QryDescChar(models.Model):
iid_id = models.IntegerField()
cid_id = models.IntegerField()
cs = models.CharField(max_length=10)
cid = models.IntegerField()
charname = models.CharField(max_length=50)
class Meta:
db_table = u'qry_desc_char'
managed = False
The documentation for the managed Meta class option is here. A relevant quote:
If False, no database table creation
or deletion operations will be
performed for this model. This is
useful if the model represents an
existing table or a database view that
has been created by some other means.
This is the only difference when
managed is False. All other aspects of
model handling are exactly the same as
normal.
Once that is done, you should be able to use your model normally. To get a list of objects you'd do something like:
qry_desc_char_list = QryDescChar.objects.all()
To actually get the list into your template you might want to look at generic views, specifically the object_list view.
If your RDBMS lets you create writable views and the view you create has the exact structure than the table Django would create I guess that should work directly.
(This is an old question, but is an area that still trips people up and is still highly relevant to anyone using Django with a pre-existing, normalized schema.)
In your SELECT statement you will need to add a numeric "id" because Django expects one, even on an unmanaged model. You can use the row_number() window function to accomplish this if there isn't a guaranteed unique integer value on the row somewhere (and with views this is often the case).
In this case I'm using an ORDER BY clause with the window function, but you can do anything that's valid, and while you're at it you may as well use a clause that's useful to you in some way. Just make sure you do not try to use Django ORM dot references to relations because they look for the "id" column by default, and yours are fake.
Additionally I would consider renaming my output columns to something more meaningful if you're going to use it within an object. With those changes in place the query would look more like (of course, substitute your own terms for the "AS" clauses):
CREATE VIEW qry_desc_char as
SELECT
row_number() OVER (ORDER BY tbl_char.cid) AS id,
tbl_desc.iid_id AS iid_id,
tbl_desc.cid_id AS cid_id,
tbl_desc.cs AS a_better_name,
tbl_char.cid AS something_descriptive,
tbl_char.charname AS name
FROM tbl_desc,tbl_char
WHERE tbl_desc.cid_id = tbl_char.cid;
Once that is done, in Django your model could look like this:
class QryDescChar(models.Model):
iid_id = models.ForeignKey('WhateverIidIs', related_name='+',
db_column='iid_id', on_delete=models.DO_NOTHING)
cid_id = models.ForeignKey('WhateverCidIs', related_name='+',
db_column='cid_id', on_delete=models.DO_NOTHING)
a_better_name = models.CharField(max_length=10)
something_descriptive = models.IntegerField()
name = models.CharField(max_length=50)
class Meta:
managed = False
db_table = 'qry_desc_char'
You don't need the "_id" part on the end of the id column names, because you can declare the column name on the Django model with something more descriptive using the "db_column" argument as I did above (but here I only it to prevent Django from adding another "_id" to the end of cid_id and iid_id -- which added zero semantic value to your code). Also, note the "on_delete" argument. Django does its own thing when it comes to cascading deletes, and on an interesting data model you don't want this -- and when it comes to views you'll just get an error and an aborted transaction. Prior to Django 1.5 you have to patch it to make DO_NOTHING actually mean "do nothing" -- otherwise it will still try to (needlessly) query and collect all related objects before going through its delete cycle, and the query will fail, halting the entire operation.
Incidentally, I wrote an in-depth explanation of how to do this just the other day.
You are trying to fetch records from a view. This is not correct as a view does not map to a model, a table maps to a model.
You should use Django ORM to fetch QryDescChar objects. Please note that Django ORM will fetch them directly from the table. You can consult Django docs for extra() and select_related() methods which will allow you to fetch related data (data you want to get from the other table) in different ways.