How to properly make migrations when adding a new unique field - django

I added a new field to one of my models:
class Agency(models.Model):
email = models.EmailField(unique=True, verbose_name=_("e-mail"))
As this field cannot be blank, django-admin makemigrations requested me to provide one-off default, which I did. Here is the generated migration:
# Generated by Django 1.9.4 on 2016-03-20 10:38
from __future__ import unicode_literals
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('accounts', '0008_auto_20160226_1226'),
]
operations = [
migrations.AddField(
model_name='agency',
name='email',
field=models.EmailField(default='example#example.fr', max_length=254, unique=True, verbose_name='e-mail'),
preserve_default=False,
),
]
As expected, django-admin migrate throwed an error:
psycopg2.IntegrityError: could not create unique index "accounts_agency_email_key"
DETAIL: Key (email)=(example#example.fr) is duplicate.
I thought I could edit the migration to set unique values before making the field unique. So I tried:
# -*- coding: utf-8 -*-
# Generated by Django 1.9.4 on 2016-03-20 10:38
from __future__ import unicode_literals
from django.db import migrations, models
from django.utils.text import slugify
def set_email(apps, schema_editor):
Agency = apps.get_model('accounts', 'Agency')
for agency in Agency.objects.all():
agency.email = '{}#example.fr'.format(slugify(agency.name))
agency.save()
class Migration(migrations.Migration):
dependencies = [
('accounts', '0008_auto_20160226_1226'),
]
operations = [
migrations.AddField(
model_name='agency',
name='email',
field=models.EmailField(default='', max_length=254, blank=True, verbose_name='e-mail'),
preserve_default=False,
),
migrations.RunPython(set_email),
migrations.AlterField(
model_name='agency',
name='email',
field=models.EmailField(max_length=254, unique=True, verbose_name='e-mail'),
preserve_default=False,
),
]
Unfortunately I get this error when running django-admin migrate:
django.db.utils.OperationalError: cannot ALTER TABLE "accounts_agency" because it has pending trigger events
My guess is that operations are not executed synchronously.
I think I could fix the issue by splitting the migration into two migrations, but I'd like to know if I can do it in only one migration. What is the common way to create migrations when adding a new unique field in a model?
PS: I also tried to use an F expression as default (default=models.F('name') + '#example.fr') but it failed:
django.db.utils.IntegrityError: could not create unique index "accounts_agency_email_key"
DETAIL: Key (email)=(F(name) + Vallu(#example.fr)) is duplicated.

Maybe it's too late but maybe it could work for someone else
You can do this in one migration via using migrations.RunSQL method
For your example code after you added the new field to your model and run the python manage.py makemigrations command (here if you have existing rows in your table command wants to choice default value you can choice "Provide a one-off default now" option and give some string value it is not important because actually we did not use it) then go to migration file and change operations part with this (Note i use postgresql you can change SQL for your database)
operations = [
migrations.RunSQL(
'ALTER TABLE "agency" ADD COLUMN "email" varchar(254) NULL;ALTER TABLE "agency" ALTER COLUMN "email" DROP DEFAULT;COMMIT;',
),
migrations.RunSQL(
"UPDATE agency SET email= Concat(country_code, '#example.fr');COMMIT;",
),
migrations.RunSQL(
'ALTER TABLE "agency" ALTER COLUMN "email" SET NOT NULL;ALTER TABLE "agency" ADD CONSTRAINT "agency_email_b551ad2a_uniq" UNIQUE ("email");ALTER TABLE "agency" ALTER COLUMN "email" DROP DEFAULT;CREATE INDEX "agency_email_b551ad2a_like" ON "agency" ("email" varchar_pattern_ops);COMMIT;'
)
]
then run "python manage.py migrate" command
that is it.

Related

How to automatically fill pre-existing database entries with a UUID in Django

I have added a UUID to the following model:
class Post(models.Model):
uuid = models.UUIDField(default=uuid.uuid4, editable=False)
...
But there already are entries in the database that were created without the uuid field.
When I run migrate, it adds the same UUID to all my previous objects.
Is there an easy way of populating the existing objects with a different UUID automatically?
I think the easiest way to fix this is to make a data migration [Django-doc]. You create a new migration file with:
python manage.py makemigrations --empty yourappname
This will create a new file in the migrations/ directory that does nothing. We can alter that migration file with something that looks like:
# Generated by Django 3.2 on 2021-05-01 12:49
from django.db import migrations
from uuid import uuid4
class Migration(migrations.Migration):
def populate_uuid(apps, schema_editor):
Post = apps.get_model('yourappname', 'Post')
posts = list(Post.objects.all())
for post in posts:
post.uuid = uuid4()
Post.objects.bulk_update(posts, ['uuid'])
dependencies = [
('yourappname', 'previous_migrationname'),
]
operations = [
migrations.RunPython(populate_uuid)
]
If you then migrate, it will load all Post objects, provide these each a unique uuid, and then update the items in bulk.

Using django-custom-user in existing django-cms project

I have an existing django-cms ver. 3.1.3 project where I would like to replace the default django User model with the one found in django-custom-user (to have email as user name for my users). I have added the custom_user app to my INSTALLED_APPS and set AUTH_USER_MODEL = 'custom_user.EmailUser'. Finally I applied the migrations.
Everything seems to work fine in my custom models, but the django-cms models that have a reference to the auth_user table (GlobalPagePermission, PagePermission, PageUser and UserSettings) are not updated to have a foreign-key reference to the new custom user table.
The django-cms documentation says that is generally advisable to add custom user models in the beginning of a project, but here I am, in the middle of a project, and would very much like to avoid having to delete my cms models (with data) to make it work.
If I were in the beginning of a project, and had added the custom user model before migrating the django-cms models, would the django-cms models actually get a reference to the custom user model table instead of the default one?
Is there any way for me to make migrations for the django-cms models, so they use the new custom user model instead of the default one?
UPDATE
Im trying to implement what yakky suggested:
from __future__ import unicode_literals
from django.conf import settings
from django.db import models, migrations
import django.db.models.deletion
from django.utils.translation import ugettext_lazy as _
def up(apps, schema_editor):
PageUser = apps.get_model("cms", "PageUser")
db_alias = schema_editor.connection.alias
for pu in PageUser.objects.all():
pu['emailuser_ptr_id'] = pu['user_ptr_id']
pu.save()
def down(apps, schema_editor):
PageUser = apps.get_model("cms", "PageUser")
db_alias = schema_editor.connection.alias
for pu in PageUser.objects.all():
pu['user_ptr_id'] = pu['emailuser_ptr_id']
pu.save()
class Migration(migrations.Migration):
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
(‘myapp’, ‘previous-migration’),
]
operations = [
migrations.RenameField(
model_name='PageUser',
old_name='user_ptr_id',
new_name='user_ptr_id_old',
),
migrations.AddField(
model_name='PageUser',
name='emailuser_ptr_id',
field=models.ForeignKey(null=True, to=settings.AUTH_USER_MODEL, verbose_name=_('user'), blank=True),
preserve_default=True,
),
migrations.RunPython(up, down),
migrations.RemoveField(
model_name='PageUser',
name='user_ptr_id_old',
),
]
It fails with KeyError: ('myapp', u'pageuser') suggesting that it looks for the PageUser model in the my custom app and not in the cms app. How do I apply these migrations to the cms models?
Being said that replacing the Django User model it's not advisable in the middle of a project lifetime (see https://docs.djangoproject.com/en/1.9/topics/auth/customizing/#substituting-a-custom-user-model) as Django does not provide any way to modify the relations after their are being created, to replace the custom user model you can write a custom migration that alter the foreignkey in the django CMS models to point to theand create a new one to the new user model.
Something like:
...
migrations.RenameField(
model_name='globalpagepermission',
old_name='user',
new_name='user_old',
),
migrations.AddField(
model_name='globalpagepermission',
name='user',
field=models.ForeignKey(null=True, to=settings.AUTH_USER_MODEL, verbose_name=_('user'), blank=True),
preserve_default=True,
),
migrations.RunPython(copy_data),
migrations.RemoveField(
model_name='globalpagepermission',
name='user_old',
),
...
Repeat for any relevant model / field.
copy_data method will copy the value for the foreign keys from user_old to user
Code is completely untested but the general idea should work.
Alternatively you can write a RunSQL migration to get the same result.
Users with the same primary keys must exist also for the new model for this to work.
Thanks to https://code.djangoproject.com/ticket/25313 I managed to make it work. This is what I did:
Backed up my database!
Created a new app email_username to hold my user model (named User as recommended in the link). A new app was necessary because my main app (myapp) is dependent on django-cms, and if I put my new user model in myapp, django-cms would become dependent on my app - this would create a circular reference.
class User(AbstractEmailUser):
# Needed to add username, first_name and last_name here, because cms.PageUser model depends on these fields
username = models.CharField(max_length=100, verbose_name=_('username'))
first_name = models.CharField(max_length=100, verbose_name=_('first_name'))
last_name = models.CharField(max_length=100, verbose_name=_('last_name'))
class Meta:
db_table = 'auth_user' # the model will use the existing auth_user table (at first)
Set AUTH_USER_MODEL = ‘email_username.User’ in settings.py
Deleted all migrations of myapp and truncated the django_migrations table
Ran manage.py makemigrations and got
Cannot resolve bases for [] This can
happen if you are inheriting models from an app with migrations (e.g.
contrib.auth) in an app with no migrations; see
https://docs.djangoproject.com/en/1.8/topics/migrations/#dependencies
for more
... so I removed the cms app from settings.py, ran makemigrations again which now suceeded, then re-added and ran makemigrations again.
Ran manage.py migrate --fake, but got
Cannot resolve bases for [ModelState: 'djangocms_file.File', ModelState: 'djangocms_video.Video', ModelState: 'djangocms_link.Link', ModelState: 'djangocms_googlemap.GoogleMap', ModelState: 'djangocms_picture.Picture', ModelState: 'djangocms_teaser.Teaser']
... so I removed these apps as well from settings.py, ran migrate --fake, re-added the apps and migrate --fake again.
Removed the db_table setting from the new User model, ran makemigrations followed by migrate (without fake!)
Everything now seems to be in order. All foreign keys that previously referenced the standard auth_user table is now referincing the table of my new user model. Existing users have even been transferred because of the db_table trick.

Django models: adding a new field with unique=True and migrate will fail due to entering a same value to existing records

I am adding a new field to a model:
class Abc(models.Model):
...
slug = models.SlugField(unique=True)
During Makemigrations I provided an empty string once for all. And then Migrate failed because of duplicate key Key (slug)=() is duplicated.
The follows are what I have tried to solve the problem. makemigrations were all OK but migrate failed due to the same reason.
1) Remove unique=True and migration again
2) Remove the slug field and migration again
3) Set unique=False and migrate again
The database can not be deleted. I am stuck and left without options. Any suggestions?
You are having this problem because there are at least two records in the table that have the same slug.
To fix this, you could do the following steps
Create a data migration. ./manage.py makemigrations <app_name> --empty
In the migration file, add django migration's RunPython operation in the file. An example would be something like the snippet below.
from __future__ import unicode_literals
from django.db import models, migrations
from django.utils.text import slugify
def forwards(apps, schema_editor):
Abc = apps.get_model('app', 'Abc')
for obj in Abc.objects.all():
if len(obj.slug) == 0:
obj.slug = slugify(obj.field1)
obj.save()
class Migration(migrations.Migration):
dependencies = [
('app', '0003_Abc'),
]
operations = [
migrations.RunPython(forwards, reverse_code=migrations.RunPython.noop),
]
Add the unique=True to the field.
Create the migration. ./manage.py makemigrations <app_name>
Migrate your database. ./manage.py migrate <app_name>

Django AutoField wrong migration for "id" field

I have quite simple Django model:
class MyModel(models.Model):
user = models.ForeignKey(User)
somestring = models.CharField(
max_length=250
)
... some other string fields...
There is no declared "id" field in model, so it has automatic primary key assigned by Django.
Initial migration looked like this:
migrations.CreateModel(
name='MyModel',
fields=[
('user', models.ForeignKey(to=settings.AUTH_USER_MODEL)),
('somestring', models.CharField(max_length=250))
... some other string fields...
],
options={},
bases=(models.Model,),
),
It was successfully applied and database table contains fields like:
- id <== autogenerated
- user
- somestring
etc...
Occasionally I've started to get messages like
Your models have changes that are not yet reflected in a migration, and so won't be applied.
Run 'manage.py makemigrations' to make new migrations, and then re-run 'manage.py migrate' to apply them.
After running manage.py makemigrations it generated very strange migration:
$ ./manage.py makemigrations
You are trying to add a non-nullable field 'id' to mymodel without a default; we can't do that (the database needs something to populate existing rows).
Please select a fix:
1) Provide a one-off default now (will be set on all existing rows)
2) Quit, and let me add a default in models.py
Select an option: 1
Please enter the default value now, as valid Python
The datetime and django.utils.timezone modules are available, so you can do e.g. timezone.now()
>>>
Please enter some code, or 'exit' (with no quotes) to exit.
>>> ''
Migrations for 'myapp':
0036_auto_20150623_1535.py:
- Add field id to mymodel
It loosk like this:
class Migration(migrations.Migration):
dependencies = [
('foo', 'bar'),
]
operations = [
migrations.AddField(
model_name='mymodel',
name='id', <== !!!
field=models.AutoField(auto_created=True, primary_key=True, default='', serialize=False, verbose_name='ID'),
preserve_default=False,
),
]
But this migration doesn't makes sense and sure it fails, because there is already "id" field in appropriate DB table.
The quick and dirty solution is to --fake this migration. Which should work locally on dev machine, but can result into migration errors on other environments (test/staging/prod).
Looks like the model's old/new fields state was calculated incorrectly, so that "id" wasn't included in old model but it is included in new one, so django descided that id field should be added.
Not sure about the cause, but my main question is - What is the proper approach to reset the model or migration state and return it to the previous one, when there is no any need in such migration?
Alright, found a viable workaround for this.
Added "id" field declaration into original migration, so it is no longer reported as missing.
migrations.CreateModel(
name='MyModel',
fields=[
!!===> ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
('user', models.ForeignKey(to=settings.AUTH_USER_MODEL)),
('somestring', models.CharField(max_length=250))
... some other string fields...
],
options={},
bases=(models.Model,),
),
This change will not be applied on environments where we already have this migration and new environments should get the proper db table structure from the start.
TIL: Django DOES NOT use SQL to fetch the current model state from DB, - but create mock of the model and applies the migrations to that mock until it gets the latest version of model according all available migration operations. And then that reconstructed model is compared to real one from models.py.
Adjusting the original migration solved the issue. Thanks to everyone who participated in discussions.

django.db.utils.IntegrityError: UNIQUE constraint failed: rango_category__new.slug

I'm learning Django from Tango with Django but I keep getting this error when I type:
python manage.py makemigrations rango
python manage.py migrate
This is the output:
django.db.utils.IntegrityError: UNIQUE constraint failed: rango_category__new.slug
Models.py:
from django.db import models
from django.template.defaultfilters import slugify
class Category(models.Model):
name = models.CharField(max_length=128, unique=True)
views = models.IntegerField(default=0)
likes = models.IntegerField(default=0)
slug = models.SlugField(unique=True)
def save(self, *args, **kwargs):
self.slug = slugify(self.name)
super(Category, self).save(*args, **kwargs)
def __unicode__(self):
return self.name
class Page(models.Model):
category = models.ForeignKey(Category)
title = models.CharField(max_length=128)
url = models.URLField()
views = models.IntegerField(default=0)
def __unicode__(self):
return self.title
The reason for this constrain could be that you didn't have any field called slug in Category class when you have initially migrated it (First Migration), and after adding this field in the model, when you ran makemigrations, you have set default value to something static value(i.e None or '' etc), and which broke the unique constrain for the Category's table's slug column in which slug should be unique but it isn't because all the entry will get that default value.
To solve this, you can either drop the database and migration files and re-run makemigrations and migrate or set a unique default value like this:
slug = models.SlugField(unique=True, default=uuid.uuid1)
Edit:
According to Migrations that add unique fields, modify your migration file to overcome unique constrain. For example, modify your migration file (which added the slug field to the model) like this:
import uuid
from app.models import Category # where app == tango_app_name
class Migration(migrations.Migration):
dependencies = [
('yourproject', '0003_remove_category_slug'),
]
def gen_uuid(apps, schema_editor):
for row in Category.objects.all():
row.slug = uuid.uuid4()
row.save()
operations = [
migrations.AddField(
model_name='category',
name='slug',
field=models.SlugField(default=uuid.uuid4),
preserve_default=True,
),
migrations.RunPython(gen_uuid),
migrations.AlterField(
model_name='category',
name='slug',
field=models.SlugField(default=uuid.uuid4, unique=True),
),
]
I got a field with attribute unique, which was not unique [eg 2-time same value]
python3 manage.py migrate --fake
then
python3 manage.py makemigrations
python3 manage.py migrate
this did the trick
This means a slug should be unique. You may have some data in your model. You need to delete all the data in that model and you need to migrate again.
In this situation, you have two ways to fix the error;
You need to delete it from the Django admin site. More often than not, it may give an error when you are trying to open the model.
Open command prompt
move to project -> py manage.py shell -> from yourappname.models import modelname -> modelname.objects.delete()
Here if you define a product manager for your model. Then you have to define a delete function. Later you should makemigrate, migrate and continue with the second way
I just met this simiilar error: Django UNIQUE constraint failed. I tried examine the code for very long time, but didn't solve it. I finally used SQLiteStudio to examine the data, and found the data is problematic: I unintentionally added two SAME instances which violates the UNIQUE constraint. To be frank I haven't thought the error could be this naive and simple, and because so it took me a lot of time to find out!
I had the same problem and tried all the suggested answers. What finally worked for me was, after I defined the slug field as a URL in models, and ran the makemigrations. I edited the file in makemigrations adding a random number at the end of a basic URL, like this
Generated by Django 3.2.3 on 2022-02-02 20:58
from django.db import migrations, models
from random import randint
class Migration(migrations.Migration):
dependencies = [
('blog', '0002_remove_post_slug1'),
]
operations = [
migrations.AddField(
model_name='post',
name='slug',
field=models.URLField(blank=True, default='http:/salpimientapa.com/' + str(randint(100000,999999))),
),
]
After I ran
python manage.py migrate
I edit the slug as a SlugModel and ran the makemigrations and migrate again
What worked for me was going to the admin and changing the value of duplicate slug, before running the migrations again.
Just delete the last migration in the migration folder
Then run
python manage.py makemigrations
python manage.py migrate
I faced the same issue and solved by populating my slugfied thro' the admin with unique values and without leaving any of them blank.
Basically: You add the field without unique=true in one operation, make a data migration that generates the correct shortuuids for you, and then change the field too unique again.
i have this error too ,
i did delete my database in djangoProject ( for example db.sqlite3 )
and then run
python manage.py makemigrations
python manage.py migrate
It's an Integrity Error probably because the migration will temper with the already exiting data in the database.
I had this error and here's what I did:
Enter in the project folder directory
Open the python interpreter
py manage.py shell
Import your Models
from yourappname.models import model
Delete existing data records in the model
model.objects.all().delete()
Exit the Python Interpreter
exit()
.
Another thing you could do is to set unique="false" on the affecting field. I think this should work; not so sure.