Using migrations, I need to add a new field (a foreign key) to a model. I know it can be done with:
migrations.AddField(
model_name='MyModel',
name='state',
field=models.ForeignKey(null=True, related_name='mymodel_state', to='msqa_common.MyModelState'),
),
However, I don't want my field to be nullable. Instead, I want to use a default value for it, corresponding to the id of MyModelState whose name is "available" (id value might change in different machines). This "available" value of table MyModelState is inserted into the database in a previous migration script, so it does exist.
I guess I should do something like:
migrations.AddField(
model_name='MyModel',
name='state',
field=models.ForeignKey(null=False, default=available_state_id, related_name='mymodel_state', to='msqa_common.MyModelState'),
),
My question: How can I get the available_state_id within my migration script?
You can't do it directly. The recommended way of doing this is to create a migration to add it with null=True, then add a data migration that uses either Python or SQL to update all the existing ones to point to available_state_id, then a third migration that changes it to null=False.
I just had the same issue and stumbled upon this answer, so here is how I did it:
operations = [
# We are forced to create the field as non-nullable before
# assigning each Car to a Brand
migrations.AddField(
model_name="car",
name="brand",
field=models.ForeignKey(
null=True,
on_delete=django.db.models.deletion.PROTECT,
to="model.Brand",
),
),
# assign_car_to_brand loops over all my Car objects and sets their
# "brand" field
migrations.RunPython(add_category_to_tags, do_nothing),
# Make the field non-nullable to force all future Car to have a Brand
migrations.AlterField(
model_name="car",
name="brand",
field=models.ForeignKey(
null=False,
on_delete=django.db.models.deletion.PROTECT,
to="model.Brand",
),
preserve_default=False
),
]
Here is a relatively complete example:
Step One
python manage.py makemigrations, set the temporary default value to None
Step Two
Change the genrated migration code to below style
import django.db.models.deletion
from django.conf import settings
from django.db import migrations, models
def set_default_author_to_blog(apps, schema_editor):
User = apps.get_model("auth", "User")
Blog = apps.get_model("blog", "Blog")
Blog.objects.update(author=User.objects.first())
def revert_set_default_autor_to_blog(apps, schema_editor):
Blog = apps.get_model("blog", "Blog")
Blog.objects.update(author=None)
class Migration(migrations.Migration):
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
('blog', '0001_auto_20220425_1017'),
]
operations = [
migrations.AddField(
model_name='blog',
name='author',
field=models.ForeignKey(null=True, db_constraint=False, on_delete=django.db.models.deletion.PROTECT,
to='auth.user', verbose_name='Author')
),
migrations.RunPython(set_default_author_to_blog, reverse_code=revert_set_default_autor_to_blog),
migrations.AlterField(
model_name='blog',
name='author',
field=models.ForeignKey(db_constraint=False, on_delete=django.db.models.deletion.PROTECT,
to='auth.user', verbose_name='Author')
),
]
Step Three
python manage.py migrate
Related
I need to change a field from
my_boolean = models.BooleanField(verbose_name="Safe Visiting Space", default=False)
to
my_boolean = models.NullBooleanField(verbose_name="Safe Visiting Space", default=None, blank=True, null=True)
So I've made the above change within the model and run makemigrations to create
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('my_app', '0164_auto_20201027_0820'),
]
operations = [
migrations.AlterField(
model_name='mymodel',
name='my_boolean',
field=models.NullBooleanField(default=None, verbose_name='Safe Visiting Space'),
),
]
But this will not set the default of the current 70k records to None but will leave them as False, so I amended the migrations file to
from django.db import migrations, models
from my_app.models import MyModel
def set_my_boolean_default(apps, schema_editor):
objects= MyModel.objects.active().filter(my_boolean=False)
for object in objectss:
object.my_boolean = None
object.save()
class Migration(migrations.Migration):
dependencies = [
('providers', '0164_auto_20201027_0820'),
]
operations = [
migrations.AlterField(
model_name='organisation',
name='infection_control_safe_visiting_space',
field=models.NullBooleanField(default=None, verbose_name='Safe Visiting Space'),
),
migrations.RunPython(set_my_boolean_default),
]
This will take hours to run. Also, a random check of the database and it doesn't seem to be updating any of the records.
What is the right / better way to do this?
It's taking you so long because of this for loop:
objects= MyModel.objects.active().filter(my_boolean=False)
for object in objects:
object.my_boolean = None
object.save() # Database roundtrip
Namely, for each object, you are hitting the database. The better way is to bulk_update the fields all at once:
objects= MyModel.objects.active().filter(my_boolean=False).update(my_boolean=None)
You can perform the query above inside your shell. It's better not to include it in your migration files for it may be executed each time you run migrations.
Enter you Django shell by python manage.py shell, import your model, and execute the query above.
class M_Post(models.Model):
''''
CODE
''''
class M_File(models.Model):
....
CODE
....
class M_Post_File(models.Model):
post = models.ForeignKey(M_Post,on_delete=models.CASCADE)
file = models.ForeignKey(M_File,on_delete=models.CASCADE,null=True)
error:
django.db.utils.NotSupportedError: Renaming the 'posts_file' table while in a transaction is not supported on SQLite because it would break referential integrity. Try adding atomic = False to the Migration class.
how to solve this error
Go to related migration file(automatically created in migrations directory after makemigrations command) and add atomic = False to the Migration class. Migration(migrations.Migration):. Then you can migrate the changes.
example code:
# Generated by Django 2.1.14 on 2019-12-02 07:07
from django.db import migrations, models
class Migration(migrations.Migration):
atomic = False # **<<< HERE**
initial = True
dependencies = [
]
operations = [
migrations.CreateModel(
name='ebayLog',
fields=[
If you still have the problem here is a example:
# Generated by Django 2.1 on 2018-08-16 21:22
from django.db import migrations
class Migration(migrations.Migration):
atomic = False # <<<< THIS LINE
dependencies = [
('shop', '0004_product_imgfeat'),
]
operations = [
migrations.RenameModel(
old_name='Category',
new_name='CategoryShop',
),
]
I migrated many times after I got this error.
Then I did what Selim said above and I also added atomic = False after class Migration(migrations.Migration): in every migration file, which was a little silly because I didn't know which file was THE related migration file...
Then I searched the "atomic=False" in Django documentation and I found these:1
2
As the error "Renaming the 'posts_file' table while in a transaction is not supported on SQLite" described we know that renaming while in a transaction is not supported on SQLite, so adding atomic=False is needed. But I don't know about DDL transactions so that's too much for me...
If you don't want to touch anything and you've just started your project (well not necessarily), you can just delete the migration files inside the migrations directory and migrate again.
Otherwise, change the atomic variable in the migrations file to False then you can migrate your changes.
Another way, if atomic=false method didn't work, is you can delete generated files in the migration folder and start again by make migrations and migrate
Iam also getting the same error at categories_coupon model.
Go to app which contains categories_coupon model.
Rename the error raising model. (i renamed the 'coupon' model to 'coupons')
open the migrations folder.
and add 'atomic=False' line to all migrations files whose names related to post_file.
then run makemigrations command.
then run migrate command.
then runserver.
For reference look at my models
0003_coupons.py
from django.conf import settings
from django.db import migrations, models
class Migration(migrations.Migration):
atomic = False #this is the line i added.
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
('categories', '0002_auto_20211008_1305'),
]
operations = [
migrations.CreateModel(
name='Coupons',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('title', models.CharField(blank=True, max_length=200, null=True)),
('code', models.CharField(blank=True, max_length=200, null=True)),
('description', models.TextField()),
('coupon_type', models.IntegerField(choices=[('%', '%'), ('RS', 'RS')])),
('discount', models.DecimalField(decimal_places=2, default=0, max_digits=20)),
('max_users', models.IntegerField(default=0)),
('used_count', models.IntegerField(default=0)),
('startd_date', models.DateField()),
('end_date', models.DateField()),
('icon', models.FileField(upload_to='uploads/banners')),
('is_active', models.IntegerField(choices=[(1, 'Active'), (2, 'Inactive'), (3, 'Deleted')])),
('created_date', models.DateTimeField(auto_now_add=True)),
('users', models.ManyToManyField(to=settings.AUTH_USER_MODEL)),
],
),
]
0004_rename_coupons_coupon.py
-----------------------------------
from django.conf import settings
from django.db import migrations
class Migration(migrations.Migration):
atomic=False #here is the line i added.
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
('categories', '0003_coupons'),
]
operations = [
migrations.RenameModel(
old_name='Coupons',
new_name='Coupon',
),
]
I have a Django model that has a foreign key to another model:
class Example(models.Model)
something = models.ForeignKey(SomeModel, db_index=True)
I want to keep the underlying DB column as a field, but to get rid of the foreign key constraint in the database.
So the model will change to:
class Example(models.Model):
something_id = models.IntegerField()
And, to be clear, something_id is the column that Django had created for the foreign key field.
I do not want to drop the column and re-create it (this is what Django does when I auto-generate migrations after changing the model as above).
I want to keep the field but I want to remove the foreign key constraint in the database with a migration. It's not clear to me how to do this with a Django migration - is there some built in support for it or do I have to run some raw SQL and, if so, how do I programatically get the name of the constraint?
This is how I managed to do it, it's based on nimasmi's answer above:
class Migration(migrations.Migration):
dependencies = [
('my_app', '0001_initial'),
]
# These *WILL* impact the database!
database_operations = [
migrations.AlterField(
model_name='Example',
name='something',
field=models.ForeignKey('Something', db_constraint=False, db_index=True, null=False)
),
]
# These *WON'T* impact the database, they update Django state *ONLY*!
state_operations = [
migrations.AlterField(
model_name='Example',
name='something',
field=models.IntegerField(db_index=True, null=False)
),
migrations.RenameField(
model_name='Example',
old_name='something',
new_name='something_id'
),
]
operations = [
migrations.SeparateDatabaseAndState(
database_operations=database_operations,
state_operations=state_operations
)
]
See SeparateDatabaseAndState. It allows you to specify a Django (state) part of the migration separately from the database
part of the migration.
Amend the field in your models file.
Create the migration, as normal. You will end up with something like:
class Migration(migrations.Migration):
dependencies = [
('my_app', '0001_whatever.py'),
]
operations = [
migrations.AlterField(
model_name='example',
name='something',
field=models.CharField(max_length=255, null=True)),
),
]
Now manually amend this to:
class Migration(migrations.Migration):
dependencies = [
('my_app', '0001_whatever.py'),
]
state_operations = [
migrations.AlterField(
model_name='example',
name='something',
field=models.CharField(max_length=255, null=True)),
),
]
operations = [
migrations.SeparateDatabaseAndState(state_operations=state_operations)
]
Note that you are not specifying any database_operations argument, so the Django relationships are amended, but the database data is unchanged.
Needless to say: take a backup before you try this.
As of Django 2.0, changing your field to models.ForeignKey(db_constraint=False, db_index=False, ...) will generate a migration that does ALTER TABLE DROP CONSTRAINT and DROP INDEX IF EXISTS, which appears to be exactly what you want.
I am trying to add an order to a ManyToMany field that I created a while ago. I basically want to order pictures in collections of pictures. I am running on Django 1.7, so no more South migrations (I was trying to follow this tutorial: http://mounirmesselmeni.github.io/2013/07/28/migrate-django-manytomany-field-to-manytomany-through-with-south/)
Here's the "through" relationship that I have:
class CollectionPictures(models.Model):
picture = models.ForeignKey(
Picture,
verbose_name=u'Picture',
help_text=u'Picture is included in this collection.',
)
collection = models.ForeignKey(
Collection,
verbose_name=u'Collection',
help_text=u'Picture is included in this collection',
)
order = models.IntegerField(
verbose_name=u'Order',
help_text=u'What order to display this picture within the collection.',
max_length=255
)
class Meta:
verbose_name = u"Collection Picture"
verbose_name_plural = u"Collection Pictures"
ordering = ['order', ]
def __unicode__(self):
return self.picture.name + " is displayed in " + self.collection.name + (
" in position %d" % self.order)
class Collection(models.Model):
pictures = models.ManyToManyField(Picture, through='CollectionPictures', null=True)
[... Bunch of irrelevant stuff after]
So this should work if I didn't have to migrate my old data (the only difference in the model is that it didn't have the through='CollectionPictures'
Here's my migration :
class Migration(migrations.Migration):
dependencies = [
('artist', '0002_auto_20141013_1451'),
('business', '0001_initial'),
]
operations = [
migrations.CreateModel(
name='CollectionPictures',
fields=[
('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
('order', models.IntegerField(help_text='What order to display this picture within the collection.', max_length=255, verbose_name='Order')),
('collection', models.ForeignKey(verbose_name='Collection', to='business.Collection', help_text='Picture is included in this collection')),
('picture', models.ForeignKey(verbose_name='Picture', to='artist.Picture', help_text='Picture is included in this collection.')),
],
options={
'ordering': ['order'],
'verbose_name': 'Collection Picture',
'verbose_name_plural': 'Collection Pictures',
},
bases=(models.Model,),
),
migrations.AlterField(
model_name='collection',
name='pictures',
field=models.ManyToManyField(to=b'artist.Picture', null=True, through='business.CollectionPictures'),
),
]
This throws an error when migrating:
ValueError: Cannot alter field business.Collection.pictures into
business.Collection.pictures - they are not compatible types (you
cannot alter to or from M2M fields, or add or remove through= on M2M
fields)
Has anybody already tried that kind of manipulation with the new 1.7 migrations?
Thanks !
The safest approach would be to create a new field and copy the data over.
Leave pictures alone and add pictures2 with your through field. Run makemigrations.
Edit the generated migration file and add a RunPython command where you copy data from the old table to the new table. Perhaps you can programmatically choose a good value for the new order column as well.
Delete the old pictures field. Run makemgirations.
Rename pictures2 to pictures. Run makemigrations.
This approach should leave you in the state you want with your data intact.
If copying over the data is a big problem you could try something else, like adding the order column in SQL, using the db_table option on CollectionPictures to make it point to the existing table, and then wiping out migrations and redoing with --fake. But that seems riskier than the approach above.
Old question, but I had this problem too, and I've found a way with Django 1.11 that works, and should work with older versions too. The needed class exists back to 1.7 and still exists in 2.0
The fix involves manually changing the migration to do what we want, using the SeparateDatabaseAndState migration class. This class lets Django update the state, but gives us control over what operations to perform. In this case we just want to rename the model table, everything else is already set up right.
The steps:
Create your new ManyToMany Through model, but specify a custom table name, and no extra fields:
class CollectionPictures(models.Model):
collection = ...
picture = ...
class Meta:
# Change myapp to match.
db_table = "myapp_collection_pictures"
unique_together = (("collection", "picture"))
Taking the existing migration, and take the operations it generates and wrap it all in a single new SeparateDatabaseAndState:
class Migration(migrations.Migration):
dependencies = [
('artist', '0002_auto_20141013_1451'),
('business', '0001_initial'),
]
operations = [
migrations.SeparateDatabaseAndState(
database_operations=[
],
state_operations=[
migrations.CreateModel(
name='CollectionPictures',
fields=[
('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
('order', models.IntegerField(help_text='What order to display this picture within the collection.', max_length=255, verbose_name='Order')),
('collection', models.ForeignKey(verbose_name='Collection', to='business.Collection', help_text='Picture is included in this collection')),
('picture', models.ForeignKey(verbose_name='Picture', to='artist.Picture', help_text='Picture is included in this collection.')),
],
options={
'ordering': ['order'],
'verbose_name': 'Collection Picture',
'verbose_name_plural': 'Collection Pictures',
},
bases=(models.Model,),
),
migrations.AlterField(
model_name='collection',
name='pictures',
field=models.ManyToManyField(to=b'artist.Picture', null=True, through='business.CollectionPictures'),
),
]
)
Remove the db_table from the class Meta, and add this operation after the SeparateDatabaseAndState, (not into the database_operations.):
migrations.AlterModelTable(
name='collectionpicture',
table=None,
),
Now if you run `./mange.py sqlmigrate myapp 0003 (pick the right number prefix!) you should with any luck see something like this as output:
BEGIN;
--
-- Custom state/database change combination
--
--
-- Rename table for collection[Pictures to None
--
ALTER TABLE "myapp_collection_pictures" RENAME TO "myapp_collectionpictures";
COMMIT;
Add your new columns ("order" in this case) and create a new migration. It's probably possible to do this at the same time, but I decided it was easier to do in two migrations.
(Step 3 isn't strictly required if you are happy keeping the custom table name there.)
And double check with ./manage.py makemigrations --check -- it should print "No changes detected".
I am trying to forward migrate a model with existing data. The model has a new field with constraints unique=True and null=False.
When I do
./manage.py schemamigration myapp --auto
South lets me specify a default value for the new field by asking:
Specify a one-off value to use for existing columns now
Usually I set this to None but since this field needs to be unique I was wondering if it is possible to pass South a unique value via:
>>> import uuid; uuid.uuid1().hex[0:35]
This gives me an error message
! Invalid input: invalid syntax
Any ideas if it is possible to pass South random unique default values when migrating via the commandline?
Thanks.
Unfortunately only the datetime module is available for use as a one-off value in a schemamigration.
However, you can achieve the same effect by splitting this up into three migrations:
add new field to the model without constraints (with null=True, unique=False)
use a datamigration to add the UUID to the new field
add the constraint on the new field (with null=False, unique=True)
Tutorial on data migrations: http://south.readthedocs.org/en/0.7.6/tutorial/part3.html#data-migrations
In django 1.7+ you can do the following. It first adds the field with no indexing and no unique. It then assigns the unique values (I based them on the name and used slugify method which you need to create) and finally alters the field again to add index and unique attributes.
from django.db import migrations
import re
import django.contrib.postgres.fields
from common.utils import slugify
import django.core.validators
def set_slugs(apps, schema_editor):
categories = apps.get_model("myapp", "Category").objects.all()
for category in categories:
category.slug = slugify(category.name)
category.save()
class Migration(migrations.Migration):
dependencies = [
('myapp', '0034_auto_20150906_1936'),
]
operations = [
migrations.AddField(
model_name='category',
name='slug',
field=models.CharField(max_length=30, validators=[django.core.validators.MinLengthValidator(2), django.core.validators.RegexValidator(re.compile('^[0-9a-z-]+$'), 'Enter a valid slug.', 'invalid')], help_text='Required. 2 to 30 characters and can only contain a-z, 0-9, and the dash (-)', unique=False, db_index=False, null=True),
preserve_default=False,
),
migrations.RunPython(set_slugs),
migrations.AlterField(
model_name='category',
name='slug',
field=models.CharField(help_text='Required. 2 to 30 characters and can only contain a-z, 0-9, and the dash (-)', unique=True, max_length=30, db_index=True, validators=[django.core.validators.MinLengthValidator(2), django.core.validators.RegexValidator(re.compile('^[0-9a-z-]+$'), 'Enter a valid slug.', 'invalid')]),
),
]
Here is the Django's official how-to on migrating unique fields.
Migrations that add unique fields
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Applying a "plain" migration that adds a unique non-nullable field to a table
with existing rows will raise an error because the value used to populate
existing rows is generated only once, thus breaking the unique constraint.
Therefore, the following steps should be taken. In this example, we'll add a
non-nullable :class:`~django.db.models.UUIDField` with a default value. Modify
the respective field according to your needs.
* Add the field on your model with ``default=...`` and ``unique=True``
arguments. In the example, we use ``uuid.uuid4`` for the default.
* Run the :djadmin:`makemigrations` command.
* Edit the created migration file.
The generated migration class should look similar to this::
class Migration(migrations.Migration):
dependencies = [
('myapp', '0003_auto_20150129_1705'),
]
operations = [
migrations.AddField(
model_name='mymodel',
name='uuid',
field=models.UUIDField(max_length=32, unique=True, default=uuid.uuid4),
),
]
You will need to make three changes:
* Add a second :class:`~django.db.migrations.operations.AddField` operation
copied from the generated one and change it to
:class:`~django.db.migrations.operations.AlterField`.
* On the first operation (``AddField``), change ``unique=True`` to
``null=True`` -- this will create the intermediary null field.
* Between the two operations, add a
:class:`~django.db.migrations.operations.RunPython` or
:class:`~django.db.migrations.operations.RunSQL` operation to generate a
unique value (UUID in the example) for each existing row.
The resulting migration should look similar to this::
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import migrations, models
import uuid
def gen_uuid(apps, schema_editor):
MyModel = apps.get_model('myapp', 'MyModel')
for row in MyModel.objects.all():
row.uuid = uuid.uuid4()
row.save()
class Migration(migrations.Migration):
dependencies = [
('myapp', '0003_auto_20150129_1705'),
]
operations = [
migrations.AddField(
model_name='mymodel',
name='uuid',
field=models.UUIDField(default=uuid.uuid4, null=True),
),
# omit reverse_code=... if you don't want the migration to be reversible.
migrations.RunPython(gen_uuid, reverse_code=migrations.RunPython.noop),
migrations.AlterField(
model_name='mymodel',
name='uuid',
field=models.UUIDField(default=uuid.uuid4, unique=True),
),
]
* Now you can apply the migration as usual with the :djadmin:`migrate` command.
Note there is a race condition if you allow objects to be created while this
migration is running. Objects created after the ``AddField`` and before
``RunPython`` will have their original ``uuid``’s overwritten.
You can manually edit your migration file:
I needed to add random character to some field so I have imported random and randint
import random
import string
and changed the value of default to
default=random.choice(string.lowercase)
It worked.
There is way to do unique value for each row with South.
Define slug in models.py as:
class Foo(models.Model):
slug = models.SlugField(unique=True, default='')
....
Create new migration
run python manage.py schemamigration --auto foo
Open new migration file, and edit it:
# Change add_column to this:
db.add_column(u'account_funnel', 'slug',
self.gf('django.db.models.foo.Foo')(default='',
unique=False,
max_length=50),
keep_default=False)
# right above this add such python code:
foos = orm['foo.Foo'].objects.all()
for foo in foos:
foo.slug = slugify(funnel.name)
foo.save()
# Modify slug as unique field
db.create_unique(u'foo_foo', ['slug'])
ps mark this migration as no_dry_run = True
pss do not forget to import slugify function from django.template.defaultfilters import slugify