Renaming Django model without breaking existing migrations - django

I want to rename a model in Django 3.2, keep my existing migrations and be able to both migrate a db with the old table name and create a db from scratch.
I've started by renaming the model class and all references to it in the code. As "./manage.py makemigrations" did not automatically create a migration, I manually created a migration that renames the model:
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('market_integrations', '0003_migration'),
]
operations = [
migrations.RenameModel('OldModelName', 'NewModelname')
]
My initial idea is that I should not update existing migrations, as I never do when creating other migrations. However, the references to the old model in the old migrations cause a LookupError when I run "./manage.py migrate". I've tried using both the model name string and apps.get_model(). Migration code samples that break:
operations = [
migrations.CreateModel(
name="OldModelName",
...
)
]
operations = [
migrations.CreateModel(
name=apps.get_model("myapp", "OldModelName"),
...
)
]
As keeping the old model name in old migrations didn't work, I replaced the old model name in old migrations with the new name. After doing that, "./manage.py migrate" ran successfully, including the model renaming migration. However, when I try to create a new database, the model renaming migration fails because that new database never had a table with the old name.
What should I do to be able to preserve my migrations and have them working in both existing and new databases?
I have already checked Django migration strategy for renaming a model and relationship fields , but I didn't find the answer to my question there.

Please attach the full model (you can delete irrelevant fields) and the migration you created in your initial question (I cannot comment on your initial post yet, so I have to mention it in an answer).
This is important because I think there are foreign keys pointing to the model you want to rename.
Or are you trying to rename Foo to Bar and create a new Foo?
Does the error also occur if you try to start the server after your initial idea?
Your initial idea of keeping existing migrations untouched seems good, because you will be able to run it on all your environments (local, dev, prod, etc), and approach 2 would require manual renaming of database tables and fields.
I might be able to extend my answer after you provide models and migrations.
migrations.RenameModel(
old_name="ModelA",
new_name="ModelB",
),
migrations.AlterField(
model_name="myrelatedmodel",
name="somefieldname",
field=models.ForeignKey(
# change the properties accordingly (best copy it over). The database may only change to="table"
null=True,
on_delete=django.db.models.deletion.SET_NULL,
to="myapp.modelb",
),
),
If this did not work, where does the error come from? You can try to isolate your migrations and models from the rest of the code (like in a new empty project attached to the same database). Maybe this will help you find out if the error is from a leftover reference to OldModelName or some weird error. Just as an idea so you don't spend time looking at the wrong place
Try to do one change and one migration at a time. You can later squash them. If django does not auto-detect the renaming of your model, it means that all fields related to this model will fail too.
Revert everything to the working state before renaming
Make sure all migrations are applied
Rename the model and create a migration to only rename the model. Are you able to apply it?
if not, try to manually modify all related fields (one2one, foreignkey, etc)
If that still fails, create a copy of your model in models.py, rename the copied model and try to use migrations.RunPython() in a new migration file to copy all the data from the old table to the new one.
If you only have one populated database, you might get away with replacing all ModelA occurrences with ModelB and change the structure of your database.
Best of luck!

Related

Django Migrations: How to mark a migration as a parent of other apps migrations?

I'm trying to create a new Django application whose migrations depend on other apps migrations. I want migrations on the other apps to have a dependency on migrations on the new application, even when the new application doesn't reference at all the models at the other apps.
For example, let's say I have application A (an existing app) and application B (my new application):
A has a migration called A.0001, and B has a migration called B.0001 which depends on A.0001. I now make a change at A.MyModel, so I need to run python manage.py makemigrations to generate a new migration A.0002.
What I want is A.0002 to automatically depend on B.0001.
How can I specify this dependency on new migrations in A without having to do it by hand each time I modify a model in A?
I tried to add empty modifications of models in A at the migration B.0001, but I haven't got it to work and it looks very hacky to me.
I found a solution that doesn't exactly solve what I asked, but can be a workaround for some people, so I will post it here.
There is an undocumented field of the django.Migration class, called run_before, that works just like an inverse dependencies.
A.0002 will depend on B.0001 if I simply define run_before = [ A.0002 ] at B.0001.

How can I remove a Django migration file?

I made a migration, and realised that I made an error (by adding a default value), I then made a new migration which allows for null.
I don't want my colleagues to run the first migration which adds a default value to thousands of records. How can I delete that migration without breaking the current migrations (usually if you just delete a migration you get a heap of errors which are a pain to fix).
I'd assume you could use a command? I'd assume it'd be something like this ~>
e.g django manage.py deletemigration <migration_id>
Squash
You can do a ./manage.py squashmigrations since one of your migrations are effectively cancelling out another the end result will be the field being nullable. Your colleagues will not have to go through the step of adding a default value.
Squashing is the act of reducing an existing set of many migrations
down to one (or sometimes a few) migrations which still represent the
same changes.
Edit the migration file
You can edit the migration file by hand to remove the changes to the column. A migration can actually have an empty migration
class Migration(migrations.Migration):
dependencies = [
(some stuff here),
]
operations = []

How to unittest a django database migration?

We've changed our database, using django migrations (django v1.7+).
The data that exists in the database is no longer valid.
Basically I want to test a migration by, inside a unittest, constructing the pre-migration database, adding some data, applying the migration, then confirming everything went smoothly.
How does one:
hold back the new migration when loading the unittest
I found some stuff about overriding settings.MIGRATION_MODULES but couldn't work out how to use it. When I inspect executor.loader.applied_migrations it still lists everything. The only way I could prevent the new migration was to actually remove the file; not a solution I can use.
create a record in the unittest database (using the old model)
If we can prevent the migration then this should be pretty straightforward. myModel.object.create(...)
apply the migration
I think I can probably work this out now that I've found the test_executor: set a plan pointing to the migration file and execute it? Um, right? Got any code for that :-D
confirm the old data in the database now matches the new model
Again, I expect this should be pretty easy: just fetch the instance created before the migration and confirm it has changed in all the right ways.
So the challenge is really just working out how to prevent the unittest from applying the latest migration script and then applying it when we're ready?
Perhaps I have the wrong approach? Should I create fixtures, and just confirm that they're all good at the end? Do fixtures get loaded before the migrations are applied, or after they're all done?
By using the MigrationExecutor and picking out specific migrations with .migrate I've been able to, maybe?, roll it back to a specific state, then roll forward one-by-one. But that is popping up doubts; currently chasing down sqlite fudging around due to the lack of an actual ALTER TABLE instruction. Jury still out.
I wasn't able to prevent the unittest from starting with the current database schema, but I did find it is quite easy to revert to earlier points in the migration history:
Where "0014_nulls_permitted" is a file in the migrations directory...
from django.db.migrations.executor import MigrationExecutor
executor.migrate([("workflow_engine", "0014_nulls_permitted")])
executor.loader.build_graph()
NB: running the executor.loader.build_graph between invocations of executor.migrate seems to be a very important part of completing the migration and making things behave as one might expect
The migrations which are currently applicable to the database can be checked with something like:
print [x[1] for x in sorted(executor.loader.applied_migrations)]
[u'0001_initial', u'0002_fix_foreignkeys', ... u'0014_nulls_permitted']
I created a model instance via the ORM then ensured the database was in the old state by running some SQL directly:
job = Job.objects.create(....)
from django.db import connection
cursor = connection.cursor()
cursor.execute('UPDATE workflow_engine_job SET next_job_state=NULL')
Great. Now I know I have a database in the old state, and can test the forwards migration. So where 0016_nulls_banished is a migration file:
executor.migrate([("workflow_engine", "0016_nulls_banished")])
executor.loader.build_graph()
Migration 0015 goes through the database converting all the NULL fields to a default value. Migration 0016 alters the schema. You can scatter some print statements around to confirm things are happening as you think they should be.
And now the test can confirm that the migration has worked. In this case by ensuring there are no nulls left in the database.
jobs = Job.objects.all()
self.assertTrue(all([j.next_job_state is not None for j in jobs]))
We have used the following code in settings_test.py to ignore the migration for the tests:
MIGRATION_MODULES = dict(
(app.split('.')[-1], '.'.join([app, 'nonexistent_django_migrations_module']))
for app in INSTALLED_APPS
)
The idea here being that none of the apps have a nonexistent_django_migrations_module folder, and thus django will simply find no migrations.

Issue with Django data migration when model changes later

Let's say I have a Django model that looks like this:
class MyEntity(models.Model):
my_first_attribute = models.CharField(max_length=50)
I create a data migration that inserts some values in MyEntity model.
Then I create a schema migration that adds another attribute "my_second_attribute".
If I run ./migrate.py on a fresh database, running the data migration fails and Django complains that myentity.my_second_attribute doesn't exist!
Is there a solution for this?
Be sure that your data migration function does this:
Person = apps.get_model("yourappname", "Person")
Rather than using this:
from yourappname.models import Person
The former will assume database schema and models as defined by previous schema migration(s) (as specified in Migration.dependencies).
The latter will use current sources which may go way ahead of the state known to migration files.

Django migrations reference a deleted module

I have a Model named FooModel defined in a my_app/models/foo.py.
After deleting foo.py, running Django (1.7) migrations raises an error since the old migration files import foo.py (import myapp.models.foo.FooModel).
How should I resolve this?
This happens when the model has an ImageField with an upload_to parameter.
There are two cases:
You moved FooModel elsewhere, then edit all your migration files to reflect that move.
You removed FooModel, in this case, follow these steps:
Put FooModel back to where it was.
Make sure there are no references to FooModel elsewhere in your code.
Run ./manage.py makemigrations my_app
Run ./manage.py squashmigrations my_app <migration created with the previous comand> — see the doc for more informations on squashing migrations.
Repeat the two previous steps for any app that references FooModel in its migrations.
Remove FooModel and the stale migration files once you ensured everything worked fine.
This should work because as FooModel is not referenced from any other model, it should be removed from the migrations files when squashing them.
However, be warned that squashing migrations is not a simple operation and may have consequences it might be a better idea to just keep the model in your codebase without using it.
Note: in this situation, the object in question is a Django model but this applies to any class, function or module referenced by migration files.
In custom database migrations you should not import your models directly because you can face this particular issue in the future. Instead you should rather use Djangos get_model function.
MyModel = apps.get_model('myapp', 'MyModel')
for row in MyModel.objects.all():
row.uuid = uuid.uuid4()
row.save(update_fields=['uuid'])
In this case the migrations will also run when you decide to delete the model in the future.
Further read: https://docs.djangoproject.com/en/2.2/howto/writing-migrations/