I made a migration, and realised that I made an error (by adding a default value), I then made a new migration which allows for null.
I don't want my colleagues to run the first migration which adds a default value to thousands of records. How can I delete that migration without breaking the current migrations (usually if you just delete a migration you get a heap of errors which are a pain to fix).
I'd assume you could use a command? I'd assume it'd be something like this ~>
e.g django manage.py deletemigration <migration_id>
Squash
You can do a ./manage.py squashmigrations since one of your migrations are effectively cancelling out another the end result will be the field being nullable. Your colleagues will not have to go through the step of adding a default value.
Squashing is the act of reducing an existing set of many migrations
down to one (or sometimes a few) migrations which still represent the
same changes.
Edit the migration file
You can edit the migration file by hand to remove the changes to the column. A migration can actually have an empty migration
class Migration(migrations.Migration):
dependencies = [
(some stuff here),
]
operations = []
Related
I want to rename a model in Django 3.2, keep my existing migrations and be able to both migrate a db with the old table name and create a db from scratch.
I've started by renaming the model class and all references to it in the code. As "./manage.py makemigrations" did not automatically create a migration, I manually created a migration that renames the model:
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('market_integrations', '0003_migration'),
]
operations = [
migrations.RenameModel('OldModelName', 'NewModelname')
]
My initial idea is that I should not update existing migrations, as I never do when creating other migrations. However, the references to the old model in the old migrations cause a LookupError when I run "./manage.py migrate". I've tried using both the model name string and apps.get_model(). Migration code samples that break:
operations = [
migrations.CreateModel(
name="OldModelName",
...
)
]
operations = [
migrations.CreateModel(
name=apps.get_model("myapp", "OldModelName"),
...
)
]
As keeping the old model name in old migrations didn't work, I replaced the old model name in old migrations with the new name. After doing that, "./manage.py migrate" ran successfully, including the model renaming migration. However, when I try to create a new database, the model renaming migration fails because that new database never had a table with the old name.
What should I do to be able to preserve my migrations and have them working in both existing and new databases?
I have already checked Django migration strategy for renaming a model and relationship fields , but I didn't find the answer to my question there.
Please attach the full model (you can delete irrelevant fields) and the migration you created in your initial question (I cannot comment on your initial post yet, so I have to mention it in an answer).
This is important because I think there are foreign keys pointing to the model you want to rename.
Or are you trying to rename Foo to Bar and create a new Foo?
Does the error also occur if you try to start the server after your initial idea?
Your initial idea of keeping existing migrations untouched seems good, because you will be able to run it on all your environments (local, dev, prod, etc), and approach 2 would require manual renaming of database tables and fields.
I might be able to extend my answer after you provide models and migrations.
migrations.RenameModel(
old_name="ModelA",
new_name="ModelB",
),
migrations.AlterField(
model_name="myrelatedmodel",
name="somefieldname",
field=models.ForeignKey(
# change the properties accordingly (best copy it over). The database may only change to="table"
null=True,
on_delete=django.db.models.deletion.SET_NULL,
to="myapp.modelb",
),
),
If this did not work, where does the error come from? You can try to isolate your migrations and models from the rest of the code (like in a new empty project attached to the same database). Maybe this will help you find out if the error is from a leftover reference to OldModelName or some weird error. Just as an idea so you don't spend time looking at the wrong place
Try to do one change and one migration at a time. You can later squash them. If django does not auto-detect the renaming of your model, it means that all fields related to this model will fail too.
Revert everything to the working state before renaming
Make sure all migrations are applied
Rename the model and create a migration to only rename the model. Are you able to apply it?
if not, try to manually modify all related fields (one2one, foreignkey, etc)
If that still fails, create a copy of your model in models.py, rename the copied model and try to use migrations.RunPython() in a new migration file to copy all the data from the old table to the new one.
If you only have one populated database, you might get away with replacing all ModelA occurrences with ModelB and change the structure of your database.
Best of luck!
I have a little problem with the order of the migrations. The fact is that in my database there is a "Products" model whose migration is one of the first in the history list, call it 001_products. After this migration, others are executed that make inserts in this same table (some inserts necessary for the basic operation of the application), called migration 002_inserts_products.
The problem appeared when modifying the "Products" model, call it 003_modify_products migration. The migration was applied after the inserts and made the tests fail (tests that generate a test database doing all the migrations), which followed this order:
001_products
002_inserts_products
003_modify_products
The solution then was to add a dependency on migrations that made inserts in "Products" with respect to the subsequent migration that modified that table. That is, make 002_inserts_products dependent on 003_modify_products.
However, this, which worked in the tests and locally (where the modification in "Products" had already been applied), does not work in production, since there the migration that has not been applied is the one that modifies the "Products" model ".
That is, the panorama in production is:
[X] 001_products
[X] 002_inserts_products
[ ] 003_modify_products
When trying to do the new migration, the error that appears is:
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration 002_inserts_products is applied before its dependency 003_modify_products on database 'default'.
The question is how to set the migrations to work both in test and in production (that is, in the context that the previous migrations have already been done)?
Unfortunately, you're trying to find a solution for a problem you created yourself by modifying an older migration to make it dependent on a newer migration to get around your tests failing.
The proper solution would be to do the following:
Remove 002_inserts_products's dependency on 003_modify_products, and return it to its original state.
Add 004_update_products to update any products inserted via 002_insert_products so that they work with the table modifications in 003_modify_products.
Update your tests to accommodate the changes made in 003_modify_products.
It's never a good idea to change the expected ordering of migrations that have already run, because while it might work in your local environment, it's very likely to blow up when you're deploying to a server on which none of those migrations have run.
Also remember that tests failing is not always indicative of something you've done wrong -- tests, especially database tests, are not necessarily future-proof. It's totally normal to have to update tests due to schema changes.
This question already has an answer here:
Migrating problems when porting Django project to Python 3 and Django 2
(1 answer)
Closed 3 years ago.
I'm working on upgrading my legacy application from Django 1.11.13 to 2.2.8. I've dutifully addressed every compatibility issue, but I've hit one I can't figure out how to resolve. When I try to start the webserver in my local environment, I get this error (only showing the end of the full error trace that appears):
File "/Users/me/my_app/my_model/migrations/0001_initial.py", line 37, in Migration
('entry', models.ForeignKey(to='my_model.Entry')),
TypeError: __init__() missing 1 required positional argument: 'on_delete'
I understand why on_delete is now required -- I just spent a while updating my models everywhere to accommodate this change -- but I have no idea how to fix this particular issue without going through dozens of old migrations files to make them conform?!
I tried squashmigrations to at least collapse the number of places I have to clean up, but I got the same exact TypeError.
I tried to use the old version of Django for squashmigrations. I was successful in avoiding the TypeError, but ended up with a huge mess of circular import errors.
Since I don't actually need the migration history to roll back, I tried to follow these instructions (scenario 2) to clear the migration history while keeping the existing database, but I couldn't run makemigrations to catch up on the changes I made to make my models Django 2.2 compliant, and when I decided I'd skip ahead and deal with that later, showmigrations failed with the same TypeError. (Is there some other way to get a fresh set of initial migrations based on the current database? It can't be based off the models since the models have upgrade-related changes not yet reflected in the database.)
I moved the migrations to a non-standard location, which got the server to start, but that makes it impossible to actually do anything migration related ever again, and of course once I move back, everything breaks again...
I've considered just deleting my entire database and all migration history, building the tables from scratch with a fresh set of initial migrations, and then resetting the data from a backup, but there are a few huge tables which would make this take quite a while... and this rather seems like the nuclear approach. Am I stuck with editing a large number of very old migrations to be compliant with Django 2.2 for no actual reason since I'm never going to roll my project that far back? How can that be right?
As Iain Shelvington mentions in a comment under the question,
First delete all of your migration files and folder, then run makemigrations with the "on_delete" - this should create some "initial" migration files. Then you'll have to log in to your DB and delete all entries for your apps and then you need to run manage.py migrate --fake - this will enter into the DB entries for the newly created migrations but will not apply them
We've changed our database, using django migrations (django v1.7+).
The data that exists in the database is no longer valid.
Basically I want to test a migration by, inside a unittest, constructing the pre-migration database, adding some data, applying the migration, then confirming everything went smoothly.
How does one:
hold back the new migration when loading the unittest
I found some stuff about overriding settings.MIGRATION_MODULES but couldn't work out how to use it. When I inspect executor.loader.applied_migrations it still lists everything. The only way I could prevent the new migration was to actually remove the file; not a solution I can use.
create a record in the unittest database (using the old model)
If we can prevent the migration then this should be pretty straightforward. myModel.object.create(...)
apply the migration
I think I can probably work this out now that I've found the test_executor: set a plan pointing to the migration file and execute it? Um, right? Got any code for that :-D
confirm the old data in the database now matches the new model
Again, I expect this should be pretty easy: just fetch the instance created before the migration and confirm it has changed in all the right ways.
So the challenge is really just working out how to prevent the unittest from applying the latest migration script and then applying it when we're ready?
Perhaps I have the wrong approach? Should I create fixtures, and just confirm that they're all good at the end? Do fixtures get loaded before the migrations are applied, or after they're all done?
By using the MigrationExecutor and picking out specific migrations with .migrate I've been able to, maybe?, roll it back to a specific state, then roll forward one-by-one. But that is popping up doubts; currently chasing down sqlite fudging around due to the lack of an actual ALTER TABLE instruction. Jury still out.
I wasn't able to prevent the unittest from starting with the current database schema, but I did find it is quite easy to revert to earlier points in the migration history:
Where "0014_nulls_permitted" is a file in the migrations directory...
from django.db.migrations.executor import MigrationExecutor
executor.migrate([("workflow_engine", "0014_nulls_permitted")])
executor.loader.build_graph()
NB: running the executor.loader.build_graph between invocations of executor.migrate seems to be a very important part of completing the migration and making things behave as one might expect
The migrations which are currently applicable to the database can be checked with something like:
print [x[1] for x in sorted(executor.loader.applied_migrations)]
[u'0001_initial', u'0002_fix_foreignkeys', ... u'0014_nulls_permitted']
I created a model instance via the ORM then ensured the database was in the old state by running some SQL directly:
job = Job.objects.create(....)
from django.db import connection
cursor = connection.cursor()
cursor.execute('UPDATE workflow_engine_job SET next_job_state=NULL')
Great. Now I know I have a database in the old state, and can test the forwards migration. So where 0016_nulls_banished is a migration file:
executor.migrate([("workflow_engine", "0016_nulls_banished")])
executor.loader.build_graph()
Migration 0015 goes through the database converting all the NULL fields to a default value. Migration 0016 alters the schema. You can scatter some print statements around to confirm things are happening as you think they should be.
And now the test can confirm that the migration has worked. In this case by ensuring there are no nulls left in the database.
jobs = Job.objects.all()
self.assertTrue(all([j.next_job_state is not None for j in jobs]))
We have used the following code in settings_test.py to ignore the migration for the tests:
MIGRATION_MODULES = dict(
(app.split('.')[-1], '.'.join([app, 'nonexistent_django_migrations_module']))
for app in INSTALLED_APPS
)
The idea here being that none of the apps have a nonexistent_django_migrations_module folder, and thus django will simply find no migrations.
I made a change, adding a unique constraint to a model, within the abc application and did a
./manage.py schemamigration abc --auto
That created a migration file but as well as the expected change the new migration file also contained a number of add_column statements which are adding columns which were previously added in an earlier migration (and which have been the subject of a migrate)
I'm really puzzled as to why this has happened and what to do about it.
Will the add_column statements just be ignored if I do another migrate ?
OK thanks to the #django-south irc channel I've figured this out.
This type of problem can arise when activity has taken place in different source control branches and, as a result of a merge, the dictionary of frozen models, which appears at the bottom of a south migration file, is missing some stuff which has already taken place. The result of this is that the next schemamigration tries to produce the "missing" changes.
The fix is to manually edit the migration file which was created by the schemamigration before doing migrate. This will get things back into synch.
There's some information about issues in the later part of this section : http://south.readthedocs.org/en/latest/tutorial/part5.html#team-workflow .
Thanks to carljm and maney on #django-south for helping me with this.