After refactoring some models in a bloated app (everything in appname/models.py) into a subfolder application (some of the models in appname/subapp/models.py) and running makemigrations, I'm getting the following error when running manage.py migrate:
ValueError: Found wrong number (2) of constraints for appname_modelname1(modelname2_id)
Getting rid of all migrations and starting over would be one option, but then I'd have to manually edit all existing production databases. Are there any alternatives to make the migrations apply smoothly?
Turns out that ./manage.py makemigrations makes overly complex migrations that will likely fail on a real constraint-enforcing database. If the objective is just to get rid of models, first make the migrations, then edit the migration file so that only the RemoveModel directives remain, and finally apply the migrations.
Related
On our staging server we observed a runtime error stating a field is missing from a database,
column our_table.our_field does not exist
LINE 1: ...d"."type", "our_table"...
The field was added during a recent update with a complicated migration squashing process. It's possible that some errors were made during this process, but "manage.py showmigrations" command shows that the migration has been applied and "manage.py makemigrations" does not create any new migration. As we do not run tests on our staging or production databases, we are trying to figure out the most effective method for identifying such errors.
In short, how can we identify mismatches between the database and Django models caused by an incorrect migration like the following?
python manage.py migrate our_app --fake
I suppose I am looking for something like
python manage.py check_database
Edit: Many thanks for the suggestions. However, this is more of a deployment than a development question because the problem likely occurred when our devops tried to apply the squashed migrations while retaining data on the staging server (which will be case on production). It was scary to learn that such inconsistency can occur when makemigrations and showmigrations do not show any problem and can therefore also happen on production.
The bottom line is that we need some way to ensure our database matches our models after deployment.
We are fairly new to Django. We we have an app and a model. We'd like to add an 'Category' object to our model. We did that, and then ran 'python manage.py makemigrations'.
We then deploy our code to a server running the older code, and run 'python manage.py migrate'. This throws 2 pages of exceptions, finishing with 'django.db.utils.ProgrammingError: (1146, "Table 'reporting.contact_category' doesn't exist")'
This seems to be looking at our models.py. If we comment out Category from our model, and all references to it, the migration succeeds.
I thought that the point of migrations is to make the database match what the model expects, but this seems to require that the model match the database before the migration.
We clearly are doing something wrong, but what?
I believe you skipped some migration in the server, so now you are missing some tables (I have been in that situation. Ensure migrations directories are on your .gitignore. You CAN NOT check in migrations files, you have to run makemigrations on the server). This can be solved by tracing back up to the point the database and models files match, but it is a risky process if it is your production database, so you should make a full backup before proceeding, and try the process on a different computer before.
This would be my advice:
Delete migration files from the server.
Comment the models that rise the error.
Set the server's migration history to the point the
database is, using python manage.py makemigrations and python manage.py migrate --fake-initial (this will update the migration files without actually attempting to modify the database).
Uncomment the models that raise the error.
Run python manage.py makemigrations and python manage.py migrate.
If, after you comment the models that raise the exception, you get a different exception, you have to keep on commenting and attempting again. Once a migrations succeeds, you can uncomment all commented models and make an actual migration.
Remember to run python manage.py makemigrations if you made changes to the models.py then run python manage.py makemigrations
Both commands must be run on the same server with the same database
I have a relatively complex set of Django models. I'm trying to start with a fresh set of migrations (rm -rf apps/*/migrations; bin/dev/manage.py makemigrations A B C...). makemigrations works fine, and there are no circular dependencies, but I'm consistently getting an InconsistentMigrationHistory exception when I migrate. Here's a graph of the dependencies between the migrations, simplified to remove the migrations with no related dependencies, and with the app names redacted for readability:
The links in red cause the error (different ones each time I run migrate), even with a run_before added to each migration that should be run before its dependency:
A/migrations/0002_whatever.py:
...
run_before = [('P', '0001_initial'),]
Here's the error text.
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration `P.migrations.0001_initial` is applied before its dependency `A.migrations.0002_whatever` on database 'default'.
Any ideas?
You need to reset the database as well.
When you created new migrations without resetting the database, the sync between your database and migrations was lost.
So when you try to migrate the new migrations, Django will find it inconsistent and fail.
It is generally not a good idea to clear all migrations and generate new migrations for the same reason.
Imagine this happening in production.
As I was changing my models.py and migrating, I got an error message saying:
python manage.py makemigrations project_profile
CommandError: Conflicting migrations detected; multiple leaf nodes in the migration graph: (0033_auto_20180217_0912, 0036_auto_20180217_0927 in project_profile).
To fix them run 'python manage.py makemigrations --merge'
So, when I tried to follow the instructions, I got another error that one of my tables that the merged migration is now depending on do not exist anymore (I renamed it). Interestingly enough, this renaming took place during the merge operation. So, really Django should have known about it in the first place.
To resolve the situation, I deleted prior migrations up to and including the migrations that was not applied, the one that caused all the headache. I tried to makemigrations and migrate again. But, Django now throws another error saying some of the models it wants to create in the database already exist. Obviously, I do not want to delete those tables and loose all that information to appease Django. So, I had to resort to some hacking solutions and actually change those tables manually and do a fake migration in order to stop Django from complaining.
Having said all of that, I feel like there should be a more logical way about this. How do I resolve migrations during the merging?
I had the same issue, Then I was able to solve this by deleting the migrations file that django pointed out and starts with name auto. It occurred 2-3 times before it finally gave up and finally worked.
Alternatively you can django-dbbackup or django-import-export packages to backup the tables then clean your database and migrations. Then you can restore them back to the same state once migrations are stable.
Sources
dbbackup : https://django-dbbackup.readthedocs.io/en/stable/
import-export : https://django-import-export.readthedocs.io/en/latest/index.html
I started a django 1.8 project, which uses the migrations system.
Somehow along the way things got messy, so I erased the migrations folders and table from the DB, and now I'm trying to reconstruct them, with no success.
I have three apps (3 models.py files), and the models reflect the tables EXACTLY!
The best approach that I've found so far was:
Erase all migrations folders. Done!
Delete everything from the django_migrations table. Done!
Run python manage.py makemigrations --empty <app> for every app. Done!
Run python manage.py migrate --fake. Done! (although it works only if I run it after every makemigrations command.
Now I add a new field, run the makemigrations command, and I receive the following error:
django.db.utils.OperationalError: (1054, "Unknown column 'accounts_plan.max_item_size' in 'field list'")
I've been burning HOURS on this thing. How the h**l can I initialize the migrations so I can continue working without migration interruptions every time?
Why is it so complicated? Why isn't there a simple one-liner: initiate_migrations_from_schema?
EDIT:
Now things get even nastier. I truncated the django_migrations table and deleted all the migrations folder.
Now I try to run python manage.py migrate --fake-initial (something I found in the DEV docs), just so it sets up all of Django's 'internal' apps (auth, session, etc) and I'm getting:
(1054, "Unknown column 'name' in 'django_content_type'").
Now, this "column" is not a real column. It's a #property defined in Django's contenttypes app. WHAT IS GOING ON HERE? Why is it identifying the name property as a real column?
Finally got it to work, although I don't know why and I hope it will work in the future.
After doing numerous trials and going through Django's dev site (link).
Here are the steps (for whoever runs into this problem):
Empty the django_migrations table: delete from django_migrations;
For every app, delete its migrations folder: rm -rf <app>/migrations/
Reset the migrations for the "built-in" apps: python manage.py migrate --fake
For each app run: python manage.py makemigrations <app>. Take care of dependencies (models with ForeignKey's should run after their parent model).
Finally: python manage.py migrate --fake-initial
After that I ran the last command without the --fake-initial flag, just to make sure.
Now everything works and I can use the migrations system normally.
I'm sure I'm not the only one who encounters this issue. It must be documented better and even simplified.
Update for Django 1.9 users:
I had this scenario again with a Django 1.9.4, and step 5 failed.
All I had to do is replace --fake-initial with --fake to make it work.
django ..., 1.8, 1.9, ...
What you want to achieve is squashing existing migrations and use replacement for them.
How to do it right without using any command when releasing (a case without impact on database and coworkers).
For every app, get rid of its migrations folder:
mv <app>/migrations/ <app>/migrationsOLD/
For each that app run: python manage.py makemigrations <app>.
Customize each new migration:
if you have a complex app, or more apps and related models between them, to avoid CircularDependencyError or ValueError: Unhandled pending operations for models:
prepare second empty migration in <app> 0002_initial2.py (put there dependency to app_other::0001_initial.py and <app>::0001_initial.py as well - all ForeignKey, M2M related to models created in 0001 migration step in other apps)
All must be in order - sometimes it will require more migrations to prepare. Take care of dependencies attribute here in each Migration.
take care of initial values - verify yourself all RunPython actions from migrationsOLD and copy the code to new initial migration if needed.
(optional for --fake-initial) Add initial=True to all new Migration classes (0002 too if was added).
Add replaces attribute in new Migration class. (like own custom a squashmigrations). Put there all old migrations from <app>
Verify everything with makemigrations.
assert "No changes detected"
Check if migrate -l show [x] everywhere
assert similar:
[X] 0001_initial
[X] 0002_initial2 (102 squashed migrations)
Example:
For old:
0001_initial.py
0002_auto.py
...
0103_auto.py
prepare:
0001_initial.py
0002_initial2.py (optional but sometimes required to satisfy dependency)
and add to replacesto last one (0002 here, can be 0001):
replaces = [(b'<app>', '0002_auto.py'), ..., (b'<app>', '0103_auto.py')]
0001_initial.py should be named the same way as old one.
0002_initial2.py is new one, but it's a replacement for old migrations so Django will treat it as loaded.
I've run into this scenario but I've never had to drop the database to solve it. Typically I delete the migrations folder from the app's, and remove the migration entries from the database.
I would try to run make migrations one app at a time. If any of the app's rely upon other tables obviously add them last.
Also I usually just run, python manage.py makemigrations then just python manage.py migrate Even with the initial migration it should work fine with Django 1.7 and 1.8.
If you are using routers, might be a problem there. Check method allow_migrate if it is executed in a right way in routers.py. Try to set return value always to be True, and check whether it resolves problem,
def allow_migrate(self, db, app_label, model_name=None, **hints):
return True