We seem to have gotten our prisma migrations into a bad state. When we get the latest code and run
prisma migrate dev
we get
Migration 20210819161149_some_migration failed to apply cleanly to
the shadow database. Error code: P3018 Error: A migration failed to
apply. New migrations cannot be applied before the error is recovered
from. Read more about how to resolve migration issues in a production
database: https://pris.ly/d/migrate-resolve
Migration name: 20210819161149_some_migration
Database error code: 1065
All the migrations in source control do match the ones in the _prisma_migrations table so I'm not sure why it thinks 20210819161149_some_migration failed. There is nothing in the logs column in _prisma_migrations for that record. I think what happened is a developer applied the migration then changed the migration.sql for it after the fact.
Anyway, we followed the steps outlined https://pris.ly/d/migrate-resolve but they don't seem to resolve the issue. It first suggests running
prisma migrate resolve --rolled-back "20210819161149_some_migration"
but that results in
Error: P3012
Migration 20210819161149_some_migration cannot be rolled
back because it is not in a failed state.
So then we tried to mark it as applied
prisma migrate resolve --applied "20210819161149_some_migration"
but that results in this error
Error: P3008
The migration 20210819161149_some_migration is already
recorded as applied in the database.
We also tried running
prisma migrate deploy
Which gives
13 migrations found in prisma/migrations WARNING The following
migrations have been modified since they were applied:
20210819161149_some_migration
but you still get the same issue above when running prisma migrate dev.
Is there any way to get prisma happy again without deleting all the data?
A possible workaround would involve baselining your development database based on your current schema/migration history. You would need a separate backup database and you will lose your existing migration history, but it should retain your data.
Here's what the process would look like
Delete all migration history from your prisma folder as well as the _prisma_migrations table in your database.
Create a new backup database.
Connect the project to your backup database and run prisma migrate dev --name baseline_migration . This will generate a migration matching your current prisma schema.
Connect back to your main Database and baseline the generated migration by running prisma migrate resolve --applied 20210426141759_baseline_migration (The numbers at the beginning of your migration name will differ).
The reason you'd be creating a backup database and running the initial baseline migrations in that backup database is because you don't want to lose the data in your main database. I realize this is not an ideal solution, but it might work if it's very important for you to keep your data while retaining your existing dev workflow.
This article on Adding Prisma Migrate to an existing project is also worth a read.
You can do how it`s described in this short video
https://youtu.be/BIfvmEhbtBE
https://www.prisma.io/docs/reference/api-reference/error-reference this site will provide you the the correct ways of troubleshooting your problem. I got a migration error (P3009) and when i investigate through the generated sql file and found that some syntax issue occured in the generated file and resolved them and fix the migration issue
Related
On our staging server we observed a runtime error stating a field is missing from a database,
column our_table.our_field does not exist
LINE 1: ...d"."type", "our_table"...
The field was added during a recent update with a complicated migration squashing process. It's possible that some errors were made during this process, but "manage.py showmigrations" command shows that the migration has been applied and "manage.py makemigrations" does not create any new migration. As we do not run tests on our staging or production databases, we are trying to figure out the most effective method for identifying such errors.
In short, how can we identify mismatches between the database and Django models caused by an incorrect migration like the following?
python manage.py migrate our_app --fake
I suppose I am looking for something like
python manage.py check_database
Edit: Many thanks for the suggestions. However, this is more of a deployment than a development question because the problem likely occurred when our devops tried to apply the squashed migrations while retaining data on the staging server (which will be case on production). It was scary to learn that such inconsistency can occur when makemigrations and showmigrations do not show any problem and can therefore also happen on production.
The bottom line is that we need some way to ensure our database matches our models after deployment.
This is a Heroku-specific issue with a Django project 1.11.24 running Python 3.6.5 and a Heroku postgres database.
During testing of two different branches during development, different conflicting migration files were deployed at different times to the Heroku server. We recognized this, and have now merged the migrations, but the order the Heroku psql db schema was migrated is out of order with the current migration files.
As a result, specific tables already exist, so on deploy applying the updated merged migration files errs with:
psycopg2.errors.DuplicateTable: relation "table_foo" already exists
In heroku run python manage.py showmigrations -a appname all of the migrations are shown as having run.
We've followed Heroku's docs and done the following:
Rolled back the app itself to before when the conflicting migrations took place and were run (https://blog.heroku.com/releases-and-rollbacks)
Rolled back the postgres db itself to a datetime before when the conflicting migrations took place and were run (https://devcenter.heroku.com/articles/heroku-postgres-rollback)
However, despite both app and db rollbacks, when we check the db tables themselves in the rollback in pql shell with \dt, the table causing the DuplicateTable err still exists, so the db rollback doesn't actually seem to effect the django_migrations table.
It's Heroku, so we can't fake the migrations.
We could attempt to drop the specific db tables that already exist (or drop the entire db, since it's a test server), but that seems like bad practice. Is there any other way to address this in Heroku? thanks
I eventually fixed this by manually modifying migration files to align with the schema dependency order that was established. Very unsatisfying fix, wish Heroku offered a better solution for this (or a longer postgres database rollback window)
I have a Postgres database full of data. And I made several changes to my Django app models.
mange.py makemigrations worked fine and created the migration files. But manage.py migrate execute only one file. And when I launch it again it doesn’t execute the rest as if they are already applied.
I deleted the migration files that were not applied and did another makemigration but it says no changes detected.
Any ideas how to reflect the models changes on the database without losing the data ?
Thanks
Django keeps track of which migrations it has applied already, so when you run the migrate command it will execute only the migrations that Django thinks that are missing.
I deleted the migration files that were not applied and did another makemigration but it says no changes detected.
This was a bad idea, it will make your migrations inconsistent.
If you want to go back in time, instead of deleting migrations, the proper way to do this is by reverting migrations. You can use the same migrate command and specify to which migration point you want your database model to be.
Check this answer for further information about reverting migrations; django revert last migration
I am running a GIS application while getting this error. I have attached error snapshot . Can Someone guide where is error ? If need to see code . Let me know which file you need.
Especially the first line of your exception value pretty much says it all.
Column users.parent_id does not exist
The application is trying to access parent.id from the users table from your database, which obviously does not exist. With other words, your database is not sync with the model structure in the source code. Probably all you have to do, is to run the migrations to add all the missing structures or changes to your database.
If you've developed some of the database structures yourself, you have to run the makemigrations command to build the new migration set for your database.
./manage.py makemigrations
If you have created new migration files or if you have installed and integrated modules into your app, you have to apply the migrations to your database.
./manage.py migrations
I need to know if getting an error while running python manage.py migrate means my database will remain in the same state it was before running the migrate command.
I'm trying to implement migrations as part of a CI system and it would be good to know if I need to do some kind of rollback if the migrations fails.
As the documentation explains, it depends on the database.
PostgreSQL can use transactions for schema alteration operations, so Django does so, and rolls back in case of failure. But MySQL does not support this.