This is a Heroku-specific issue with a Django project 1.11.24 running Python 3.6.5 and a Heroku postgres database.
During testing of two different branches during development, different conflicting migration files were deployed at different times to the Heroku server. We recognized this, and have now merged the migrations, but the order the Heroku psql db schema was migrated is out of order with the current migration files.
As a result, specific tables already exist, so on deploy applying the updated merged migration files errs with:
psycopg2.errors.DuplicateTable: relation "table_foo" already exists
In heroku run python manage.py showmigrations -a appname all of the migrations are shown as having run.
We've followed Heroku's docs and done the following:
Rolled back the app itself to before when the conflicting migrations took place and were run (https://blog.heroku.com/releases-and-rollbacks)
Rolled back the postgres db itself to a datetime before when the conflicting migrations took place and were run (https://devcenter.heroku.com/articles/heroku-postgres-rollback)
However, despite both app and db rollbacks, when we check the db tables themselves in the rollback in pql shell with \dt, the table causing the DuplicateTable err still exists, so the db rollback doesn't actually seem to effect the django_migrations table.
It's Heroku, so we can't fake the migrations.
We could attempt to drop the specific db tables that already exist (or drop the entire db, since it's a test server), but that seems like bad practice. Is there any other way to address this in Heroku? thanks
I eventually fixed this by manually modifying migration files to align with the schema dependency order that was established. Very unsatisfying fix, wish Heroku offered a better solution for this (or a longer postgres database rollback window)
Related
I'm trying to implement Google Login on my Django production website. It works fine on development server, but on my production server, I face this issue after having run python manage.py migrate. I'm not sure what to do, I already tried deleting all the migrations and re-running makemigrations and migrate.
Checklist
run showmigrations to see which migrations are done.
if the relevant migration containing the table appears to be migrated, rollback and run migrate again.
when you want to regenerate migration files, you need to first rollback while you have the old migration files.
if you'd like to check which tables exist, use dbshell or other shell depending on the database you're using.
Other things
I would not recommend regenerating migration files just because there is some db issue.
I have a learning project deployed on Heroku. It had a Postgres database provisioned. I introduced some major changes in the models of my Django project and destroyed the old database and provisioned a new one, which is totally empty, but it is not working like an empty database.
When I run the command heroku run python manage.py makemigrations, I get the error message
You are trying to add a non-nullable field....
Why am I getting this message when I have destroyed the old database?
First of all, you should never run manage.py makemigrations on Heroku.
By the time your code gets there no model changes should exist to generate new migrations. Run makemigrations locally to create migration files. Run migrate locally and on Heroku to apply migrations to your database.
Now that that's out of the way, this is likely caused by existing migrations files, not anything in your database. If you truly want to start over you can delete the files from each of yours apps' migrations/ directories.
Finally, there is no need to destroy and reprovision your database to reset it. Instead you can use heroku pg:reset:
The PostgreSQL user your database is assigned doesn’t have permission to create or drop databases. To drop and recreate your database use pg:reset.
use this command
heroku pg:reset
I have deployed a django application to heroku but I need to reset those migrations. Is that possible without deleting the entire project and redeploying it? I have some test data in that database that I would prefer not to have to enter all over again.
I'd like to delete all migration files and create new ones. Thing is, I deleted all migration files from my local machine and created new ones so now the migration files on my local machine are all 001. Pushing that to heroku says there were no changes because the 001 migration files already exist on there. Only deleting them would work. Basically something similar to
find . -path "*/migrations/*.py" -not -name "__init__.py" -delete
find . -path "*/migrations/*.pyc" -delete
but for heroku.
Pushing to Heroku is part of the solution, but not all of it. This updates the migration files, but not the django_migrations table in your database.
If you still have the old code running on Heroku:
Back your database up
Reverse all of your migrations:
heroku run python manage.py migrate appname zero
Then deploy your new code:
git push heroku master
Then run the new migrations:
heroku run python manage.py migrate
If you've already deployed the new code to Heroku:
Back your database up
Manually delete your app's tables from your database, as well as rows from the django_migrations table where the app matches your app name e.g. via heroku pg:psql (this may get tricky if you have other apps installed and inter-app dependencies)
Run your new migrations:
heroku run python migrate
Another, probably safer, option if you've already deployed the new code is to rollback to a version that reflects your database schema, then use option 1 above. In this case you can rollback again to the new version instead of doing a regular release.
Yes it can be done by two ways.
You can generate fixture for your current database schema if no such column is deleted or altered.
You can dump your database than just remove database from your db server and migrate again! Than load dumped data into fresh db, this will keep your previous data safe.
I'm new to Django but I'm deploying a Django-based website to Heroku, using Postgresql. The deployment was successful, and the website is online and has established connection with the database. However, none of the data from my local database has migrated to the heroku database, causing it to be a blank database. If I go into the admin section and manually input a datapoint, it appears on my site, so I know that database is correctly serving data. What is the proper way for migrating data from your local database to your online, heroku version of the database? I thought the following code would migrate the data:
heroku run python manage.py makemigrations
heroku run python manage.py migrate
But apparently I'm missing something.
make migrations will create a migration that contains your schema, but no data. The migrate command applies the migration to the database.
In order to provide data to be sent over as part of the migrate command you need to either create a data migration or use a fixture.
Another option you have is to dump your local database and do an import into Heroku Postgres
All in all, it depends on how much local data you have that you want copied over. If its only a few rows, I would use either a data migration or a fixture, if its 100s or 1000s of rows an export/import of your dataset is your best bet.
Background :-
I am using Django 1.3. We are using South as the module for DB migration and Git SCM.
Problem:-
What is the correct way to deal with the migrations Folder that is formed?
The main problem is I make changes in the DB schema in the development machine, when I upload it to the production server I have to migrate the existing schema. While doing that there is always some issue with the migration files.
Should I just add the migrations folder to the gitignore ? or is there a better way to go about it ?
You should add the migrations folder to your version control system and use the same files for production and development. You may run into some problems on your production system if you introduced your migrations not from the beginning and you have already existing tables.
Therefore you have to fake the first migration, which normally does the same thing as syncdb did when you created your database for the first time.
So when trying to apply migrations for your app for the first time on the production machine, execute manage.py migrate app_name 0001 --fake. This lets South know, that the first migration has already been applied (which already happend with syncdb) and when you run migrate again, it would continue with the following migrations.