I have a python Django project with several applications and multiple git branches for each features.
It's a true hell to do the migration by hand because there are too many apps, and have to diff showmigrations to see what have changed...
Is there a way to automate this, or at least a way to generate the migration I should restore the db to ?
Related
While developing a Django project, all your migrations are stored within each app folder, however, in production I don't want those migrations, I want to keep a Production database, and a Development database:
How do I handle Django migrations in a Production and Development environment?
I'm asking this question because it's been really hard to update my deployed project with new additions in the development one, my ideal scenario would be to keep each set of migrations in a folder outside my source code, just like the databases.
The best idea is to keep production and development migrations the same and while developing you clean migrations before pushing the code and you should push migrations into your Version Control System too.
In development, you might end up deleting a table and re-creating it so make sure you don't push the un-intended migrations. The thing is you should treat migrations as code, not an automated script. I have done a lot of mistakes in the past, so, I came to the conclusion of including migrations in code. and that's effective and gives more control.
Moreover you might have to do data migrations in production, how will you do if you wont push the code?
I have a running Django application on Heroku with migrations auto-run on release. While in most times this works fine sometimes there is a problem when:
There are more than one migration in given release (they can be in different apps)
Some migration will fail, but not the first one
In this case manage.py migrate will fail so Heroku will not finish the release and will not deploy the new code. This means that code is in the old version and the database is in the state "somewhere between old and new".
Is there a simple way to autorun Django run reversed migrations in case of the failure of the release command on Heroku?
Transactions won't help here as there might be more than one migration (multiple apps) and Django run each migration in seperate transaction.
As I couldn't find any existing solutions I am posting a gist I have written to solve this.
https://gist.github.com/pax0r/0591855e73b9892c28d3e3cdd15f4985
The code stores the state of migrations before running the migration and in case of any exception reverts back to this state. It also checks if all migrations are reversible during the migrate step.
It's not yet well tested, but I will work forward to create a library from it for easier use by others.
What are some good best practices to prevent the following.
Often, during django development I use postgres. If I roll back to an earlier commit, often new migrations will break things irreparably, because the postgres context is unknown to the older code. I have tried the syncdb etc, I have tried makemigrations on single apps, but neither seem to fix the problem.
Also, if creating a new postgres db and deleting all the migrations and pycache folders, this too often causes problems that are not straightforward to fix.
Is the answer to simply use sqlite during development, has anyone else encountered this?
Details: django 2.0.5 postgres: most current.
I am following online instructions on starting a Django project the right way.
The instructions are based on an earlier version of Django. From my (admittedly limited) knowledge of Django. The latest release of Django (1.10 at the time of posing this question), already handles migrations seemingly well - by way of the manage.py script.
My question then is this: Do I still need to install South to manage my migrations, or can I simply skip that part of the instructions, and use manage.py to deal with my db schema changes?
No, South was for Django before 1.7. With 1.10 everything you would have used it for is baked into Django itself.
We have migrated the Django 1.4 application to Django 1.8 successfully. The Django 1.4 version of applicaiton is still in use in production until we go live with Django 1.8. The problem is that lots of data have been updated on production server which needs to be migrated to version 1.8. Is there any way I can migrate the data from database of 1.4 to 1.8 except manually doing that? Note that the model/database columns are different in both the version.
Can anybody suggest some good options ?
Thanks.
Required pre-reading
Django migrations:
https://docs.djangoproject.com/en/1.8/ref/django-admin/#makemigrations-app-label
Assuming you were using South:
https://docs.djangoproject.com/en/1.8/topics/migrations/#upgrading-from-south
Getting Started
Firstly dump your local database I prefer using mysql/postgres/whatever docs for this rather than using ./manage.py dumpdata.
You will also want to dump your production database as well just for safekeeping.
Next in your local environment I would dump the database and create a new database.
Then I would test that all your migrations actually work on a blank database.
These are the instructions for django 1.8
./manage.py makemirgrations
./manage.py migrate
That will help show up if any migrations are in an inconsistent state operating from a blank slate. If you encounter any errors they should be fixed first.
Given that works, now I would test that your migrations actually work against your production data.
So drop your local database, create a fresh one then load the production dump in.
If the tables are already configured correctly (i.e. they are in an up to date state in your production database) then you will need to fake all migrations.
./manage.py migrate --fake <appname>
However given that you have changed some models since upgrading to 1.8 in your local environment, then you might need to fake only some of the migrations. This could be the tricky part depending on the timing of when you upgraded and when you created the migrations.
Because django 1.7 will just create an initial migration for each app, you may need to actually break up the migration for some apps. That is, instead of 0001_initial you might need to manually break up that migration into 2 components:
1. a migration to match the current state of your production database
2. a migration to match any additional changes you have made to your model since then.
One way to do this is to checkout your first commit once you had django 1.8 working correctly locally then run
./manage.py makemigrations
then commit that
then go forward to your latest commit then run
./manage.py makemigrations
Now you should have 2+ migrations in each app that you have modified since upgrading to django 1.8.
Then you can fake initial on those apps that have 2+ new migrations for django 1.8
./manage.py migrate --fake-initial app1 app2
and the rest just
./manage.py migrate app3 app4
Now run your tests to confirm everything is working locally.
If you have changed migrations you will again want to test locally against a blank database to test that they work smoothly
Once that is working, record the 'migrate' commands that you used - then deploy your app to production and run just those migration commands once you have upgraded to django 1.8 on your server.
After Successful Completion
Take new dumps of your local and production databases
Uninstall South (assuming you had it installed before) from local and production environments
I'm sure there are a couple of holes in the above but hopefully that gives you the gist of what you need to do.