Background :-
I am using Django 1.3. We are using South as the module for DB migration and Git SCM.
Problem:-
What is the correct way to deal with the migrations Folder that is formed?
The main problem is I make changes in the DB schema in the development machine, when I upload it to the production server I have to migrate the existing schema. While doing that there is always some issue with the migration files.
Should I just add the migrations folder to the gitignore ? or is there a better way to go about it ?
You should add the migrations folder to your version control system and use the same files for production and development. You may run into some problems on your production system if you introduced your migrations not from the beginning and you have already existing tables.
Therefore you have to fake the first migration, which normally does the same thing as syncdb did when you created your database for the first time.
So when trying to apply migrations for your app for the first time on the production machine, execute manage.py migrate app_name 0001 --fake. This lets South know, that the first migration has already been applied (which already happend with syncdb) and when you run migrate again, it would continue with the following migrations.
Related
I have a django REST project and a PostgreSQL database deployed to DigitalOcean. When I develop locally, I have a separate dockerized REST server and a separate PostgreSQL database to test backend features without touching production data.
My question arises when I'm adding/modifying model fields that require me to make migrations using python [manage.py](https://manage.py) makemigrations and python [manage.py](https://manage.py) migrate command. Here is my current situation so far:
What I was supposed to do
IN LOCAL ENV, to create the migration files,
python manage.py makemigrations
python manage.py migrate
Now commit these newly created files, something like below.
git add app/migrations/...
git commit -m 'add migration files' app/migrations/...
IN PRODUCTION ENV, run only the below command.
python manage.py migrate
What I did so far
IN LOCAL ENV, created the migration files,
python manage.py makemigrations
python manage.py migrate
I committed & pushed the changes to production WITHOUT the created migration file
IN PRODUCTION ENV, ran BOTH commands.
python manage.py makemigrations
python manage.py migrate
The production server successfully added the isActive field to the database and is working fine, but I still have a 0011_user_isActive.py migration file in my local changes that hasn't been staged/committed/pushed to github repo.
And because I ran makemigrations command in production env, it probably created the same migration file that I haven't pushed from local env.
My questions are:
What happens if I push the local migration file to production? Wouldn't it create a conflict when I run migration command on digitalocean console in the future?
How should I fix this situation?
I am just scared I'm going to corrupt/conflict my production database as I'm very inexperienced in databases and have too much to risk at the moment. Would appreciate any tips on best practices when dealing with such situations!
As docs says:
The migration files for each app live in a “migrations” directory inside of that app, and are designed to be committed to, and distributed as part of, its codebase. You should be making them once on your development machine and then running the same migrations on your colleagues’ machines, your staging machines, and eventually your production machines.
So it's best practice is to push your migration files and make your local and production migration files sync.
And if you got conflict when pushing migraions files and pulling them, the makemigrations --merge command is for solving that.
Also docs says:
Because migrations are stored in version control, you’ll occasionally come across situations where you and another developer have both committed a migration to the same app at the same time, resulting in two migrations with the same number.
Don’t worry - the numbers are just there for developers’ reference, Django just cares that each migration has a different name. Migrations specify which other migrations they depend on - including earlier migrations in the same app - in the file, so it’s possible to detect when there’s two new migrations for the same app that aren’t ordered.
When this happens, Django will prompt you and give you some options. If it thinks it’s safe enough, it will offer to automatically linearize the two migrations for you. If not, you’ll have to go in and modify the migrations yourself - don’t worry, this isn’t difficult, and is explained more in Migration files below.
Also be aware that in case of updating existed data in production, you can use RunPython in migration file. Read about it here.
So I started working on a project that is right now on production but the preovious dev kinda made a mess with migrations (or maybe I am wrong), the migrations in developement and in production are not the same (he gitignored them) and I can not start working because there are missing migrations in development so the database is no fully migrated.
Im thinking I can just delete all migrations in development, reset them but I don't think that is a good idea in production. What should I do?
Migrations have two purposes: 1. Creating schema of database, 2. Migrating existing data of database (For example when you change a field from IntegerField to CharField, you need to write some migrations to convert integers saved in database to their equal char.) This one is only needed for production. If you are missing these migrations, it is not a problem, Because you only need schema of database to develop. But how can you make sure you have the correct schema? Run python manage.py makemigrations and then python manage.py migrate. You are good to go and there is no need to delete any prior migrations.
I have a few projects with lots of South migrations, including ones that contain a fair amount of custom SQL that need to be run in a specific order. After upgrading to Django 1.7, this is the recommendation on how to convert a project to use South (from the Django documentation):
If you already have pre-existing migrations created with South, then the upgrade process to use django.db.migrations is quite simple:
Ensure all installs are fully up-to-date with their migrations.
Remove 'south' from INSTALLED_APPS.
Delete all your (numbered) migration files, but not the directory or __init__.py - make sure you remove the .pyc files too.
Run python manage.py makemigrations. Django should see the empty migration directories and make new initial migrations in the new format.
Run python manage.py migrate. Django will see that the tables for the initial migrations already exist and mark them as applied without running them.
In short, "wipe your existing migrations and Django will take care of the rest".
What is not mentioned here is what to do when existing South migrations don't only consist of model changes, but instead contain direct SQL, data migrations, etc, that need to be run in order. In this case, the auto-generated Django migrations will miss a lot of things, since not all of these changes are obvious from introspecting a models file.
Ideally, one would be able to run the existing migrations using South, and then have Django migrations take over. What might be the best way to go about this? If this is not possible or very much not recommended, what is the best alternative?
Maybe this post can help you. Essentially you have to:
Change your current migration directory from 'migrations' to 'south_migrations'
Update your settings with this line
SOUTH_MIGRATION_MODULES = {
'your_app': 'your_project.your_app.south_migrations',
}
The application I'm developing is starting to need migration for the database schemas. I've thought about django-south, but since I have no experience with that kind of stuff I'm a bit lost, this is my situation:
Development code: latest models, I didn't keep track of what changes I've made to the models.
Production code: running code, has old models. We have the server configured so we can make deployments just with a git pull :)
How can I update the code in production (and the DB) without breaking anything? I saw about the --initial statement but I don't think it works for this case, and also for convert_to_south to fake a migrationhistory, but I still don't get what should I do. Any help please?
Imo it would be better to create versioning for your project and deploy it with something like Fabric. This will contain your production environment nicely.
There's no magic with south, just add south to the installed apps setting and run an initial schemamigration then run a fake migration (migrate <app_name> --fake) so south 'knows' the current state of your models. In future releases (that contain schema changes) you can run schemamigration <app_name> --auto and migrate <app_name> to update the models accordingly.
To keep your migrations in one place use the migrations setting in your settings file:
SOUTH_MIGRATION_MODULES = {
'app_name_1': 'project_name.migrations.page',
'app_name_2': 'project_name.migrations.medialibrary',
}
You'll have to checkout the production version (to get the models back to production state), create an initial migration, copy that migration to your current development branch then create a schemamigration.
I am using South to manage schema and data migrations on my development and production environments. As such I keep my migrations in my git repository so changes I make in development are properly migrated in production.
The apps and projects I am developing are currently private and only developed by me. At some point, I would like to publish/distribute my apps. I am assuming that at that point, I will have a "final" schema and therefor won't "need" South. Since these apps haven't been distributed before (except on my environments), the public version won't need the migrations that I used while developing the apps.
I have a two parter question:
Is it good (or acceptable) to remove or at least "clean up" the migrations based on my assumptions?
What's the best way of doing so? I imagine keeping a branch for the public/distributed base could work, but I'm fairly new to git, so I don't know what my options are.
Thanks,
This is a common point of confusion for people dealing with South and version control. You should, of course, commit migrations with your project, as others will need to run the same migrations themselves. However, you should clean up your migrations before committing them, which is actually deceptively easy.
If it's a brand new app or a brand new project in general:
Rollback the app to "zero":
python manage.py migrate myapp zero
Delete all migrations for the app (Everything in the "migrations" directory except __init__.py).
Generate a new initial migration:
python manage.py schemamigration --initial myapp
If it's an existing app, then the process is largely the same, but you're only going to rollback to just before the first new migration your created. And, then you will simply generate a new auto migration instead of an initial migration. For example, if the app was already at migration 0005 and you create migrations 0006, 0007, and 0008:
Rollback to just before the first migration you created (0006):
python manage.py migrate myapp 0005
Delete all new migrations you created (0006, 0007, and 0008)
Generate a new auto migration:
python manage.py schemamigration --auto myapp
Either way, you'll end up with just one file encapsulating all of your changes instead of multiple files. Then, you commit this to your source control.