I'm currently using Django evolutions to manage my product's database evolutions. It's not perfect but I've learned to live with its flaws. For instance, I always have to copy my production database to test before moving out a new schema because the "evolve" command cannot always evolve a database that was changed in several small migrations (on test I did A->B->C, but A->C will not evolve correctly.)
Will South fix all of those problems? Is it worth the effort of learing a new tool?
I just started using South, and I'm 100% sold on it. It's also one of the few that's still under very active development.
South should be able to properly handle the issues you've describe above. For each change to the db, it creates a file that has 2 methods "foward" and "backwards". Here's a sample automatically generated migration:
# > manage.py schemamigration issuetracker added-status-field --auto
# 0004_added-status-field.py
class Migration:
def forwards(self, orm):
# Adding field 'Issue.status'
db.add_column('issuetracker_issue', 'status', orm['issuetracker.issue:status'])
def backwards(self, orm):
# Deleting field 'Issue.status'
db.delete_column('issuetracker_issue', 'status')
A couple of the nice things about it....
South lets you rollback to a specific migration # if you'd like
If your production site is on migration 0002 and your SVN commit is on 0004, South will do 0003 then 0004 to bring the production db up to speed.
If you've gone ahead and made the changes yourself, you can tell South to run a 'fake' migration. Normally a migration system would throw a hissy fit, but this makes it really easy to have flexible control over your db.
manage.py migrate [appname] --fake
If you need to have something custom happen, like say copying the data in a column to another column, because the migration files are just python files it's easily to modify the forward/backwards functions.
Migrating to South after having already deployed an application was rather easy. The latest version 0.6 includes a command for it actually.
manage.py convert_ to _south [appname]
And of course, how could I forget, my favorite feature is the automatic generation of migration files
manage.py schemamigration [appname] [description] --auto
Gotchas
I figured I should add in some tips for mistakes I made when getting started with South. Not everything is 100% intuitive.
After you've run the convert_to_south command on your development database, don't forget to run migrate --fake on your production database, otherwise South will think it's out of date.
If you're creating a new app, you use the --initial flag
Stop using manage.py syncdb. Really.
Editing a model is a 3 step process --
1.) save the model changes
2.) run schemamigration --auto
3.) run migrate to actually commit the changes to the database
Edit -- To clarify the comments below, South was officially voted by the core contributors to not be included with Version 1.2. This was in part because the author of South had requested that it not yet be included. Even still, there is a lot of community support for South, and some reusable app makers are beginning to include South migrations with their app.
Edit #2 -- I made some updates to reflect the new manage.py command structure from the current trunk version of South. "startmigration" has been split to "schemamigration" and "datamigration" depending on what you're doing.
Related
The application I'm developing is starting to need migration for the database schemas. I've thought about django-south, but since I have no experience with that kind of stuff I'm a bit lost, this is my situation:
Development code: latest models, I didn't keep track of what changes I've made to the models.
Production code: running code, has old models. We have the server configured so we can make deployments just with a git pull :)
How can I update the code in production (and the DB) without breaking anything? I saw about the --initial statement but I don't think it works for this case, and also for convert_to_south to fake a migrationhistory, but I still don't get what should I do. Any help please?
Imo it would be better to create versioning for your project and deploy it with something like Fabric. This will contain your production environment nicely.
There's no magic with south, just add south to the installed apps setting and run an initial schemamigration then run a fake migration (migrate <app_name> --fake) so south 'knows' the current state of your models. In future releases (that contain schema changes) you can run schemamigration <app_name> --auto and migrate <app_name> to update the models accordingly.
To keep your migrations in one place use the migrations setting in your settings file:
SOUTH_MIGRATION_MODULES = {
'app_name_1': 'project_name.migrations.page',
'app_name_2': 'project_name.migrations.medialibrary',
}
You'll have to checkout the production version (to get the models back to production state), create an initial migration, copy that migration to your current development branch then create a schemamigration.
I have an app that currently has 35 south migrations. These take a while to go through when setting up a new deployment (we create new deployments often), and the app is continually evolving--adding more migrations. Additionally, the migrations include some potentially complex data migrations and a few custom migrations that break SQLite3 (not a huge problem right now since everything is on Postgres, but its nice to be able to set up a quick test environment), and generally just more things that can go wrong.
All of our deployments and developers are up to date, and I'd like to clear out all of the app's migrations and create a single initial migration (0001) that captures the current state of the app, and then go forward with new migrations from there. I did this a few years ago with a different app, and it worked out nicely, but I've since forgotten what the process was and lost track of the blog post that explained how to do it. Can anyone break down this process for me?
I figured this out (wasn't too bad). To set up the migration reset, I do the following:
rm <app-dir>/migrations/*
python manage.py schemamigration <app-name> --initial
python manage.py migrate <app-name> 0001 --fake --delete-ghost-migrations
I commit the changes to the repository, and then for each deployment of the code elsewhere, run:
python manage.py migrate <app-name> 0001 --fake --delete-ghost-migrations
Make sure you don't add anything new between the time you last migrated everywhere else and you reset things or the new 0001 migration won't match up with the schema!
Caveats: See guettli's comment (and my responses)
I am using South to manage schema and data migrations on my development and production environments. As such I keep my migrations in my git repository so changes I make in development are properly migrated in production.
The apps and projects I am developing are currently private and only developed by me. At some point, I would like to publish/distribute my apps. I am assuming that at that point, I will have a "final" schema and therefor won't "need" South. Since these apps haven't been distributed before (except on my environments), the public version won't need the migrations that I used while developing the apps.
I have a two parter question:
Is it good (or acceptable) to remove or at least "clean up" the migrations based on my assumptions?
What's the best way of doing so? I imagine keeping a branch for the public/distributed base could work, but I'm fairly new to git, so I don't know what my options are.
Thanks,
This is a common point of confusion for people dealing with South and version control. You should, of course, commit migrations with your project, as others will need to run the same migrations themselves. However, you should clean up your migrations before committing them, which is actually deceptively easy.
If it's a brand new app or a brand new project in general:
Rollback the app to "zero":
python manage.py migrate myapp zero
Delete all migrations for the app (Everything in the "migrations" directory except __init__.py).
Generate a new initial migration:
python manage.py schemamigration --initial myapp
If it's an existing app, then the process is largely the same, but you're only going to rollback to just before the first new migration your created. And, then you will simply generate a new auto migration instead of an initial migration. For example, if the app was already at migration 0005 and you create migrations 0006, 0007, and 0008:
Rollback to just before the first migration you created (0006):
python manage.py migrate myapp 0005
Delete all new migrations you created (0006, 0007, and 0008)
Generate a new auto migration:
python manage.py schemamigration --auto myapp
Either way, you'll end up with just one file encapsulating all of your changes instead of multiple files. Then, you commit this to your source control.
Background :-
I am using Django 1.3. We are using South as the module for DB migration and Git SCM.
Problem:-
What is the correct way to deal with the migrations Folder that is formed?
The main problem is I make changes in the DB schema in the development machine, when I upload it to the production server I have to migrate the existing schema. While doing that there is always some issue with the migration files.
Should I just add the migrations folder to the gitignore ? or is there a better way to go about it ?
You should add the migrations folder to your version control system and use the same files for production and development. You may run into some problems on your production system if you introduced your migrations not from the beginning and you have already existing tables.
Therefore you have to fake the first migration, which normally does the same thing as syncdb did when you created your database for the first time.
So when trying to apply migrations for your app for the first time on the production machine, execute manage.py migrate app_name 0001 --fake. This lets South know, that the first migration has already been applied (which already happend with syncdb) and when you run migrate again, it would continue with the following migrations.
I'm using Django South for managing my database schema updates. As I'm currently developing locally, my models are changing a lot, and it's really annoying to change things with South:
$ bin/django schemamigration --auto core
(Please provide a default value for new field...)
...
$ bin/django migrate core
It often takes forever to do simple things like add and remove columns from the database, as South prompts me to provide default values even for columns I'm deleting.
Is there a setting that will cause South to work much more like Hibernates hibernate.hbm2ddl.auto setting and automatically, promptlessly, awesomely update my database schema without any fuss? Is there an alternative library for doing this?
South is really important to me for deployment migrations, but I need something to help me change things fast as I'm rapidly prototyping things.
I use fabric to help with local and production changes. This is a function in my fabfile.py. It helps with any changes I want to make.
def run_local():
"""
Installs requirements, syncs the database, migrates with south, and runs the server.
"""
local('pip install -r conf/requirements.txt')
local('python manage.py syncdb')
local('python manage.py migrate')
local('python manage.py runserver')