I am using South to manage schema and data migrations on my development and production environments. As such I keep my migrations in my git repository so changes I make in development are properly migrated in production.
The apps and projects I am developing are currently private and only developed by me. At some point, I would like to publish/distribute my apps. I am assuming that at that point, I will have a "final" schema and therefor won't "need" South. Since these apps haven't been distributed before (except on my environments), the public version won't need the migrations that I used while developing the apps.
I have a two parter question:
Is it good (or acceptable) to remove or at least "clean up" the migrations based on my assumptions?
What's the best way of doing so? I imagine keeping a branch for the public/distributed base could work, but I'm fairly new to git, so I don't know what my options are.
Thanks,
This is a common point of confusion for people dealing with South and version control. You should, of course, commit migrations with your project, as others will need to run the same migrations themselves. However, you should clean up your migrations before committing them, which is actually deceptively easy.
If it's a brand new app or a brand new project in general:
Rollback the app to "zero":
python manage.py migrate myapp zero
Delete all migrations for the app (Everything in the "migrations" directory except __init__.py).
Generate a new initial migration:
python manage.py schemamigration --initial myapp
If it's an existing app, then the process is largely the same, but you're only going to rollback to just before the first new migration your created. And, then you will simply generate a new auto migration instead of an initial migration. For example, if the app was already at migration 0005 and you create migrations 0006, 0007, and 0008:
Rollback to just before the first migration you created (0006):
python manage.py migrate myapp 0005
Delete all new migrations you created (0006, 0007, and 0008)
Generate a new auto migration:
python manage.py schemamigration --auto myapp
Either way, you'll end up with just one file encapsulating all of your changes instead of multiple files. Then, you commit this to your source control.
Related
I have a few projects with lots of South migrations, including ones that contain a fair amount of custom SQL that need to be run in a specific order. After upgrading to Django 1.7, this is the recommendation on how to convert a project to use South (from the Django documentation):
If you already have pre-existing migrations created with South, then the upgrade process to use django.db.migrations is quite simple:
Ensure all installs are fully up-to-date with their migrations.
Remove 'south' from INSTALLED_APPS.
Delete all your (numbered) migration files, but not the directory or __init__.py - make sure you remove the .pyc files too.
Run python manage.py makemigrations. Django should see the empty migration directories and make new initial migrations in the new format.
Run python manage.py migrate. Django will see that the tables for the initial migrations already exist and mark them as applied without running them.
In short, "wipe your existing migrations and Django will take care of the rest".
What is not mentioned here is what to do when existing South migrations don't only consist of model changes, but instead contain direct SQL, data migrations, etc, that need to be run in order. In this case, the auto-generated Django migrations will miss a lot of things, since not all of these changes are obvious from introspecting a models file.
Ideally, one would be able to run the existing migrations using South, and then have Django migrations take over. What might be the best way to go about this? If this is not possible or very much not recommended, what is the best alternative?
Maybe this post can help you. Essentially you have to:
Change your current migration directory from 'migrations' to 'south_migrations'
Update your settings with this line
SOUTH_MIGRATION_MODULES = {
'your_app': 'your_project.your_app.south_migrations',
}
The application I'm developing is starting to need migration for the database schemas. I've thought about django-south, but since I have no experience with that kind of stuff I'm a bit lost, this is my situation:
Development code: latest models, I didn't keep track of what changes I've made to the models.
Production code: running code, has old models. We have the server configured so we can make deployments just with a git pull :)
How can I update the code in production (and the DB) without breaking anything? I saw about the --initial statement but I don't think it works for this case, and also for convert_to_south to fake a migrationhistory, but I still don't get what should I do. Any help please?
Imo it would be better to create versioning for your project and deploy it with something like Fabric. This will contain your production environment nicely.
There's no magic with south, just add south to the installed apps setting and run an initial schemamigration then run a fake migration (migrate <app_name> --fake) so south 'knows' the current state of your models. In future releases (that contain schema changes) you can run schemamigration <app_name> --auto and migrate <app_name> to update the models accordingly.
To keep your migrations in one place use the migrations setting in your settings file:
SOUTH_MIGRATION_MODULES = {
'app_name_1': 'project_name.migrations.page',
'app_name_2': 'project_name.migrations.medialibrary',
}
You'll have to checkout the production version (to get the models back to production state), create an initial migration, copy that migration to your current development branch then create a schemamigration.
I have an app that currently has 35 south migrations. These take a while to go through when setting up a new deployment (we create new deployments often), and the app is continually evolving--adding more migrations. Additionally, the migrations include some potentially complex data migrations and a few custom migrations that break SQLite3 (not a huge problem right now since everything is on Postgres, but its nice to be able to set up a quick test environment), and generally just more things that can go wrong.
All of our deployments and developers are up to date, and I'd like to clear out all of the app's migrations and create a single initial migration (0001) that captures the current state of the app, and then go forward with new migrations from there. I did this a few years ago with a different app, and it worked out nicely, but I've since forgotten what the process was and lost track of the blog post that explained how to do it. Can anyone break down this process for me?
I figured this out (wasn't too bad). To set up the migration reset, I do the following:
rm <app-dir>/migrations/*
python manage.py schemamigration <app-name> --initial
python manage.py migrate <app-name> 0001 --fake --delete-ghost-migrations
I commit the changes to the repository, and then for each deployment of the code elsewhere, run:
python manage.py migrate <app-name> 0001 --fake --delete-ghost-migrations
Make sure you don't add anything new between the time you last migrated everywhere else and you reset things or the new 0001 migration won't match up with the schema!
Caveats: See guettli's comment (and my responses)
Background :-
I am using Django 1.3. We are using South as the module for DB migration and Git SCM.
Problem:-
What is the correct way to deal with the migrations Folder that is formed?
The main problem is I make changes in the DB schema in the development machine, when I upload it to the production server I have to migrate the existing schema. While doing that there is always some issue with the migration files.
Should I just add the migrations folder to the gitignore ? or is there a better way to go about it ?
You should add the migrations folder to your version control system and use the same files for production and development. You may run into some problems on your production system if you introduced your migrations not from the beginning and you have already existing tables.
Therefore you have to fake the first migration, which normally does the same thing as syncdb did when you created your database for the first time.
So when trying to apply migrations for your app for the first time on the production machine, execute manage.py migrate app_name 0001 --fake. This lets South know, that the first migration has already been applied (which already happend with syncdb) and when you run migrate again, it would continue with the following migrations.
I changed a lot in my models.py, including deleting a lot of fields, and renaming a couple of classes. schemamigration --auto worked fine, but trying migrate threw a bunch of errors.
All my code is currently in development so I don't mind too much losing the data. So I want South to "unconvert" or "unmanage" an app so I can rebuild all the tables with syncdb again.
Or I could delete all migration list and do schemamigration --initial again.
Yes, just delete the migrations and run schemamigration --initial again. You should do that anyways as normal course before moving to production. If you've already gone to production at least once, don't delete all the migrations -- just the ones you've created in the current development cycle and then run schemamigration --auto to get just one migration instead of the potential multiple ones.
FWIW, to "unconvert" an app using South, you merely delete the "migrations" directory, but in this scenario, there's no need.
UPDATE
It was pointed out that if you have already migrated your app, and you delete all the migrations and generate a single new one, South will complain about migrations still in the database. The actual process you should follow is:
Rollback to the just before the newest migration you created in the current development cycle. For instance, if you were already at 0005 and you created three new migrations for the development work you were doing (now at 0008), you would rollback to 0005. If all of the migrations are new, you rollback to zero:
python manage.py migrate yourapp zero
Delete all of the migrations you're going to merge. In the above example, that would be 0006, 0007, and 0008, or for a new app, everything in the migrations directory but __init__.py.
Generate a new migration to cover the ones you just deleted. If it's a new app, use --initial, or if it was a pre-existing app, use --auto.
python manage.py schemamigration --initial yourapp
Migrate
python manage.py migrate yourapp