My deployment strategy looks like this (using Fabric):
create a new virtualenv
deploy new code in new virtualenv
show a maintenance page
copy the current db to new db
migrate new db
point new code to new db
symlink current virtualenv to new venv
restart services
remove maintenance page
I want to iterate fast. Now, most of the code changes do not contain migrations. Also, the db is growing, so there is much overhead created by copying the database everytime I deploy a (mostly small) change. To avoid copying the database I want to check whether there are migrations that need to be deployed (prior to step 4). If there are no migrations, I can go straight from step 2 to step 7. If there are, I will follow all the steps. For this, I need to check programmatically whether there are migrations that need to be deployed. How can I do this?
In step 2 while deploying the new code, you could deploy a script which when run on the server will detect if there are new migrations.
Example code is as follows:
# copied mostly from south.management.commands.migrate
from south import migration
from south.models import MigrationHistory
apps = list(migration.all_migrations())
applied_migrations = MigrationHistory.objects.filter(app_name__in=[app.app_label() for app in apps])
applied_migrations = ['%s.%s' % (mi.app_name,mi.migration) for mi in applied_migrations]
num_new_migrations = 0
for app in apps:
for migration in app:
if migration.app_label() + "." + migration.name() not in applied_migrations:
num_new_migrations = num_new_migrations + 1
return num_new_migrations
If you wrap the above code up in a script, your fabric deployment script can use the run operation to get the number of new migrations.
If this returns zero, then you can skip the steps associated with copying the database.
./manage.py migrate --all --merge --list | grep "( )"
Will tell and show you which migrations. If you want a return code or count, use wc.
This has the advantages of not copying and pasting code like the accepted answer (violating DRY), and also if the internal south api changes your code will still work.
UPDATE:
Django 1.7 changed the output to use bracket instead of parenthesis and Django 1.8 introduced a showmigration command:
./manage.py showmigrations --list | grep '[ ]'
dalore's answer updated for Django 1.7+
./manage.py migrate --list | grep "\[ ]"
If you just want the count then:
./manage.py migrate --list | grep "\[ ]" | wc -l
Why are you moving the databases around? The whole point of migrations is to apply the changes you made in development to your production database in place.
Your steps should really be:
create a new virtualenv
deploy new code in new virtualenv
show a maintenance page
migrate new db
symlink current virtualenv to new venv
restart services
remove maintenance page
And the migration step doesn't take that long if there's no actual new migrations to run. It'll just run through each app saying it's already up to date.
If you're copying the database to have a backup, that's something that should be running anyways on cron or something, not as part of your deployment.
Actually, I'm confused on the creating a new virtualenv each time too. The normalized (read: most typical) deployment is:
deploy new code
migrate db
restart services
If you want to add back in the maintenance page stuff, you can, but the process takes only a minute or two total.
Related
I am developing Django Wagtail application on my local machine connected to a local postgres server.
I have a test server and a production server.
However when I develop locally and try upload it there is always some issue with makemigration and migrate e.g. KeyError etc.
What are the best practices of ensuring I do not get run into these issues? What files do I need to port across?
so ill tell you what i do and what most of the companies that i worked as a django developer did and i can tell you by experience that worked pretty well.
First containerize your application, this will make your life much more easy and you will remove external influence in your code, also will get you an easy way to reproduce your environment.
Your Dockerfile should be from some python image and should do 3 basically things:
Install your requirements dependencies
Run the python manage.py migrate --noinput command
Run a http server such as gunicorn with gunicorn -c /gunicorn.py wsgi:application
You ill do the makemigration in your local machine and make sure that everything is working before commit then to the repo.
In your gunicorn.py you ill put your settings to run the app such as the number of CPU, the binding port, the folder that your app is, something like:
import os
import multiprocessing
# Chdir to specified directory before apps loading.
# https://docs.gunicorn.org/en/stable/settings.html#chdir
chdir = '/app/'
# Bind the application on localhost both on ipv6 and ipv4 interfaces.
# https://docs.gunicorn.org/en/stable/settings.html#bind
bind = '0.0.0.0:8000'
Second containerize your other stuff, for example the postgres database, the redis (for cache), a connection pooler for the database depending on the size of your application.
Its highly recommend that you have a step in the pipeline to do tests, they need to run before everything, maybe just after lint
Ok what now? now you need a way to deploy that stuff, the best for that scenario is: pull your image to github registry, and you can add a tag to that for example:
IMAGE_ID=ghcr.io/${{ github.repository_owner }}/$IMAGE_NAME
# Change all uppercase to lowercase
IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
docker tag $IMAGE_NAME $IMAGE_ID:staging
docker push $IMAGE_ID:staging
This can be add in a github action in the build step for example.
After having your new code in a new image inside github you just need to update the current one, this can be done by creaaing a script to do it in the server and running that script from github action, is something like:
docker pull ghcr.io/${{ github.repository_owner }}/$IMAGE_NAME
echo 'Restarting Application...'
docker stop {YOUR_CONTAINER} && docker up -d
sudo systemctl restart nginx
echo 'Cleaning old images'
sudo docker system prune -af
You can see that i create the image with a staging tag, you can create a rule in github actions for example to trigger that action when you create a new release for example, and create another action to be trigger in every new commit and build/deploy for a dev tag.
For the migration problem, the first thing is, when your application go live squash every migration to the first one (you can drop the database and all the migration then create the database and run the makemigration command again to reach this), so you can have a clean migration in the server. Never creates unnecessary relation between the tables, prefer always doing cached properties instead of add new columns, use UUID for unique ids, and try to not do breaking changes in the database, its hard but if you plan the database before is not so difficult to do.
Hit me if you have any questions. A lot of the stuff that i said can be done in a lot of other platforms such as gitlab, travis, circle ci, but i use the github action in the example because i think is more simple to picture.
EDIT:
I forgot to tell you to have a cron in your server doing backups of your databases, the migrate command ill apply the changes only after the verification but if something else break the database this can save your life.
I have a django app connected to postgresql database (RDS instance running on AWS)
Recently, I added new fields to my database tables, but forgot to run makemigrations locally and pushed the code to the server. The server went down since changes were not applied, and in order to fix the problem quickly I decided to run makemigrations and migrate directly in docker container of the application, I thought it would generate migration files, but unfortunately, it didn't. The migration was applied on database level, but there is no migration file of the changes.
When running python manage.py showmigrations in the container, it outputs:
redirect
[X] 0001_initial
[X] 0002_redirect_clicks
[X] 0003_redirect_name
But in database, running SELECT * FROM django_migrations; tells that there is one more migration:
103 | redirect | 0004_auto_20210707_2039 | 2021-07-07 14:39:19.058233+00
What would you advise? I thought of running makemigrations locally, generate the file and push it to vcs, and on the server apply the changes using migrate --fake.
I don't know what this does exactly and I am too scared to do it without guidance, any help would be appreciated
Current solution (not ideal):
I generated the migration file locally, pushed it to VCS, and on remote server i had to unapply all migrations for the app using python manage.py migrate <app_name> zero (this caused a data loss for the database tables in the app, so think twice before doing this) and then applied them again, but now with the migration file generated in the 1st step. Everything is back to normal, but obviously there is a better solution
I am migrating a project from Django 1.1 to Django 3.0 I am done with the project. When I am dumping the production dump to my local in the newly converted project I get "Table already exists".
Here's what I am doing.
mysql> create database xyx;
docker exec -i <container-hash> mysql -u<user> -p<password> xyx < dbdump.sql
then I run the migrate, as I have had to do some changes to the previously given models.
./manage.py migrate --fake-initial
this is the output I get
_mysql.connection.query(self, query)
django.db.utils.OperationalError: (1050, "Table 'city' already exists")
So, what to do ?
Alright boys and girls, here's the approach I followed to solve this problem.
I dumped the entire database.
docker exec -i <container-hash> mysql -u<username> -p<password> <dbname> < dump.sql
Now I listed all the migrations I made using
./manage.py showmigrations <app-name>
This will give me the list of all the migrations I have applied, now from inspecting the migrations, I realized that from the 7th migration to the 30th migration I had done my changes.
Here's the tedious part which any sys admin can write a script to do in less than 4 lines of bash script. You can generate the raw SQL of any migration with this command.
./manage.py sqlmigrate <app-name> <migration-name> > changes-i-made.sql
Now that I have created my changes-i-made.sql file I'll need to run this script 22 more times but with >> otherwise everytime you run the command with a single > it will keep overwriting your changes file.
Now once all of your migration changes are recorded inside a file, open up your sql shell connect to the database and start pasting the changes or do some sql magic to pick all the changes directly from the file.
Once you're done go ahead and fake all the migrations, cause you don't need Django to do them you already did.
./manage.py migrate --fake
and then login to your production instance and get ready to fuck with your senior team lead who said you couldn't do it.
I just checked to see if this approach is working and the future migrations will be working, so I created one and everything works like a breeze.
I have deployed a django application to heroku but I need to reset those migrations. Is that possible without deleting the entire project and redeploying it? I have some test data in that database that I would prefer not to have to enter all over again.
I'd like to delete all migration files and create new ones. Thing is, I deleted all migration files from my local machine and created new ones so now the migration files on my local machine are all 001. Pushing that to heroku says there were no changes because the 001 migration files already exist on there. Only deleting them would work. Basically something similar to
find . -path "*/migrations/*.py" -not -name "__init__.py" -delete
find . -path "*/migrations/*.pyc" -delete
but for heroku.
Pushing to Heroku is part of the solution, but not all of it. This updates the migration files, but not the django_migrations table in your database.
If you still have the old code running on Heroku:
Back your database up
Reverse all of your migrations:
heroku run python manage.py migrate appname zero
Then deploy your new code:
git push heroku master
Then run the new migrations:
heroku run python manage.py migrate
If you've already deployed the new code to Heroku:
Back your database up
Manually delete your app's tables from your database, as well as rows from the django_migrations table where the app matches your app name e.g. via heroku pg:psql (this may get tricky if you have other apps installed and inter-app dependencies)
Run your new migrations:
heroku run python migrate
Another, probably safer, option if you've already deployed the new code is to rollback to a version that reflects your database schema, then use option 1 above. In this case you can rollback again to the new version instead of doing a regular release.
Yes it can be done by two ways.
You can generate fixture for your current database schema if no such column is deleted or altered.
You can dump your database than just remove database from your db server and migrate again! Than load dumped data into fresh db, this will keep your previous data safe.
I've accumulated quite a few migrations using South (0.7) and Django (1.1.2) which are starting to consume quite a bit of time in my unit tests. I would like to reset the baseline and start a fresh set of migrations. I've reviewed the South documentation, done the usual Google/Stackoverflow searching (e.g. "django south (reset OR delete OR remove) migration history") and haven't found anything obvious.
One approach I've contemplated would involve "starting over" by "removing" South or "clearing" the history manually (e.g. clear the db table, remove migration files from the migrations director) and just re-run,
./manage.py schemamigration southtut --initial
So, if anyone has done this before and has some tips/suggestions they would be greatly appreciated.
If you need to selectively (for just one app) reset migrations that are taking too long, this worked for me.
rm <app-dir>/migrations/*
python manage.py schemamigration <app-name> --initial
python manage.py migrate <app-name> 0001 --fake --delete-ghost-migrations
Don't forget to manually restore any dependencies on other apps by adding lines like depends_on = (("<other_app_name>", "0001_initial"),("<yet_another_app_name>", "0001_initial")) to your <app-dir>/migrations/0001_initial.py file, as the first attribute in your migration class just below class Migration(SchemaMigration):.
You can then ./manage.py migrate <app-name> --fake --delete-ghost-migrations on other environments, per this SO answer. Of course if you fake the delete or fake the migrate zero you'll need to manually delete any left-over db tables with a migration like this.
A more nuclear option is to ./manage.py migrate --fake --delete-ghost-migrations on the live deployment server followed by a [my]sqldump. Then pipe that dump into [my]sql on the environments where you need the migrated, fully-populated db. South sacrilege, I know, but worked for me.
EDIT - I'm putting a comment below at the top of this as it's important to read it before the > accepted answer that follows #andybak
#Dominique: Your advice regarding manage.py reset south is dangerous
and may destroy the database if there are any third party apps using
south in the project, as pointed out by #thnee below. Since your
answer has so many upvotes I'd really appreciate it if you could edit
it and add at least a warning about this, or (even better) change it
to reflect #hobs approach (which is just as convenient, but doesn't
affect other apps) - thanks! – chrisv Mar 26 '13 at 9:09
Accepted answer follows below:
First, an answer by the South author:
As long as you take care to do it on all deployments simultaneously, there shouldn't be any problem with this. Personally, I'd do:
rm -r appname/migrations/
./manage.py reset south
./manage.py convert_to_south appname
(Notice that the “reset south” part clears migration records for ALL apps, so make sure you either run the other two lines for all apps or delete selectively).
The convert_to_south call at the end makes a new migration and fake-applies it (since your database already has the corresponding tables). There's no need to drop all the app tables during the process.
Here's what I'm doing on my dev + production server when I need to get rid of all these unneeded dev migrations:
Make sure we have the same DB schema on both sides
delete every migrations folder on both sides
run ./manage.py reset south (as the post says) on both sides = clears the south table *
run ./manage.py convert_to_south on both sides (faking 0001 migration)
then I can re-start to make migrations and push the migrations folders on my server
* except if you want to clean only one app among others, if so you'll need to edit your south_history table and delete only the entries about your app.
Thanks to the answers by Dominique Guardiola and hobs, it helped me solve a hard problem.
However there are a couple of issues with the solution, here is my take on it.
Using manage.py reset south is not a good idea if you have any third party apps that uses South, for example django-cms (basically everything uses South).
reset south will delete all migration history for all apps that you have installed.
Now consider that you upgrade to the latest version of django-cms, it will contain new migrations like 0009_do_something.py. South will surely be confused when you try to run that migration without having 0001 through 0008 in the migration history.
It is much better/safer to selectively reset only the apps that you are maintaining.
First of all, make sure that your apps don't have any desync between migrations on disk, and migrations that have been executed on the database. Otherwise there will be headache.
1. Delete migration history for my apps
sql> delete from south_migrationhistory where app_name = 'my_app';
2. Delete migrations for my apps
$ rm -rf my_app/migrations/
3. Create new initial migrations for my apps
$ ./manage.py schemamigration --initial my_app
4. Fake execute the initial migrations for my apps
This inserts the migrations into south_migrationhistory without touching actual tables:
$ ./manage.py migrate --fake my_app
Step 3 and 4 is actually just a longer variant of manage.py convert_to_south my_app, but I prefer that extra control, in such delicate situation as modifying the production database.
Like thnee (see her answer), we're using a gentler approach to the South author's (Andrew Godwin) suggestion quoted elsewhere here, and we're separating what we do with the code base from what we do to the database, during deployment, because we need the deployments to be repeatable:
What we do in the code:
# Remove all the migrations from the app
$ rm -fR appname/migrations
# Make the first migration (don't touch the database)
$ ./manage.py schemamigration appname --initial
What we do to the database once that code is deployed
# Fake the migration history, wiping out the rest
$ ./manage.py migrate appname --fake --delete-ghost-migrations
If you are just working on the dev machine, I wrote a management command that does pretty much what Dominique suggested.
http://balzerg.blogspot.co.il/2012/09/django-app-reset-with-south.html
In contrast of the south author suggestion, this will NOT HARM other installed apps using south.
Following is only if you want to reset all apps. Please backup your all databases prior to that work. Also note down your depends_on in initial files if there are any.
For once:
(1) find . -type d -name migrations -exec git rm -rf '{}' \;
(2) find . -type d -name migrations -exec rm -rf '{}' \;
(3) ./manage.py schemamigration <APP_NAME> --initial
(4) [GIT COMMIT]
Test bootstrapping your project before push. Then, for each local/remote machine, apply following:
(5) [GIT PULL]
(6) ./manage.py reset south
(7) ./manage.py migrate --fake
Do initial (3) for each app you want to re-involve. Note that, reset (6) will delete only migration history, therefore not harmful to libraries. Fake migrations (7) will put back migration history of any 3rd party apps installed.
delete necessary file from app folder
instance path
cd /usr/local/lib/python2.7/dist-packages/wiki/south_migrations
wiki -is my app