I have recently recovered from a rather evil server crash where my hosts (hilariously) deleted the entire server in a failed rack migration. Fortunately I had taken some backups and the server is back up and running. I've been meaning to start using south and feel that this could potentially help me perform upgrades in the future.
Currently I develop on a local machine and then push changes out to my production server. Previously this had been a pretty painful experience, but south looks like it could make it all easier.
As it stands, my development machine has some changes which haven't made it as far as the production server but I'd like to roll those changes out. Both development machine and production server are linked to a subversion account which stores all the code.
Questions..
How do I go about installing south on both machines.
How do I use it to migrate changes on the dev machine to production.
Quick example..
For example, would this work:
Install south for each app on the production server and create initial migrations.
Commit changes to svn
Dump database from production server
Check out latest changes from svn on local machine.
At this point the dev machine should match the production server right? Now to update my changes from dev to production server:
Copy my local changes over the checked out copy on the dev machine and use south to create a migration for my updates.
Commit changes to .svn
Run the migration on the production server to update the changes to the schema and migrate data to the new database
Will this work?
take a look at
http://south.readthedocs.org/en/latest/convertinganapp.html#converting-an-app
other than that it fairly simple:
on dev:
modify models
./manage.py schemamigration app --auto
./manage.py migrate
check thigs work
commit
on prod:
checkout
./manage.py migrate
restart server
Related
I've built an app using Django etc. I ran into a whole bunch of problems with database on Heroku after accidentally pushing to production before running heroku run python manage.py makemigrations and heroku run python manage.py migrate.
I decided to just completely destroy the database currently on Heroku and want to recommit my app's models to heroku. My hope is to completely start from a clean slate.
I just want to make sure I do this correctly so I'm asking for clear instructions about how to recommit my app's database to Heroku after destroying the database in the dashboard.
Current error I get when I open my app online is:
settings.DATABASES is improperly configured. Please supply the NAME or OPTIONS['service'] value.
Thanks!
If you completely removed your database you'll have to provision another one. Assuming you are using Heroku Postgres¹, you can do that using the Heroku CLI like so:
heroku addons:create heroku-postgresql:hobby-dev
Here we are asking Heroku to provision the heroku-postgresql add-on using the free hobby-dev tier. Modify as needed.
You can also provision add-ons using the Dashboard if you prefer.
As part of the provisioning process, Heroku Postgres will automatically set a new DATABASE_URL for your app. As long as your project was using this variable before you deleted your old database, it should find the new database without requiring any changes.
¹If you were using a different database add-on the solution should be more or less the same. Provision a new instance of the add-on, let it set its environment variable, and if your app was using that variable before it should pick it up on its own.
I'm having trouble with my Django-app that's been deployed.
It was working fine but I had to do a minor modification (augmented the max_length) of a Charfield of some model. I did migrations and everything was working fine in the local version.
Then I commited the changes without a problem and the mentioned field of the web version now accepts more characters, as expected, but whenever I click the save button a Server Error rises.
I assume I have to do some kind of migration/DB update for the web version but I don't seem to find how.
(I'm working with Django 1.11, postgresql 9.6, and DigitalOcean).
EDIT
I've just realized that the 'minor modification' also included a field deletion in the model.
Short answer
You have to run
python manage.py migrate
on the server, too. Before you do that, make sure all migration scripts you have locally are also present on the server.
Explanation
After changing the model, you probably locally ran
python manage.py makemigrations
This creates migration scripts that'll transform database schema accordingly. Hopefully, you've committed these newly created scripts to Git, together with the changed model. (If not, you can still do so now.)
after running makemigrations (either before or after committing, that shouldn't matter), you've probably locally ran
python manage.py migrate
This applies the migration scripts to the database that haven't been applied to it, yet. (The information which ones have already been applied is stored in the database itself.)
You probably (and hopefully) haven't checked in your local database into Git, so when you pushed your tracked changes to a remote repo and pulled them down on your server (or however else the new Git revisions got there), the changes to the server database haven't happened, yet. So you have to repeat the last local step (migrate) on the server.
Further reading
For more information, refer to the Django 1.11 documentation w.r.t. migrations. (You can e.g. limit migration creation or migration application to a single Django app, instead of the whole Django project.) To get the grip of these things, I can recomment the free Django Girls tutorial.
We have migrated the Django 1.4 application to Django 1.8 successfully. The Django 1.4 version of applicaiton is still in use in production until we go live with Django 1.8. The problem is that lots of data have been updated on production server which needs to be migrated to version 1.8. Is there any way I can migrate the data from database of 1.4 to 1.8 except manually doing that? Note that the model/database columns are different in both the version.
Can anybody suggest some good options ?
Thanks.
Required pre-reading
Django migrations:
https://docs.djangoproject.com/en/1.8/ref/django-admin/#makemigrations-app-label
Assuming you were using South:
https://docs.djangoproject.com/en/1.8/topics/migrations/#upgrading-from-south
Getting Started
Firstly dump your local database I prefer using mysql/postgres/whatever docs for this rather than using ./manage.py dumpdata.
You will also want to dump your production database as well just for safekeeping.
Next in your local environment I would dump the database and create a new database.
Then I would test that all your migrations actually work on a blank database.
These are the instructions for django 1.8
./manage.py makemirgrations
./manage.py migrate
That will help show up if any migrations are in an inconsistent state operating from a blank slate. If you encounter any errors they should be fixed first.
Given that works, now I would test that your migrations actually work against your production data.
So drop your local database, create a fresh one then load the production dump in.
If the tables are already configured correctly (i.e. they are in an up to date state in your production database) then you will need to fake all migrations.
./manage.py migrate --fake <appname>
However given that you have changed some models since upgrading to 1.8 in your local environment, then you might need to fake only some of the migrations. This could be the tricky part depending on the timing of when you upgraded and when you created the migrations.
Because django 1.7 will just create an initial migration for each app, you may need to actually break up the migration for some apps. That is, instead of 0001_initial you might need to manually break up that migration into 2 components:
1. a migration to match the current state of your production database
2. a migration to match any additional changes you have made to your model since then.
One way to do this is to checkout your first commit once you had django 1.8 working correctly locally then run
./manage.py makemigrations
then commit that
then go forward to your latest commit then run
./manage.py makemigrations
Now you should have 2+ migrations in each app that you have modified since upgrading to django 1.8.
Then you can fake initial on those apps that have 2+ new migrations for django 1.8
./manage.py migrate --fake-initial app1 app2
and the rest just
./manage.py migrate app3 app4
Now run your tests to confirm everything is working locally.
If you have changed migrations you will again want to test locally against a blank database to test that they work smoothly
Once that is working, record the 'migrate' commands that you used - then deploy your app to production and run just those migration commands once you have upgraded to django 1.8 on your server.
After Successful Completion
Take new dumps of your local and production databases
Uninstall South (assuming you had it installed before) from local and production environments
I'm sure there are a couple of holes in the above but hopefully that gives you the gist of what you need to do.
I am using South for database migration for a Django project. And I was wondering if it is a good idea to commit the migration scripts that were generated by my dev server to the repository & then reusing it on the production server?
Yes. That is the point of migrations to allow you to develop and test database schema changes and then deploy in test and production in a reproducible way.
I wouldn't consider it reuse so much as developed and tested in development and then deployed on production.
Make sure you also develop and test backward migrations to ensure you can retreat.
We are starting a web project with Django (and are pretty new to this), and I was wondering what would been the best way to setup a productive development environment.
Here is some info:
We are 2 developers working on the project
We have a development/production server hosted on Webfactional (using an apache server)
The DB (MySQL) is hosted on this server
We are both using Eclipse with Pydev
The project is hosted on a Github repository
So far, we didn't setup any local development server, we are syncing modifications to the server through Github. But it is not really convenient...
We were thinking of setting up apache servers locally, that uses the remote database, and only sync once in a while.
Do you think it would be better?
Do you have other ideas/additional tips?
Thanks!
Don't try and share servers and databases while you're developing. You'll only get confused.
Each developer should have a local copy of MySQL on their own machine, and run the development server as mipadi recommends. Manage db schema changes via South, and keep data in fixtures or use South's data migrations. Each developer should commit to their local version of the git repository, but only push changes to Github when the particular task is complete and working (or, better, use a remote branch so that changes are synched, but only merge back to master when complete).
A good idea is to have a continuous integration server like Hudson/Jenkins that, after each commit to master, runs the tests and - if they pass - builds an up-to-date version of the site, which you can use for functional/user testing.
Edit to add There's no relationship between the development server and any particular database backend. As I recommend above, it is quite simple to install MySQL or PostgreSQL on your local machine, and use the development server against that. I have been working this way for several years, after initially encountering some of the issues you were worried about when switching between sqlite3 and the production MySQL db.
Django has its own development server that you can use for local testing.