Noob here... :)
I'm working on a small and personal project that is already in "production", but development still is under way. In the last weeks I've managed to handle the updates in a hacky way. Usually, I make a dump of the (still small) database into json files, separed by app or sometimes by table, drop everything in the database, implement the model's changes in json level trough scripts, syncdb a new database, and put everything back on. I known, it's dumb, but I'm lacking knowledge of a better alternative. So, now that I'm borderline insane with this strategy I come to you guys.
I've looked into South, but I failed to understand how exactly is it's workflow regarding the Data migration (in opposition of it's schema migration that is obvious).
So, how do you guys do it?
Thanks in advance.
South creates python scripts. So you can use South to create schema migrations, and then change these scripts to include your own data migration.
If you just add models and fields you don't need to do this, you can just use plain South.
Related
I have setup a git repository and cloned open source code which I am planning on modifying from github to start development. I committed the codebase in our repository.
I now have added few users and posts and other "stuff" into database.
I want to commit this change as well so that my teammate can check out and we have same settings and database throughout.
Is this possible by using south migrations? i.e will database bot contents and schema be in sync as well?
I have the project where I am writing the code as well the actual app. Should I commit both of them.
What should the github repository look like after doing the "right" thing
Data and database structure
This is possible using south migrations, data migrations and fixtures.
The easiest way for development is to just use a SQLite database, which is a binary file that you can commit. The test_project of django-autocomplete-light demonstrates such a possibility: http://django-autocomplete-light.readthedocs.org/en/latest/demo.html
you must use south anyway !
Apps in the project repo
I think you should keep apps as small and loosely coupled as possible.
If sound, make another repository and python package for the app:
In some cases it makes sense at the beginning, ie. a "blog" app that you know you will reuse,
In some cases it makes sense later, ie. you tought your app was really project specific but then you want to reuse it in another project,
In some cases it never makes sense (ie. the app is only useful to that particular project).
Best practice
As for best practices, there is http://12factor.net, http://lincolnloop.com/django-best-practices/ and pinax projects which are really interresting.
If you're going to reuse and extend external apps, then maybe this article on best practice reusing apps can help.
If there is data that every developer needs, these can be provided by initial_fixtures. I have just begun to use south, but I think it is mainly for migrations, (changing your models, without having to delete your data, and resync) not for sharing data. Schema should always be in sync using south, because a developer can pull the south migration, and apply it.
I recently added a new field to one of my models and forgot to add the appropriate column to the table in the database. I have test cases that test adding a new instance of this model and changing an existing instance. Neither of these test cases failed. Yet when I try to change an instance with the live site I get
DatabaseError no such column
I have made some attempts to detect this error from within a TestCase but no such luck.
Any help is greatly appreciated.
The problem is that you (I really, really hope) don't use your production database for testing with, so all that you could detect is that the column doesn't exist on your test database, which is (presumably) recreated from scratch based on your model definitions, not that the column is missing from your production database.
A better approach to this problem is to use a migration tool like South, and automate the deployment process so that migrations are run as new code is deployed.
This will only work for small(ish) sites - you might find naively running migrations causes pain if you've got a high-traffic site. If you're in that situation, you may find David Cramer's write-up on schema changes informative.
Unfortunetly, using syncdb command is not enough to update the database.
You can insert the fields manually with your database management system or use a solution like South.
Use following link it as very well explanation for your problem.
http://amionrails.wordpress.com/2013/11/17/django-databaseerror-no-such-column-error/
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
update django database to reflect changes in existing models
I've used Django in the past and one of the frustrations I've had with it as an ORM tools is the inability to update an existing database with changes in the model. (Hibernate does this very well and makes things really easy for updating and heavily modifying a model and applying this to an existing database.) Is there a way to do this without wiping the database every time? It gets really old having to regenerate admin users and sites after every change in the model which I'd like to play with.
You will want to look into South. It provides a migrations system to migrate both schema changes as well as data from one version to the next.
It's quite powerful and the vast majority of changes can be handled simple by going
manage.py schemamigration --auto
manage.py migrate
The auto functionality does have it limits, and especially if the change is going to be run on a production system eventually you should check the code --auto generated to be sure it's doing what you expect.
South has a great guide to getting started and is well documented. You can find it at http://south.aeracode.org
No.
As the documentation of syncdb command states:
Syncdb will not alter existing tables
syncdb will only create tables
for models which have not yet been installed. It will never issue
ALTER TABLE statements to match changes made to a model class after
installation. Changes to model classes and database schemas often
involve some form of ambiguity and, in those cases, Django would have
to guess at the correct changes to make. There is a risk that critical
data would be lost in the process.
If you have made changes to a model and wish to alter the database
tables to match, use the sql command to display the new SQL structure
and compare that to your existing table schema to work out the
changes.
South seems to be how most people solve this problem, but a really quick and easy way to do this is to change the db directly through your database's interactive shell. Just launch your db shell (usually just dbshell) and manually alter, add, drop the fields and tables you need changed using your db syntax.
You may want to run manage.py sqlall appname to see the sql statements Django would run if it was creating the updated table, and then use those to alter the database tables and fields as required.
The Making Changes to a Database Schema section of the Django book has a few examples of how to do this: http://www.djangobook.com/en/1.0/chapter05/
I manually go into the database - whatever that may be for you: MySQL, PostgreSQL, etc. - to change database info, and then I adjust the models.py accordingly for reference. I know there is Django South, but I didn't want to bother with using another 3rd party application.
Due to my little confidence with Django and my sheer horror at the thought of seriously messing up my beautiful project, I shall ask for proper advice/instructions here.
The database that my Django project is sitting on top of has been changed (a few field types have been changed) and my models are now out-of-sync. Funnily enough, my Django still works (God knows how) but I still want to update the models. How do I go about doing this the proper way. Thank you very much indeed in advance.
Marked as answered. My actual discover was:
./manage.py inspectdb > <file>
//Hands you all the tables from the database.
//Then you update the models accordingly.
SIMPLE! :)
It's probably a bit late, but you might want to take a look at South, which is a migrations system for Django.
The normal practice for your situation would be to run manage.py reset appname, where appname is the name of the app which contains the models you've changed. You'll obviously want to dump the data in the affected tables first (find out what tables are going to be affected by running manage.py sqlreset appname).
Finally, it's quite possible your site is still running happily because you've not restarted the webserver (I'm assuming you're talking about a production environment, the development server reloads most changes automatically).
If you've already made the changes to the live database, you can probably just change the models and restart your webserver.
As long as your Field names match between the database and the models you shouldn't have any issues.
That being said, it is a much better idea to use a migration tool like south (as Dominic suggested already)
Overview
I'm building a website in django. I need to allow people to begin to add flatpages, and set some settings in the admin. These changes should be definitive, since that information comes from the client. However, I'm also developing the backend, and as such will am creating and migrating tables. I push these changes to the hub.
Tools
django
git
south
postgres
Problem
How can I ensure that I get the database changes from the online site down to me on my lappy, and also how can I push my database changes up to the live site, so that we have a minimum of co-ordination needed? I am familiar with git hooks, so that option is in play.
Addendum:
I guess I know which tables can be modified via the admin. There should not be much overlap really. As I consider further, the danger really is me pushing data that would overwrite something they have done.
Thanks.
For getting your schema changes up to the server, just use South carefully. If you modify any table they might have data in, make sure you write both a schema migration and as necessary a data migration to preserve the sense of their data.
For getting their updated data back down to you (which doesn't seem critical, but might be nice to work with up-to-date test data as you're developing), I generally just use Django fixtures and the dumpdata and loaddata commands. It's easy enough to dump a fixture and commit it to your repo, then a loaddata on your end.
You could try using git hooks to automate some of this, but if you want automation I do recommend trying something like Fabric instead. Much of this stuff doesn't need to be run every single time you push/pull (in particular, I usually wouldn't want to dump a new data fixture that frequently).
You should probably take a look at South:
http://south.aeracode.org/
It seems to me that you could probably create a git hook that triggers off South if you are doing some sort of continuous integration system.
Otherwise, every time you do a push you will have to manually execute the migration steps yourself. Don't forget to put up the "site is under maintenance" message. ;)
I recommend that you use mk-table-sync to pull changes from live server to your laptop.
mk-table-sync takes a lot of parameters so you can automate this process by using fabric. You would basically create a fabric function that executes mk-table-sync on each tablet that you want to pull from the server.
This means that you can not make dabatase changes yourself, because they will be overwritten by the pull.
The only changes that you would be making to the live database are using South. You would push the code to the server and then run migrate to update the database schema.