What is the best solution if I want to upgrade (alter) my database schema (add new fields to tables by adding them just to Django models) without losing data in these tables? "syncdb" not adding them of course, so I need your advices how to alter tables without deleting them and recreating again with syncdb.
When south isn't an option I just manually write scripts for small changes. and big ones i use
./manage.py dumpdata appname
http://docs.djangoproject.com/en/dev/ref/django-admin/#dumpdata-appname-appname-appname-model
Throw that into a file. Run a regex replace to update any added /removed fields and then a reset of that app is possible. I have to admit i haven't done this in a while but i can get some specific code to do this for you if needed.
it loads back up with loaddata
edit
Django dump data for a single model? This Question is similar and might have the info i was talking about.
Still let me know if you need and i'll dig up my old script (or write out a nice simple one) for you.
UPDATE
./manage.py dumpdata appname --indent=4 > appname.json
#open your fav text editor and do a find/replace
./manage.py reset appname
./manage.py loaddata appname.json
That should do it. When you do a find replace you only need to remove fields that you don't have any more and add fields that aren't nullable. (as a minimum).
Notes: the --indent=4 nicely formats everything for you. It means 4 spaces.
the ./manage.py reset only works in pre django 1.3 (gah!) in django 1.3 you will have to do a ./manage dbshell and drop table. The sql for that is found from command ./manage.py sqlreset appname.
Learning curve...
http://south.aeracode.org/
South was merged into django core at version 1.7.
There is now a native django feature for data migration on schema changes.
Django 1.7 has built-in migrations support.
See https://docs.djangoproject.com/en/dev/releases/1.7/#schema-migrations
Related
I am upgrading Django 1.4 to 1.8, and there are some 3rd party applications whose schema changed dramatically to support the upgrade. I want to reset these applications so I can recreate the corresponding tables from them. In the previous Django iterations, I can do either
./manage.py sqlclear appname
or
./manage.py reset appname
But both sqlclear and reset are already deprecated in Django 1.8. Is there a clean way to do this aside from manually erasing the tables from the database?
As of Django 1.8+ migrate appname zero unapplies all the migrations and deletes the tables docs
Take a look at Django Extensions, which give a slew of useful commands, including:
manage.py reset_db
https://github.com/django-extensions/django-extensions
Good luck!
Drop your Database and create it again its the fastest way !
I have a django 1.8 app working with a db.
I'm trying to change the schema of a table using the built-in migration.
Here are the steps I did:
In my dev invironment, I grabbed the app source and ran
python manage.py sycdb
then I ran
python manage.py loaddata ~/my_data.json
then I modified modes.py. Added a field and renamed a field...all from the same table 'TABLE1' which had no data.
then
python manage.py makemigrations myapp
python manage.py migrate
Error: django.db.utils.OperationalError: table "myapp_someother_table" already exists
then ran
python manage.py migrate --fake-initial
worked!
but when I browsed to the admin page for TABLE1, I get this error:
OperationalError: no such column: myapp_table1.my_new_field_id
I checked the db and yes, there is no such column.
How can I procceed from here? I prefer to fix this via django.
If I fix it straight in the db, then the migration goes out of sync.
Migrations do not automagically see that you have made changes. Migrations detect changes by comparing the current model with the historical model saved in the migration files.
In this case, you didn't have any historical models, since you didn't have any migrations. Django was not able to detect any changes in your models, even though they were different from your database.
The correct way to make changes to your model is to first run manage.py makemigration <my_app>, and then make the changes to your model, followed by another manage.py makemigrations.
You might not be able to do it via pure django and keep your data. I don't have personal experience with south but there are a lot of mentions if this tool. Just in case if nothing else works for you...
Here is what I did to make things work, but there must be a better way so please add more answers/comments...
I deleted the sqlite db and the migration folder
I made the desired changes to model.py
ran syncdb
ran loaddata to load the json data dump that I had saved previously.
just started the dev server
We all love South migration in django, normally you first use convert_to_south and then make model class changes, then run schemamigration and migrate
I am not sure if the following scenario has happened to anyone, the project is not tracked under version control (git stash would be awesome), you changed the model class so much and lost the undo/redo history, and you realised the app has not yet converted to south. Is there a command to remedy this?
Here's one idea.. read all the disclaimers here first, like which databases are supported for detecting FKs.
https://docs.djangoproject.com/en/dev/ref/django-admin/#django-admin-inspectdb
Update: the real answer is to use a mix of the bottom two ideas. Basically, inspectdb is a nice human readable way to figure out the difference between your model and the database. Then, I'd manually build alter table statements to match. Luckily, sqlall outputs the create statements, so most of it will be cut and paste for new fields.
Make a backup of everything.
Copy your models.py somewhere.
Type in python manage.py inspectdb
Find the model you're looking for and paste it into the original models.py
Set up south.
Paste the new / updated models.py back into your models.py
Run schemamigration then migrate.
You can also just do the migrations manually, then convert to south.
python manage.py dbshell
Describe table (depends on your db)
Find missing rows... use alter table statements as necessary. (SQL hints in python manage.py sqlall <myapp>
I'm using Django ( < v1.7), with SQLite3 as the database engine.
I am trying to add a new field to an already existing model's class. This class already has data associated to it.
manage.py syncdb is not adding the new fields to the db.
Is there was a way to:
Add the new Field to the already existing class, and make sure a db entry is allocated to it?
Populate this new field with an empty string for all the already existing db entries?
This answer is still getting visibility but is outdated. Since 1.7 Django ships with a built-in migration system, written by the same author as South, and has deprecated syncdb though it will still work.
You will simply need to run a few commands to automatically add new columns:
python manage.py makemigrations
python manage.py migrate
It will be useful to understand what's happening under the hood, but those are the basics. Please ask new questions to get answers on 1.7 and migrations if you are still reading this old post.
For django < 1.7
syncdb will not add any new columns. See
http://docs.djangoproject.com/en/dev/ref/django-admin/#syncdb
You will have to add them manually. For example,. replace <> with relevant info:
python manage.py dbshell
ALTER TABLE <appname_modelname> ADD COLUMN <column_type> DEFAULT '';
You can see what Django might have done to generate columns on a fresh syncdb by using:
python manage.py sqlall app_name
and copying and pasting ALTER TABLE statements from there.
Otherwise, you can look into third-party apps like Django-South which are database migration tools.
Install south in your django and you can easily handle the existing tables. check this
If you really want to use Django-South, install it on your django, after adding your new fields to your existing model run
python manage.py schemamigration --initial
It will create a file in your project app. then,
python manage.py migrate
thats it your table is altered.
There are other options of migration apps (although South is the most used).
I have used django-evolution for my projects and it was very easy to install and start using.
South seems to be more complete, but for simpler tasks, django-evolution may be suitable.
After making some changes in my models (eg. new field in a model and a new model) what is the best way of reflecting these changes to my populated database?
PS: I wanted to see many solutions in one place rated. Apparently more solutions are already listed here.
Another technique is to use the dumpdata and loaddata arguments to manage.py, killing your database in-between:
python manage.py dumpdata > dump.json
With an external tool, drop any affected tables, or kill the whole db
python manage.py loaddata dump.json
See manage.py docs for more.
I've asked a similar question here and got quite a few answers.
There are quite a lot of ways of doing it, like manually doing the dumping and reloading with SQL, using fixtures or using one of the "emerging" schema-evolution packages for Django:
Django Evolution
South
dmigrations
(there's a DjangoCon video of a panel on schema-evolution in Django where these 3 solutions are discussed)
Depends on the scope of the changes. If it's beyond an ALTER, you're doing major surgery. Make backups of model as well as database so you can go back.
My preference is to put your new (revised, corrected, expanded) model in as a NEW application. It won't have URL's or anything, just a model.
Creating the new model as a new application. Create tests, just to be sure it works.
syncdb to build this interim implementation of the new model.
Write a little one-time utility to query your old model, and load your new model. You might want to try this in pure SQL. I prefer to write a simple query, build and save loop.
After the new model is loaded, you can dump this to a JSON file.
Once you've pulled the data out of your old model, you can rebuild your DB in the preferred new format.
Move the new model into your existing application.
Drop the old versions of the application's tables.
syncdb to build the new tables.
Load the JSON file with the data.
Django now has its own built-in migrations, documented at:
https://docs.djangoproject.com/en/dev/topics/migrations/
Look with manage.py sqlall what the parameters are for the new columns and manually add them in your database with Alter table statements. This way you don't have to redo your database; It requires some SQL knowledge though...
Take a look here (Scroll down to "Making Changes to a Database Schema")
Perform these steps in order may help you:
For more details,
clickhere: http://south.readthedocs.org/en/latest/
1) python manage.py schemamigration apps.appname --initial
Above step creates migration folder as default.
2) python manage.py migrate apps.appname --fake
generates a fake migration.
3) python manage.py schemamigration apps.appname --auto
Then you can add fields as you wish and perform the above command.
4) python manage.py migrate apps.appname
Then migrate the files to the database.