We all love South migration in django, normally you first use convert_to_south and then make model class changes, then run schemamigration and migrate
I am not sure if the following scenario has happened to anyone, the project is not tracked under version control (git stash would be awesome), you changed the model class so much and lost the undo/redo history, and you realised the app has not yet converted to south. Is there a command to remedy this?
Here's one idea.. read all the disclaimers here first, like which databases are supported for detecting FKs.
https://docs.djangoproject.com/en/dev/ref/django-admin/#django-admin-inspectdb
Update: the real answer is to use a mix of the bottom two ideas. Basically, inspectdb is a nice human readable way to figure out the difference between your model and the database. Then, I'd manually build alter table statements to match. Luckily, sqlall outputs the create statements, so most of it will be cut and paste for new fields.
Make a backup of everything.
Copy your models.py somewhere.
Type in python manage.py inspectdb
Find the model you're looking for and paste it into the original models.py
Set up south.
Paste the new / updated models.py back into your models.py
Run schemamigration then migrate.
You can also just do the migrations manually, then convert to south.
python manage.py dbshell
Describe table (depends on your db)
Find missing rows... use alter table statements as necessary. (SQL hints in python manage.py sqlall <myapp>
Related
I am trying to simply add a boolean field to a model in my database following the simple rules:
- add field to model
- python manage.py makemigrations app
- python manage.py migrate app
Works all but 99% of the time. So during the second step (makemigrations), with my newly added field in my model raring to go, i get an error:
django.db.utils.OperationalError: (1054, "Unknown column 'model.field' in 'field list'")
Excellent. its not letting me make migrations by adding a new field..... because it cant find the field that I am trying to newly add... makes perfect sense!
Anyway, I have gone as far as deleting all my migrations, removing my new field, making migrations again, migrating... all fine - so now i have only 1 migration file (0001)...
Follow the same steps as above... ERROR
Am i missing something ridiculous here? I mean, adding a field to a model is very simple, and I have done it probably 1000 times. Why does Django tease me so
EDIT: Answer:
OK I have done it.
After deleting the migrations file, truncating the migrations table and migrating with 0001_initial.py, I made an empty migrations file (python manage.py makemigrations --empty app) and added the field in manually.... then I migrated and it worked! Baffled at this to be honest, but at least the change has been made:
Delete all migration files
Truncate the django_migrations table
comment the new boolean field
run python manage.py makemigrations
run python manage.py migrate --fake
run python manage.py makemigrations --empty app
add field in manually to the empty migrations file in the operations:
migrations.AddField('modelName', 'fieldName', models.BooleanField(default=False)),
run python manage.py migrate
uncomment the new boolean field so it represents what you made in the migrations operations
Disclaimer- Follow this only on local system, for production do understand the steps and then execute.
Kindly follow these steps:
Delete all migration files
Truncate the django_migrations table
comment the new boolean field
run python manage.py makemigrations
run python manage.py migrate --fake
Uncomment the boolean field
run python manage.py makemigrations
run python manage.py migrate
Generally these steps solve any kind of migration problem
An another reason can be if you are using django_rest_framework then the serialiser too needs to be updated as per your model change.
This is an issue that has persisted with me and lead me down a lot of rabbit holes dropping tables etc. A simple solution I have found is answering "N" when django asks you if you are renaming a field of that model (when running makemigrations). What that then essentially performs is a deletion of your previous field and creates the new field. Be careful as you may lose data on existing field so this works with fields which are either new or relatively easy to 'refill' their data required. You may need to run --fake if you get an error with regards to not being able to 'drop field' when migrating after makemigrations. If you would like to go back to your migration state before the problem you may need to delete the migrations you have done since then and try the above solution.
Update:
I did the above for a Boolean field and my data was kept. Even though I said N, it seems as if it is essentially a renaming.
Another source of the problem: I have a ModelForm based on the model. The ModelForm definition defines two extra fields. I had to comment out those two fields from the Form definition before doing the migration steps. Once the migration was done, I un-commented the two fields.
I have trouble with django model migrations.
I have some models in my app, and I already have some data inside.
When I added some models in my application, and I run makemigrations, the app report that there is no change.
I know that sometimes some errors came when migrate, so I delete django_migrations table in my database and run makemigrations again, and now program found my new fields.
The problem now is that if I run migrate system tell me that some tables already exist. (Which is ok and correct, because they do). I don't want to delete those tables, because I have data already inside.
I can't run migrate --fake, because program will think that I already have all the tables, which is not true.
So, I am looking for a way to tell the program : run migration, if table exist skip it. (--fake it)
Another question is why is this happening to me, that makemigrations don't recognise my changes (some cache problems,...)?
How about doing this way ?
python manage.py makemigrations
(Skip this step if you have already have migration file ready)
It will create migrations for that package lets say with a name like 0001_initial.py
Edit the file manually so that you delete all models there except that was already created in database.
Now you do a fake migration. This will sync your database with models.
python manage.py migrate --fake
Then run makemigrations again to have rest of the tables created along with a new migration file.
python manage.py makemigrations
Regarding your other question, Why makemigrations didn't recogonize your models can be because of reasons like:
Migrations for those changes are already there in some migration file.
You missed it to mention package_name in INSTALLED_APPS but i believe you did it here.
every time you make changes to your models, try these steps :
python manage.py makemigrations [your app name]
then:
python manage.py migrate
it should work fine. but remember if you have already data(rows) in your tables you should specify the default value for each one the queries.
if not, Django prompt you to specify the default value for them
or you can just try to use blank=True or null=True in your fields like below :
website = models.URLField(blank=True)
the possible cause or this is that you have another migration in the same folder starts with the same prefix... maybe you make another migration on the same table on another branch or commit so it's saved to the db with the same prefix ie: 00010_migration_from_commit_#10, 00010_migration_from_commit_#11
the solution for this is to rename the migration file like this 00011_migration_from_commit_#11
I tried to edit the related migration file and commented the part where it creates that specific column, then ran python manage.py migrate
The main problem is the existing tables that are disabling the migration of the new tables, so the solution is straight-forward:
** Try to add managed = False to the existing dB so it won't be detected by migrate
** Redo it for all existing old tables :
class Meta:
managed=False
It sometimes gets boring when we have a lot of tables in the same application but it works perfectly!
I have a django 1.8 app working with a db.
I'm trying to change the schema of a table using the built-in migration.
Here are the steps I did:
In my dev invironment, I grabbed the app source and ran
python manage.py sycdb
then I ran
python manage.py loaddata ~/my_data.json
then I modified modes.py. Added a field and renamed a field...all from the same table 'TABLE1' which had no data.
then
python manage.py makemigrations myapp
python manage.py migrate
Error: django.db.utils.OperationalError: table "myapp_someother_table" already exists
then ran
python manage.py migrate --fake-initial
worked!
but when I browsed to the admin page for TABLE1, I get this error:
OperationalError: no such column: myapp_table1.my_new_field_id
I checked the db and yes, there is no such column.
How can I procceed from here? I prefer to fix this via django.
If I fix it straight in the db, then the migration goes out of sync.
Migrations do not automagically see that you have made changes. Migrations detect changes by comparing the current model with the historical model saved in the migration files.
In this case, you didn't have any historical models, since you didn't have any migrations. Django was not able to detect any changes in your models, even though they were different from your database.
The correct way to make changes to your model is to first run manage.py makemigration <my_app>, and then make the changes to your model, followed by another manage.py makemigrations.
You might not be able to do it via pure django and keep your data. I don't have personal experience with south but there are a lot of mentions if this tool. Just in case if nothing else works for you...
Here is what I did to make things work, but there must be a better way so please add more answers/comments...
I deleted the sqlite db and the migration folder
I made the desired changes to model.py
ran syncdb
ran loaddata to load the json data dump that I had saved previously.
just started the dev server
What is the best solution if I want to upgrade (alter) my database schema (add new fields to tables by adding them just to Django models) without losing data in these tables? "syncdb" not adding them of course, so I need your advices how to alter tables without deleting them and recreating again with syncdb.
When south isn't an option I just manually write scripts for small changes. and big ones i use
./manage.py dumpdata appname
http://docs.djangoproject.com/en/dev/ref/django-admin/#dumpdata-appname-appname-appname-model
Throw that into a file. Run a regex replace to update any added /removed fields and then a reset of that app is possible. I have to admit i haven't done this in a while but i can get some specific code to do this for you if needed.
it loads back up with loaddata
edit
Django dump data for a single model? This Question is similar and might have the info i was talking about.
Still let me know if you need and i'll dig up my old script (or write out a nice simple one) for you.
UPDATE
./manage.py dumpdata appname --indent=4 > appname.json
#open your fav text editor and do a find/replace
./manage.py reset appname
./manage.py loaddata appname.json
That should do it. When you do a find replace you only need to remove fields that you don't have any more and add fields that aren't nullable. (as a minimum).
Notes: the --indent=4 nicely formats everything for you. It means 4 spaces.
the ./manage.py reset only works in pre django 1.3 (gah!) in django 1.3 you will have to do a ./manage dbshell and drop table. The sql for that is found from command ./manage.py sqlreset appname.
Learning curve...
http://south.aeracode.org/
South was merged into django core at version 1.7.
There is now a native django feature for data migration on schema changes.
Django 1.7 has built-in migrations support.
See https://docs.djangoproject.com/en/dev/releases/1.7/#schema-migrations
After making some changes in my models (eg. new field in a model and a new model) what is the best way of reflecting these changes to my populated database?
PS: I wanted to see many solutions in one place rated. Apparently more solutions are already listed here.
Another technique is to use the dumpdata and loaddata arguments to manage.py, killing your database in-between:
python manage.py dumpdata > dump.json
With an external tool, drop any affected tables, or kill the whole db
python manage.py loaddata dump.json
See manage.py docs for more.
I've asked a similar question here and got quite a few answers.
There are quite a lot of ways of doing it, like manually doing the dumping and reloading with SQL, using fixtures or using one of the "emerging" schema-evolution packages for Django:
Django Evolution
South
dmigrations
(there's a DjangoCon video of a panel on schema-evolution in Django where these 3 solutions are discussed)
Depends on the scope of the changes. If it's beyond an ALTER, you're doing major surgery. Make backups of model as well as database so you can go back.
My preference is to put your new (revised, corrected, expanded) model in as a NEW application. It won't have URL's or anything, just a model.
Creating the new model as a new application. Create tests, just to be sure it works.
syncdb to build this interim implementation of the new model.
Write a little one-time utility to query your old model, and load your new model. You might want to try this in pure SQL. I prefer to write a simple query, build and save loop.
After the new model is loaded, you can dump this to a JSON file.
Once you've pulled the data out of your old model, you can rebuild your DB in the preferred new format.
Move the new model into your existing application.
Drop the old versions of the application's tables.
syncdb to build the new tables.
Load the JSON file with the data.
Django now has its own built-in migrations, documented at:
https://docs.djangoproject.com/en/dev/topics/migrations/
Look with manage.py sqlall what the parameters are for the new columns and manually add them in your database with Alter table statements. This way you don't have to redo your database; It requires some SQL knowledge though...
Take a look here (Scroll down to "Making Changes to a Database Schema")
Perform these steps in order may help you:
For more details,
clickhere: http://south.readthedocs.org/en/latest/
1) python manage.py schemamigration apps.appname --initial
Above step creates migration folder as default.
2) python manage.py migrate apps.appname --fake
generates a fake migration.
3) python manage.py schemamigration apps.appname --auto
Then you can add fields as you wish and perform the above command.
4) python manage.py migrate apps.appname
Then migrate the files to the database.