I need to change the name of the options in a model status field, which needs to become
STATUS = Choices(
('option_A', 'Option A'),
('option_B', 'Option B'),
)
Before this change, I had the same options but the names where different. Now I changed everything in the project to respect the new names, but the problem is about updating the database in order to display those new names. I use South for data migrations, and as far as I understand it is fairly straightforward to have it write an automatic migration if you need to add or delete a column in the database but I can't find a way to do this update to my existing column.
I use Django 1.6.
What you're looking for is a datamigration.
$ python manage.py datamigration your_app_label
This will create a new, blank migration in your migration folders. You have to manually create the forwards and backwards methods to change the data how you want it to be:
def forwards(self, orm):
orm.MyModel.objects.filter(status='old_option_A').update(status='option_A')
...
def backwards(self, orm):
# reverse of forwards, or raise an error to prevent backwards migrations
You'll need to use orm.MyModel instead of directly importing the model for this to work.
Then, simply run the migration:
$ python manage.py migrate your_app_label
Be sure to include both the migration and the change in your options in the same commit in your version control system, so these will always be synced (as long as people remember to migrate on a new version).
You could write a quick script to make your changes.Something like this:
>>Model.objects.filter(status = "some_old_value").update(status = "new_value")
Where 'status' is field name.
You can repeat the above step to change any kind of old values to new values in the same model.
Update() is much faster and shouldn't take long time to run on a moderately sized database
https://docs.djangoproject.com/en/dev/topics/db/queries/#updating-multiple-objects-at-once
Also since it would be a one-time script, speed should not be a major issue.
Hope this helps
After you change your fields you should run:
$ python manage.py makemigrations your_app_label
$ python manage.py migrate
(This work for Django 1.7 where South are included into Django package)
In case of Django 1.6:
pip install South
-python manage.py schemamigration your_app_name --initial
-python manage.py migrate your_app_name
make change to your model
manage.py schemamigration your_app_name --auto (migration with your
changes will be added)
python manage.py migrate your_app_name
Also here are first steps tutorial for South Migration.
Related
I run python manage.py makemigrations and I get:
No changes detected
Then, python manage.py migrate and I get:
No migrations to apply.
Then, I try to push the changes to production:
git push heroku master
Everything up-to-date
Then, in production, I repeat the command:
heroku run python manage.py migrate
No migrations to apply.
Just in case, I run makemigrations in production:
heroku run python manage.py makemigrations
No changes detected
WHY then I get a
ProgrammingError at ....
column .... does not exist
"No changes detected" means the database is coherent with the code.
How can I debug this?¡?
I got the same problem (column not exist) but when I try to run migrate not with makemigrations (it is the same issue I believe)
Cause: I removed the migration files and replaced them with single pretending intial migration file 0001 before running the migration for the last change
Solution:
Drop tables involved in that migration of that app (consider a backup workaround if any)
Delete the rows responsible of the migration of that app from the table django_migrations in which migrations are recorded, This is how Django knows which migrations have been applied and which still need to be applied.
And here is how solve this problem:
log in as postgres user (my user is called posgres):
sudo -i -u postgres
Open an sql terminal and connect to your database:
psql -d database_name
List your table and spot the tables related to that app:
\dt
Drop them (consider drop order with relations):
DROP TABLE tablename ;
List migration record, you will see migrations applied classified like so:
id | app | name | applied
--+------+--------+---------+
SELECT * FROM django_migrations;
Delete rows of migrations of that app (you can delete by id or by app, with app don't forget 'quotes'):
DELETE FROM django_migrations WHERE app='yourapp';
log out and run your migrations merely (maybe run makemigrations in your case):
python manage.py migrate --settings=your.settings.module_if_any
Note: it is possible that in your case will not have to drop all the tables of that app and not all the migrations, just the ones of the models causing the problem.
I wish this can help.
Django migrations are recorded in your database under the 'django_migrations' table. This is how Django knows which migrations have been applied and which still need to be applied.
Have a look at django_migrations table in your DB. It may be that something went wrong when your migration was applied. So, delete the row in the table which has the migration file name that is related to that column that 'does not exist'. Then, try to re-run a migration.
Here's what i tried and it worked:
Go and add manually the column to your table
run python manage.py makemigrations
go back drop that column you added
run python manage.py migrate
I had a similar issue - the error message appeared when I clicked on the model on the django-admin site. I solved it by commenting out the field in models.py, then running migrations. Following this I uncommented the field and re ran the migrations. After that the error message disappeared.
My case might be a bit obscure, but if it helps someone, it is worth documenting here.
I was calling a function in one of my migrations, which imported a Model of said migration regularly, i.e.
from myApp.models import ModelX
The only way models should be imported in migrations would be using e.g. RunPython:
def myFunc(apps, schema_editor):
MyModel = apps.get_model('myApp 'MyModel')
and then calling that function like so:
class Migration(migrations.Migration):
operations = [
migrations.RunPython(initialize_mhs, reverse_code=migrations.RunPython.noop),
]
Additionally the original import worked until I modified the model in a later migration, making this error harder to locate.
So, I always run into this sort of problem, so today I decided to try and work it out at the database level. Thing is, I changed a model field name which Django didn't bother reflecting in a migration file. I only found out later when I ran into problems. I later looked at the migration files and discovered there was no migration for that change. But I didn't notice because I made other changes as well, so once I saw a migration file I was happy.
My advice. Create migration for each change one at a time. That way you get to see if it happened or not.
So here's my working through it in MySQL.
open mysql console.
show databases; # see all my dbs. I deleted a few
drop database <db-name>; # if needed
use <db-name>; # the database name for your django project
show tables; # see all tables in the database
DESCRIBE <table-name>; # shows columns in the database
SHOW COLUMNS FROM <db-name>; # same thing as above
ALTER TABLE <table-name> CHANGE <old-column-name> <new-column-name> <col-type>; # now I manually updated my column name
If you're using postgresql, just google the corresponding commands.
The issue was in the Models for me, for some reason Django was adding '_id' to the end of my Foreign Key column. I had to explicitly set the related named to the Foreign Key. Here 'Cards' is the parent table and 'Prices' is the child table.
class Cards(models.Model):
unique_id = models.CharField(primary_key=True, max_length=45)
name = models.CharField(max_length=225)
class Prices(models.Model):
unique_id = models.ForeignKey(Cards, models.DO_NOTHING)
Works after changing to:
class Cards(models.Model):
unique_id = models.CharField(primary_key=True, max_length=45)
name = models.CharField(max_length=225)
class Prices(models.Model):
unique_id = models.ForeignKey(Cards, models.DO_NOTHING, db_column='unique_id')
When I get this error, my extreme way to solve it is to reset my database:
Reset your database
For Postgresql on Heroku:
Heroku > your_app > Resources > database > add-ons > click on your database and open it
For postgresql
settings > Reset database
Delete all files in your_app > migrations > __pycache__ except __init.py__
Delete all files in your_app > migrations except __pycache__ folder and __init.py__
Then run:
python manage.py makemigrations
python manage.py migrate
python manage.py createsuperuser
type in to create your superuser, then run:
python manage.py makemigrations
python manage.py migrate
python manage.py
If you are able to inspect your models from your admin section, then it should be all okay now.
Just remove corresponding row migrations for that model in 'django_migrations' model in database.
And re run python manage.py migrate app_name
I tried all these answers with not much luck! What I did to have this problem solved with no harm was to go back through the migration files and find where the actual model was being created for the first time then manually add the field (in the column not being existed error message). Until if you run makemigrations --dry-run you get/see "No changes detected" and that worked. Basically, in my case, I had to carefully take my desired db changes back in time in proper migration file, rather creating a new migration now at the end of migration dependency chain.
Open the latest py file created after running the makemigrations command inside migrations folder of that particular app.
Inside class Migration there is a list attribute called 'operations'.
Remove the particular elements migrations.RemoveField(...).
Save and run python manage.py migrate.
A easier solution to the problem is to make your models exactly like it is in the migration first. and run python manage.py migrate.
Then revert those changes
Run
python manage.py makemigrations
python manage.py migrate
To check for which migrations are applied and which are not, use -:
python manage.py showmigrations
I solved a similar problem by deleting all migrations files (Don't forget to make a backup) and python manage.py makemigrations all of them into one clean file in development and pulling new files on the server. Before then I had dropped existing tables on the PostgreSQL.
I was getting this error for some reason when Django was looking for a column of type ForeignKey named category_id when the actual name in the database was category. I tried every Django solution I could imagine (renaming field, explicitly setting column name, etc.). I didn't want to drop tables or rows as this was a production database. The solution was simply to rename the column manually using SQL. In my case:
ALTER TABLE table_name
RENAME COLUMN category TO category_id;
Make sure you backup your database, ensure this won't break any other applications consuming that particular table, and consider having a fallback column if necessary.
What helped me in the end was simply dropping the database and creating it again as well as deleting all migrations files (including cache). (only removing migrations file didn't work for me at all)
sudo su - postgres
psql
DROP DATABASE 'yourdatabase';
CREATE DATABASE 'yourdatabase';
GRANT ALL PRIVILEGES ON DATABASE 'yourdatabase' to 'yourdjangouser';
then just
python manage.py makemigrations
python manage.py migrate
python manage.py runserver
If you're in development and you make some examples of data that's not important, this step is beneficial for me: just flush your data, make migrations, and migrate:
python manage.py flush
python manage.py makemigrations
python manage.py migrate
After that, you may create a new database from scratch, I hope this information was helpful.
Solved this issue by running
python manage.py migrate
in Heroku Bash shell
Using django 1.7.7 I want to use django's migration to add or remove a field.
so I modified model.py and ran
python manage.py makemigrations myproj
Migrations for 'myproj':
0001_initial.py:
- Create model Interp
- Create model InterpVersion
python manage.py migrate myproj
Operations to perform:
Apply all migrations: myproj
Running migrations:
Applying myproj.0001_initial... FAKED
python manage.py runserver
Then checked the admin page, it is not updated.
Then I tried removing the migration folder and tried again; the migrate command says there are no migrations to apply.
How can I do the migration?
Note: I want to use the new technique using django's migration not the old south approach.
Make sure that the migrations/ folder contains a __init__.py file
Lost half an hour over that.
Deleting the migration directory is never a good idea, because Django then loses track of which migration has been applied and which not (and once an app is deployed somewhere it can become quite difficult to get things back in sync).
Disclaimer: whenever things like that occur it is best to backup the database if it contains anything valuable. If in early development it is not necessary, but once things on the backend get out of sync there is a chance of things getting worse. :-)
To recover, you could try resetting your models to match exactly what they were before you have added/removed the fields. Then you can run
$ python manage.py makemigrations myproj
which will lead to an initial migration (0001_initial...). Then you can tell Django to fake that migration, which means to tell it to set its internal counter to this 0001_initial:
With Django 1.7:
$ python manage.py migrate myproj
With Django >= 1.8:
$ python manage.py migrate myproj --fake-initial
Now, try to change your model and run makemigrations again. It should now create a 0002_foobar migration that you could run as expected.
In my case, the migrations were not being reflected in mysql database. I manually removed the row of 'myapp'(in your case 'myproj') from the table 'django_migrations' in mysql database and ran the same commands again for migration.
Most of the above solutions would help in the issue, however, I wanted to point out another possible (albeit rare) possibility that the allow_migrate method of database router may be returning False when it should have returned None.
Django has a setting DATABASE_ROUTERS which will be used to determine which database to use when performing a database query.
From the docs:
if you want to implement more interesting database allocation behaviors, you can define and install your own database routers.
A database router class implements up to four methods:
db_for_read(model, **hints)
db_for_write(model, **hints)
allow_relation(obj1, obj2, **hints)
allow_migrate(db, app_label, model_name=None, **hints)
From the documentation:
allow_migrate(db, app_label, model_name=None, **hints)
Determine if the migration operation is allowed to run on the database with alias db. Return True if the operation should run, False if it shouldn’t run, or None if the router has no opinion.
It is possible that one of the database routers in sequence is returning False for the migration that you're trying to run, in which case the particular operation will not be run.
I find Django migrations a bit of a mystery, and tend to prefer external tools (liquibase, for example).
However, I just ran into this "No migrations to apply" problem as well. I also tried removing the migrations folder, which doesn't help.
If you've already removed the migrations folder, here's an approach that worked for me.
First, generate the new "clean" migrations:
$ python manage.py makemigrations foo
Migrations for 'foo':
dashboard/foo/migrations/0001_initial.py
- Create model Foo
- Create model Bar
Then look at the SQL and see if it looks reasonable:
$ python manage.py sqlmigrate foo 0001
BEGIN;
--
-- Create model Foo
--
CREATE TABLE "foo" ("id" serial NOT NULL PRIMARY KEY, ... "created_at" timestamp with time zone NOT NULL, "updated_at" timestamp with time zone NOT NULL);
CREATE INDEX "..." ON "foo" (...);
COMMIT;
Then apply execute that same SQL on your database.
I'm using Postgres but it will be similar for other engines.
One way is to write the content to a file:
$ python manage.py sqlmigrate foo 0001 > foo.sql
$ psql dbname username < foo.sql
BEGIN
CREATE TABLE
CREATE INDEX
COMMIT
Another is pipe the SQL directly:
$ python manage.py sqlmigrate foo 0001 | psql dbname username
Or copy and paste it, etc.
pip install django-extensions
and add this in the install app of settings.py
INSTALLED_APPS = [
'django_extensions'
]
Then run
python ./manage.py reset_db
Then run migrations again
python manage.py makemigrations
python manage.py migrate
Now, run migrations for your installed apps
python manage.py makemigrations your_app_name
python manage.py migrtate your_app_name
Done! See Your Database...
In addition to the other answers, make sure that in models.py, you have managed = True in each table's meta
You can remove your db
python manage.py makemigrations
python manage.py migrate
python manage.py migrate --run-syncdb
and see your data base this is working :)
Similar to Andrew E above but with a few changes especially where you haven't deleted the migrations folder in your quest to resolve the issue
1 - In your intact migration folder just examine the 000*.py files counting from the highest down to initial.py till you find the one where your Model is defined, say 0002_entry.py
2 - python manage.py sqlmigrate app-name 0002 > 0002_sql.txt to capture the SQL commands
3 - Edit this file to ensure there are no hard CR/LFs and the ALTER, CREATE INDEX commands are each on own single line
4 - Log into your DB (I have Postgres) and run these commands
In Database delete row myproj from the table django_migrations.
Delete all migration files in the migrations folder.
Then run python manage.py makemigrations and python manage.py migrate commands.
I just added the num object, and tried adding it to my admin, but I get the following error: Exception Value:
no such column: game_riddle.num
Here is the class:
class Riddle(models.Model):
"""represents a riddle, comprising a question and hints"""
world = models.ForeignKey(World)
question = models.TextField()
num = models.IntegerField()
I have a small database and the last time I added the object I had to run:sqlclear and then syncdb
How can I fix my bug without clearing the database?
Read this documention: http://south.readthedocs.org/en/0.7.6/installation.html
easy_install South
After install, put 'south' in your installed_app in settings
Don't forget to sync: python manage.py syncdb
Run this command: python manage.py schemamigration app_name --auto
The migrate: python manage.py migrate
That's it. Next if you add new model or have changes. Just do step 5 and 6 and your model will be updated.
You will need to use a migration tool such as South:
South brings migrations to Django applications. Its main objectives
are to provide a simple, stable and database-independent migration
layer to prevent all the hassle schema changes over time bring to your
Django applications.
I am using Django with Postgres and Heroku. and I have a post table with looks something like this.
class Post( models.Model ):
.
.
content1 = models.CharField(max_length=1000)
.
One of my friends told me that limit is too small, now I want to change that maxlength to 10000. One way I can think of alternating the table by creating a new column named content2 then copying the content from content1, then deleting content1. I am fairly new to this and was wondering what was the best way to approach this problem.
Also is 10000 is a good length for a blog post? :D
You can either use dumpdata to save the data to a json-file and reload it after you re-create it, you can also manually ALTER the table with the new parameters. Or you can use south.
South provides databas migrations (as its called to change a table layout) for django.
pip install south
Add south to INSTALLED_APPS then run
python manage.py schemamigration --initial your_app_name
python manage.py migrate --fake your_app_name
Do your changes to the models.py, then run
python manage.py schemamigration --auto your_app_name
python manage.py migrate your_app_name
Now if you have the same database layout to change on several machines, check in the created migrations-files to git and run the "migrate" commands on all the machines that needs to update.
I am trying to get started with South. I had an existing database and I added South (syncdb, schemamigration --initial).
Then, I updated models.py to add a field and ran ./manage.py schemamigration myapp --auto. It seemed to find the field and said I could apply this with ./manage.py migrate myapp. But, doing that gave the error:
django.db.utils.DatabaseError: table "myapp_tablename" already exists
tablename is the first table listed in models.py.
I am running Django 1.2, South 0.7
since you already have the tables created in the database, you just need to run the initial migration as fake
./manage.py migrate myapp --fake
make sure that the schema of models is same as schema of tables in database.
Although the table "myapp_tablename" already exists error stop raising
after I did ./manage.py migrate myapp --fake, the DatabaseError shows
no such column: myapp_mymodel.added_field.
Got exactly the same problem!
1.Firstly check the migration number which is causing this. Lets assume it is: 0010.
2.You need to:
./manage.py schemamigration myapp --add-field MyModel.added_field
./manage.py migrate myapp
if there is more than one field missing you have to repeat it for each field.
3.Now you land with a bunch of new migrations so remove their files from myapp/migrations (0011 and further if you needed to add multiple fields).
4.Run this:
./manage.py migrate myapp 0010
Now try ./manage.py migrate myapp
If it doesn't fail you're ready. Just doublecheck if any field's aren't missing.
EDIT:
This problem can also occur when you have a production database for which you install South and the first, initial migration created in other enviorment duplicates what you already have in your db. The solution is much easier here:
Fake the first migration:
./manage migrate myapp 0001 --fake
Roll with the rest of migrations:
./manage migrate myapp
When I ran into this error, it had a different cause.
In my case South had somehow left in my DB a temporary empty table, which is used in _remake_table(). Probably I had aborted a migration in a way I shouldn't have. In any case, each subsequent new migration, when it called _remake_table(), was throwing the error sqlite3.pypysqlite2.dbapi2.OperationalError: table "_south_new_myapp_mymodel" already exists, because it did already exist and wasn't supposed to be there.
The _south_new bit looked odd to me, so I browsed my DB, saw the table _south_new_myapp_mymodel, scratched my head, looked at South's source, decided it was junk, dropped the table, and all was well.
If you have problems with your models not matching your database, like #pielgrzym, and you want to automatically migrate the database to match the latest models.py file (and erase any data that won't be recreated by fixtures during migrate):
manage.py schemamigration myapp --initial
manage.py migrate myapp --fake
manage.py migrate myapp zero
manage.py migrate myapp
This will only delete and recreate database tables that exist in your latest models.py file, so you may have garbage tables in your database from previous syncdbs or migrates. To get rid of those, precede all these migrations with:
manage.py sqlclear myapp | manage.py sqlshell
And if that still leaves some CRUFT lying around in your database then you'll have to do an inspectdb and create the models.py file from that (for the tables and app that you want to clear) before doing the sqlclear and then restore your original models.py before creating the --initial migration and migrating to it. All this to avoid messing around with the particular flavor of SQL that your database needs.
Perform these steps in order may help you:
1) python manage.py schemamigration apps.appname --initial
Above step creates migration folder as default.
2) python manage.py migrate apps.appname --fake
generates a fake migration.
3) python manage.py schemamigration apps.appname --auto
Then you can add fields as you wish and perform the above command.
4) python manage.py migrate apps.appname
If you have an existing database and app you can use the south conversion command
./manage.py convert_to_south myapp
This has to be applied before you do any changes to what is already in the database.
The convert_to_south command only works entirely on the first machine you run it on. Once you’ve committed the initial migrations it made into your VCS, you’ll have to run ./manage.py migrate myapp 0001 --fake on every machine that has a copy of the codebase (make sure they were up-to-date with models and schema first).
ref: http://south.readthedocs.org/en/latest/convertinganapp.html
As temporary solution, you can comment the Table creation in the migration script.
class Migration(migrations.Migration):
dependencies = [
(...)
]
operations = [
#migrations.CreateModel(
# name='TABLE',
# fields=[
# ....
# ....
# ],
#),
....
....
Or
If the existing table contains no rows (empty), then consider deleting the table like below. (This fix is recommended only if the table contains no rows). Also make sure this operation before the createModel operation.
class Migration(migrations.Migration):
dependencies = [
(...),
]
operations = [
migrations.RunSQL("DROP TABLE myapp_tablename;")
]
One more solution(maybe a temporary solution).
$ python manage.py sqlmigrate APP_NAME MIGRATION_NAME
eg.,.
$ python manage.py sqlmigrate users 0029_auto_20170310_1117
This will list all the migrations in raw sql queries. You can pick the queries which you want to run avoiding the part which creates the existing table