Adding a column in South gives error - django

Question

I added a couple of columns and fields to my Django model and was able to run South commands such as:
$ python manage.py schemamigration [model_name] --auto
$ python manage.py migrate
fine on local machine without any error. However, when I switch to the server, pulled all new migration+code files and stuff there and did this:
$ python manage.py migrate
It gives me the following error pasted below. I use version MySQL 5.6 (with InnoDB), if that is going to be a useful info. I've tried looking around online and Stackoverflow to find out how to solve this (e.g., this post comes pretty close to helping me understand the issue, but is not exactly what I'm facing...), but I cannot seem to find the solution to this.
What should I try now to successfully proceed the migration?
[root#server: t4s (develop)]$ python manage.py migrate
Running migrations for accounts:
- Migrating forwards to 0012_auto__add_timelimit.
> accounts:0005_auto__add_twilioaccount
> accounts:0006_auto__add_field_twilioaccount_created_at__add_field_twilioaccount_crea
> accounts:0007_auto__del_twilioaccount
> accounts:0008_auto__add_twilioaccount
> accounts:0009_auto__chg_field_twilioaccount_created_by__add_unique_twilioaccount_cre
> accounts:0010_auto__del_twilioaccount
> accounts:0011_auto__add_twilioaccount
> accounts:0012_auto__add_timelimit
- Loading initial data for accounts.
Installed 0 object(s) from 0 fixture(s)
Running migrations for smsmessages:
- Nothing to migrate.
- Loading initial data for smsmessages.
Installed 0 object(s) from 0 fixture(s)
Running migrations for organizations:
- Nothing to migrate.
- Loading initial data for organizations.
Installed 0 object(s) from 0 fixture(s)
Running migrations for campaigns:
- Migrating forwards to 0003_auto__add_field_taskqueue_twilio.
> campaigns:0003_auto__add_field_taskqueue_twilio
! Error found during real run of migration! Aborting.
! Since you have a database that does not support running
! schema-altering statements in transactions, we have had
! to leave it in an interim state between migrations.
! You *might* be able to recover with: - no dry run output for delete_foreign_key() due to dynamic DDL, sorry
= ALTER TABLE `campaigns_taskqueue` DROP COLUMN `twilio_id` CASCADE; []
! The South developers regret this has happened, and would
! like to gently persuade you to consider a slightly
! easier-to-deal-with DBMS (one that supports DDL transactions)
! NOTE: The error which caused the migration to fail is further up.
Error in migration: campaigns:0003_auto__add_field_taskqueue_twilio
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/lib/python2.6/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
utility.execute()
File "/usr/lib/python2.6/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/lib/python2.6/site-packages/django/core/management/base.py", line 242, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/lib/python2.6/site-packages/django/core/management/base.py", line 285, in execute
output = self.handle(*args, **options)
File "/usr/lib64/python2.6/site-packages/south/management/commands/migrate.py", line 111, in handle
ignore_ghosts = ignore_ghosts,
File "/usr/lib64/python2.6/site-packages/south/migration/__init__.py", line 220, in migrate_app
success = migrator.migrate_many(target, workplan, database)
File "/usr/lib64/python2.6/site-packages/south/migration/migrators.py", line 254, in migrate_many
result = migrator.__class__.migrate_many(migrator, target, migrations, database)
File "/usr/lib64/python2.6/site-packages/south/migration/migrators.py", line 329, in migrate_many
result = self.migrate(migration, database)
File "/usr/lib64/python2.6/site-packages/south/migration/migrators.py", line 133, in migrate
result = self.run(migration, database)
File "/usr/lib64/python2.6/site-packages/south/migration/migrators.py", line 114, in run
return self.run_migration(migration, database)
File "/usr/lib64/python2.6/site-packages/south/migration/migrators.py", line 8
migration_function()
File "/usr/lib64/python2.6/site-packages/south/migration/migrators.py", line 6
return (lambda: direction(orm))
File "/opt/dev/t4sfeature/t4sdev/t4s/campaigns/migrations/0003_auto__add_field
keep_default=False)
File "/usr/lib64/python2.6/site-packages/south/db/generic.py", line 47, in _ca
return func(self, table, *args, **opts)
File "/usr/lib64/python2.6/site-packages/south/db/generic.py", line 418, in ad
self.execute(sql)
File "/usr/lib64/python2.6/site-packages/south/db/generic.py", line 282, in ex
cursor.execute(sql, params)
File "/usr/lib/python2.6/site-packages/django/db/backends/util.py", line 69, i
return super(CursorDebugWrapper, self).execute(sql, params)
File "/usr/lib/python2.6/site-packages/django/db/backends/util.py", line 53, i
return self.cursor.execute(sql, params)
File "/usr/lib/python2.6/site-packages/django/db/backends/mysql/base.py", line
return self.cursor.execute(query, args)
File "/usr/lib64/python2.6/site-packages/MySQLdb/cursors.py", line 175, in exe
if not self._defer_warnings: self._warning_check()
File "/usr/lib64/python2.6/site-packages/MySQLdb/cursors.py", line 89, in _war
warn(w[-1], self.Warning, 3)
_mysql_exceptions.Warning: TIME/TIMESTAMP/DATETIME columns of old format have been upgraded
When I tried running python manage.py migrate again, I got a slightly different error as follows.
[root#server: t4s (develop)]$ python manage.py migrate
Running migrations for accounts:
- Nothing to migrate.
- Loading initial data for accounts.
Installed 0 object(s) from 0 fixture(s)
Running migrations for smsmessages:
- Nothing to migrate.
- Loading initial data for smsmessages.
Installed 0 object(s) from 0 fixture(s)
Running migrations for organizations:
- Nothing to migrate.
- Loading initial data for organizations.
Installed 0 object(s) from 0 fixture(s)
Running migrations for campaigns:
- Migrating forwards to 0003_auto__add_field_taskqueue_twilio.
> campaigns:0003_auto__add_field_taskqueue_twilio
FATAL ERROR - The following SQL query failed: ALTER TABLE `campaigns_taskqueue` ADD COLUMN `twilio_id` integer NULL;
The error was: (1060, "Duplicate column name 'twilio_id'")
! Error found during real run of migration! Aborting.
! Since you have a database that does not support running
! schema-altering statements in transactions, we have had
! to leave it in an interim state between migrations.
! You *might* be able to recover with: - no dry run output for delete_foreign_key() due to dynamic DDL, sorry
= ALTER TABLE `campaigns_taskqueue` DROP COLUMN `twilio_id` CASCADE; []
! The South developers regret this has happened, and would
! like to gently persuade you to consider a slightly
! easier-to-deal-with DBMS (one that supports DDL transactions)
! NOTE: The error which caused the migration to fail is further up.
Error in migration: campaigns:0003_auto__add_field_taskqueue_twilio
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/lib/python2.6/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
utility.execute()
File "/usr/lib/python2.6/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/lib/python2.6/site-packages/django/core/management/base.py", line 242, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/lib/python2.6/site-packages/django/core/management/base.py", line 285, in execute
output = self.handle(*args, **options)
File "/usr/lib64/python2.6/site-packages/south/management/commands/migrate.py", line 111, in handle
ignore_ghosts = ignore_ghosts,
File "/usr/lib64/python2.6/site-packages/south/migration/__init__.py", line 220, in migrate_app
success = migrator.migrate_many(target, workplan, database)
File "/usr/lib64/python2.6/site-packages/south/migration/migrators.py", line 254, in migrate_many
result = migrator.__class__.migrate_many(migrator, target, migrations, database)
File "/usr/lib64/python2.6/site-packages/south/migration/migrators.py", line 329, in migrate_many
result = self.migrate(migration, database)
File "/usr/lib64/python2.6/site-packages/south/migration/migrators.py", line 133, in migrate
result = self.run(migration, database)
File "/usr/lib64/python2.6/site-packages/south/migration/migrators.py", line 114, in run
return self.run_migration(migration, database)
File "/usr/lib64/python2.6/site-packages/south/migration/migrators.py", line 84, in run_migration
migration_function()
File "/usr/lib64/python2.6/site-packages/south/migration/migrators.py", line 60, in <lambda>
return (lambda: direction(orm))
File "/opt/dev/t4sfeature/t4sdev/t4s/campaigns/migrations/0003_auto__add_field
keep_default=False)
File "/usr/lib64/python2.6/site-packages/south/db/generic.py", line 47, in _ca
return func(self, table, *args, **opts)
File "/usr/lib64/python2.6/site-packages/south/db/generic.py", line 418, in ad
self.execute(sql)
File "/usr/lib64/python2.6/site-packages/south/db/generic.py", line 282, in ex
cursor.execute(sql, params)
File "/usr/lib/python2.6/site-packages/django/db/backends/util.py", line 69, i
return super(CursorDebugWrapper, self).execute(sql, params)
File "/usr/lib/python2.6/site-packages/django/db/backends/util.py", line 53, i
return self.cursor.execute(sql, params)
File "/usr/lib/python2.6/site-packages/django/db/utils.py", line 99, in __exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/lib/python2.6/site-packages/django/db/backends/util.py", line 53, i
return self.cursor.execute(sql, params)
File "/usr/lib/python2.6/site-packages/django/db/backends/mysql/base.py", line
return self.cursor.execute(query, args)
File "/usr/lib64/python2.6/site-packages/MySQLdb/cursors.py", line 173, in exe
self.errorhandler(self, exc, value)
File "/usr/lib64/python2.6/site-packages/MySQLdb/connections.py", line 36, in
raise errorclass, errorvalue
django.db.utils.OperationalError: (1060, "Duplicate column name 'twilio_id'")
The lastest south migration file (named 0003_auto__add_field_taskqueue_twilio.py) regarding campaigns models has this:
u'campaigns.taskqueue': {
'Meta': {'object_name': 'TaskQueue'},
'campaign': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['campaigns.Campaign']"}),
'created_at': ('django.db.models.fields.DateTimeField', [], {'auto_now_add': 'True', 'blank': 'True'}),
'created_by': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['auth.User']"}),
'groups': ('django.db.models.fields.related.ManyToManyField', [], {'to': u"orm['organizations.Group']", 'symmetrical': 'False'}),
u'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
'launch_time': ('django.db.models.fields.DateTimeField', [], {}),
'status': ('django.db.models.fields.CharField', [], {'default': "'pending'", 'max_length': '30'}),
'twilio': ('django.db.models.fields.related.ForeignKey', [], {'to': u"orm['accounts.TwilioAccount']", 'null': 'True'})
},
As far as I remember, I just added this line to the campaigns/models.py file:
class TaskQueue(models.Model):
campaign = models.ForeignKey(Campaign)
launch_time = models.DateTimeField()
twilio = models.ForeignKey(TwilioAccount, null=True) # new line recently added
created_at = models.DateTimeField(auto_now_add=True)
created_by = models.ForeignKey(User)
and TwilioAccount model is like this:
class TwilioAccount(models.Model):
name = models.CharField(max_length=50, blank=True)
number = models.CharField(max_length=16, blank=True)
sid = models.CharField(max_length=100, blank=True)
token = models.CharField(max_length=100, blank=True)
created_at = models.DateTimeField(auto_now_add=True)
Thank you very much for your answers.

Answer 1

I found a solution to this. I read through South's migration file and git commit logs as well as checked my DB tables and realized there's a column named twilio already created (not sure when and how). After ensuring everything looks good, I used the recommendation of this very useful post (provided in the question above) by doing:
$ python manage.py migrate 0003 --fake
That allowed me to skip the migration #3 (I don't need it because I already have twilio column created in my DB) and proceed to next steps in coding. I hope someone who face the same error will find this useful.