django db migration failed with postgres - django

Question

I made some changes to my models and then ran a
python manage.py makemigrations
python manage.py migrate
and I got this traceback:
Operations to perform:
Synchronize unmigrated apps: staticfiles, messages
Apply all migrations: sessions, admin, study, auth, quiz, contenttypes, main
Synchronizing apps without migrations:
Creating tables...
Running deferred SQL...
Installing custom SQL...
Running migrations:
Rendering model states... DONE
Applying quiz.0013_auto_20151005_0644...Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 330, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 393, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 444, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 222, in handle
executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 110, in migrate
self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 154, in apply_migration
self.recorder.record_applied(migration.app_label, migration.name)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/recorder.py", line 67, in record_applied
self.migration_qs.create(app=app, name=name)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 348, in create
obj.save(force_insert=True, using=self.db)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 734, in save
force_update=force_update, update_fields=update_fields)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 762, in save_base
updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 846, in _save_table
result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 885, in _do_insert
using=using, raw=raw)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/manager.py", line 127, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 920, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 974, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 97, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: duplicate key value violates unique constraint "django_migrations_pkey"
DETAIL: Key (id)=(27) already exists.
I didn't know quite how to interpret the error that the primary key already exists. Strangely enough it added the field into the database because I was seeing it output on my site and when I ran a shell I could see that the fields were added and the default data 'that I chose' was populated.
I decided to run the migration again to see if it would pass after a second time and I got a different traceback...
Operations to perform:
Synchronize unmigrated apps: staticfiles, messages
Apply all migrations: sessions, admin, study, auth, quiz, contenttypes, main
Synchronizing apps without migrations:
Creating tables...
Running deferred SQL...
Installing custom SQL...
Running migrations:
Rendering model states... DONE
Applying quiz.0013_auto_20151005_0644...Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 330, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 393, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 444, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 222, in handle
executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 110, in migrate
self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 148, in apply_migration
state = migration.apply(state, schema_editor)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py", line 115, in apply
operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/fields.py", line 62, in database_forwards
field,
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 398, in add_field
self.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 111, in execute
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 97, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column "lang1_back_to_choice" of relation "quiz_langpairinstructions" already exists
Now if I try it again I will always get the second traceback. I am wondering what I should do so I can make migrations in the futures. Why am I getting this error if everything appears to have gone smoothly in the db itself?

Answer 1

There is another way. Since you know the last id value in django_migrations table, say it is 100. Then, you need to do this:
ALTER SEQUENCE django_migrations_id_seq RESTART WITH 101;

Answer 2

I overlooked the name of the 'django_migrations_pkey' column and I didn't realize that was in the database. Somehow django was one step behind the actual database and trying to use the primary key (pk=27) instead of the number it was actually on (which should have been 28)
I was unsure about how to change this from within django so instead I just deleted the last column from the database so it was one primary key shorter and on track with django and it started working just fine.

Answer 3

I got this error after I had previously manually tinkered with the django migrations tables entries (after some previously dodgy migrations were added then removed).
This left the django_migrations IDs out of order (so to speak) with the default position of the entry in the django_migrations table.
The issue is that django takes the ID of the last entry and +1 to create a new entry (in table below: 197 + 1 = 198), however as you can see Key (id)=(198) already exists in the table.
187 | foo_app | 0016_auto_20161220_2332 | 2017-01-06 05:22:07.666172+00
198 | bar_app | 0004_auto_20160423_2122 | 2017-01-13 05:38:31.922738+00
197 | baz_app | 0013_auto_20170203_2311 | 2017-02-08 18:50:22.70143+00
As you can see the numbers for the primary key on the left are out of order.
What I did was manually delete the out of order entries in the django_migration then re-add them.
DELETE from django_migrations where name = '0004_auto_20160423_2122';
DELETE from django_migrations where name = '0013_auto_20170203_2311';
INSERT INTO django_migrations VALUES (188, 'baz_app', '0013_auto_20170203_2311', date());
INSERT INTO django_migrations VALUES (189, 'bar_app', '0004_auto_20160423_2122', date());
Also of note is that the migration that I was initially trying to add 'new_app, '0002_auto_00000000_1111' actually modified the new_app table in postgres, however the django_migrations table didn't get updated.
So the second time I tried to re-run the migration 0002_auto_00000000_1111 I got the error:
django.db.utils.ProgrammingError: column "new_app_field" of relation "new_app_new" already exists
My solution above fixed the problem for me.

Answer 4

Simply reindexing the django_migrations table worked for me.
Follow these steps in your terminal
Change user to root
$ sudo su
So that you can then then change user to postgres
$ su postgres
Open postgres terminal client with your db selected
$ psql name_of_your_db
In the postgres client issue the following command
REINDEX TABLE django_migrations;
And then quit the postgres client
\q