I can run all unit tests successfully, I can even run selenium tests successfully if I run an independent server, but when I try to use LiveServerTestCases to test everything in a self-contained manner, each LiveServerTestCase test ends with the following error after completing the tearDown function:
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\db\backends\base\base.py", line 239, in _commit
return self.connection.commit()
django.db.utils.IntegrityError: FOREIGN KEY constraint failed
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\test\testcases.py", line 209, in __call__
self._post_teardown()
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\test\testcases.py", line 908, in _post_teardown
self._fixture_teardown()
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\test\testcases.py", line 943, in _fixture_teardown
inhibit_post_migrate=inhibit_post_migrate)
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\core\management\__init__.py", line 148, in call_command
return command.execute(*args, **defaults)
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\core\management\base.py", line 353, in execute
output = self.handle(*args, **options)
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\core\management\commands\flush.py", line 80, in handle
emit_post_migrate_signal(verbosity, interactive, database)
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\core\management\sql.py", line 51, in emit_post_migrate_signal
**kwargs
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\dispatch\dispatcher.py", line 175, in send
for receiver in self._live_receivers(sender)
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\dispatch\dispatcher.py", line 175, in <listcomp>
for receiver in self._live_receivers(sender)
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\contrib\auth\management\__init__.py", line 79, in create_permissions
Permission.objects.using(using).bulk_create(perms)
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\db\models\query.py", line 471, in bulk_create
obj_without_pk._state.db = self.db
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\db\transaction.py", line 212, in __exit__
connection.commit()
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\db\backends\base\base.py", line 261, in commit
self._commit()
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\db\backends\base\base.py", line 239, in _commit
return self.connection.commit()
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\db\utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\db\backends\base\base.py", line 239, in _commit
return self.connection.commit()
django.db.utils.IntegrityError: FOREIGN KEY constraint failed
I'm pretty sure I don't have any errors in my database models, as all the unit tests run fine and the selenium tests run fine when I fire up a seperate server instance to run in parallel, so I'm guessing it has to do with selenium?
I've tried using the Chrome webdriver, IE webdriver and Firefox webdriver. Same results.
It doesn't appear to be related to my database as the error only occurs for LiveServerTestCases.
Environment Details
Django version 2.1
SQLite3 version 3.20.0
Some more information regarding the Django version, Database type and version along with your code trials would have helped us to debug this issue in a better way.
However, this error message...
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\core\management\sql.py", line 51, in emit_post_migrate_signal **kwargs
.
File "C:\Users\Win7\.virtualenvs\lang-QbOXb8q_\lib\site-packages\django\db\backends\base\base.py", line 239, in _commit
return self.connection.commit()
django.db.utils.IntegrityError: FOREIGN KEY constraint failed
...implies that an IntegrityError was raised while attempting to save an existing model instance.
As per Django 2.0 release notes:
Foreign key constraints are now enabled on SQLite: This was a backwards-incompatible change (IntegrityError: FOREIGN KEY constraint failed) if attempting to save an existing model instance that’s violating a foreign key constraint.
Foreign Keys are now created with DEFERRABLE INITIALLY DEFERRED instead of DEFERRABLE IMMEDIATE. So the tables may need to be rebuilt to recreate foreign keys with the new definition, particularly if you’re using a pattern as follows;
from django.db import transaction
with transaction.atomic():
Book.objects.create(author_id=1)
Author.objects.create(id=1)
If you don’t recreate the foreign key as DEFERRED, the first create() would fail as the foreign key constraints are enforced.
#dirkgroten in this discussion provided an example as follows:
Look for patterns like this in your code:
# in pagetree/models.py, line 810
#classmethod
def create_from_dict(cls, d):
return cls.objects.create() # what happens to d by the way?
This will definitely fail with a ForeignKey constraint error since a PageBlock must have section, so you can't call create without first assigning it.
Related
I'm working on Django 1.9 and python 3.3 project using multiple databases (different schema in a same postgresql database). When I try to migrate the project for the first time, I'm getting this error
Running migrations:
Rendering model states... DONE
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying MyApp.0001_initial...Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
psycopg2.ProgrammingError: ERROR: relation "auth_user" does not exist
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/usr/lib/python3.4/site-packages/django/core/management/__init__.py", line 350, in execute_from_command_line
utility.execute()
File "/usr/lib/python3.4/site-packages/django/core/management/__init__.py", line 342, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/lib/python3.4/site-packages/django/core/management/base.py", line 348, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/lib/python3.4/site-packages/django/core/management/base.py", line 399, in execute
output = self.handle(*args, **options)
File "/usr/lib/python3.4/site-packages/django/core/management/commands/migrate.py", line 200, in handle
executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
File "/usr/lib/python3.4/site-packages/django/db/migrations/executor.py", line 92, in migrate
self._migrate_all_forwards(plan, full_plan, fake=fake, fake_initial=fake_initial)
File "/usr/lib/python3.4/site-packages/django/db/migrations/executor.py", line 121, in _migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
File "/usr/lib/python3.4/site-packages/django/db/migrations/executor.py", line 198, in apply_migration
state = migration.apply(state, schema_editor)
File "/usr/lib/python3.4/site-packages/django/db/backends/base/schema.py", line 90, in __exit__
self.execute(sql)
File "/usr/lib/python3.4/site-packages/django/db/backends/base/schema.py", line 110, in execute
cursor.execute(sql, params)
File "/usr/lib/python3.4/site-packages/django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/usr/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/lib/python3.4/site-packages/django/db/utils.py", line 95, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/lib/python3.4/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/usr/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: ERROR: relation "auth_user" does not exist
This error seems to appeard in other projects when the auth_user table isn't migrated. In my case beginning my migration with manage.py migrate auth before migrating the app who need it doesn't solve the problem.
I suspect that the problem come from the use of different database in Django. My auth_user table is stored in the default database and the content of the models.py is routed to an other database.
Is the migration process looking for the auth_user table in the same database than my app database? Is it something absolutely different?
Answer to the problem
It is in fact a cross database reference problem. Django can't create cross database foreign keys.
From the Django 1.8 documentation (and there isn't any solution in the next versions (current version is 1.10)) :
Cross-database relations
Django doesn’t currently provide any support for foreign key or
many-to-many relationships spanning multiple databases. If you have
used a router to partition models to different databases, any foreign
key and many-to-many relationships defined by those models must be
internal to a single database.
This is because of referential integrity. In order to maintain a
relationship between two objects, Django needs to know that the
primary key of the related object is valid. If the primary key is
stored on a separate database, it’s not possible to easily evaluate
the validity of a primary key.
If you’re using Postgres, Oracle, or MySQL with InnoDB, this is
enforced at the database integrity level – database level key
constraints prevent the creation of relations that can’t be validated.
However, if you’re using SQLite or MySQL with MyISAM tables, there is
no enforced referential integrity; as a result, you may be able to
‘fake’ cross database foreign keys. However, this configuration is not
officially supported by Django
How to workaround and keep separate databases
In my case, because router are working, there is a little hack to simplify the cross database objects.
class CrossDBUser(models.Model):
user = models.IntegerField()
def get_user(self):
return User.objects.get(id=self.user)
def set_user(self, user):
self.user = user.id
class MyClassWithCrossDB(CrossDBUser):
field1 = models.CharField(max_length=200, blank=False)
field2 = models.IntegerField(default=0)
With this I can use the method set_user and get_user to work with the user stored in my object MyClassWithCrossDB.
Of course it's not perfect because it doesn't allow automated actions like on_delete=models.CASCADE and I'm obliged to work with methods instead of the instance variable. But it is a solution to workaround.
I'm trying to run the test for my application, Its giving messsage like
django.db.utils.ProgrammingError: column "status" cannot be cast automatically to type integer
HINT: Specify a USING expression to perform the conversion.
Oh, Its because i have altered the CharField to IntegerField using SQL syntax (Although migration was created for this change)
ALTER TABLE accounts ALTER COLUMN status TYPE integer USING (status::integer);
So when running the test command, is failing while creating the test database.
Is there any way around to this ?
Update:
Right now i'm skipping the part of running migrations using third party app using https://pypi.python.org/pypi/django-test-without-migrations/
Now test tables are being created but failing for a cache table
File "/home/naveen/ENV/automaticfleet/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 128, in handle
created_models = self.sync_apps(connection, executor.loader.unmigrated_apps)
File "/home/naveen/ENV/automaticfleet/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 229, in sync_apps
sql, references = connection.creation.sql_create_model(model, no_style(), seen_models)
File "/home/naveen/ENV/automaticfleet/local/lib/python2.7/site-packages/django/db/backends/creation.py", line 82, in sql_create_model
db_params = f.db_parameters(connection=self.connection)
File "/home/naveen/ENV/automaticfleet/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py", line 548, in db_parameters
type_string = self.db_type(connection)
File "/home/naveen/ENV/automaticfleet/local/lib/python2.7/site-packages/jsonfield/fields.py", line 73, in db_type
db_type = cache.get(cache_key)
File "/home/naveen/ENV/automaticfleet/local/lib/python2.7/site-packages/django/core/cache/backends/db.py", line 65, in get
"WHERE cache_key = %%s" % table, [key])
File "/home/naveen/ENV/automaticfleet/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/home/naveen/ENV/automaticfleet/local/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/home/naveen/ENV/automaticfleet/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "accounts_cache" does not exist
I am getting this error after upgrading to django 1.4 while running tests using sqlite:
Creating test database for alias 'default'...
Problem installing fixture '/home/devasia/pyserver/project/mysite/app/fixtures/test_data.json': Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/loaddata.py", line 196, in handle
obj.save(using=using)
File "/usr/local/lib/python2.7/dist-packages/django/core/serializers/base.py", line 165, in save
models.Model.save_base(self.object, using=using, raw=True)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 529, in save_base
rows = manager.using(using).filter(pk=pk_val)._update(values)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 557, in _update
return query.get_compiler(self.db).execute_sql(None)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 986, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 818, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/base.py", line 337, in execute
return Database.Cursor.execute(self, query, params)
IntegrityError: Could not load auth.Permission(pk=31): columns content_type_id, codename are not unique
----------------------------------------------------------------------
Ran 1 test in 0.011s
OK
Destroying test database for alias 'default'...
This is my sample test case:
class CheckTest(TestCase):
fixtures = ['test_data.json']
def test_check(self):
c = Client()
c.login(username = 'test', password = 'test')
The fixtures are not that of old version. I have created this fixture using django 1.4 starting from 'syncdb' command. The tests are running well if I use postgresql as my test database(the tests run really slow).
Did I miss something while upgrading? I found this https://code.djangoproject.com/ticket/14731 but couldn't resolve the issue. (Please note that the only code/data from old project is the models)
Thanks
I have a Django site that's working fine. It has a small amount of data in the database which I want to use for testing. I've done dumpdata to generate a few .json fixtures. But, when I try to run a test on my app ("tagger") I get a Postgresql error, and I can't work out how to solve it:
(django-projectname)$ ./manage.py test tagger
Creating test database for alias 'default'...
Problem installing fixture '/Users/phil/Projects/RIG/projectname/django-projectname/projectname/tagger/fixtures/auth_testdata.json': Traceback (most recent call last):
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/core/management/commands/loaddata.py", line 174, in handle
obj.save(using=using)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/core/serializers/base.py", line 165, in save
models.Model.save_base(self.object, using=using, raw=True)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/models/base.py", line 526, in save_base
rows = manager.using(using).filter(pk=pk_val)._update(values)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/models/query.py", line 491, in _update
return query.get_compiler(self.db).execute_sql(None)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 869, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 735, in execute_sql
cursor.execute(sql, params)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
return self.cursor.execute(query, args)
IntegrityError: duplicate key value violates unique constraint "auth_permission_content_type_id_codename_key"
DETAIL: Key (content_type_id, codename)=(3, add_htuser) already exists.
I tried doing dumpdata for the auth app using the --natural flag too, but that didn't change anything. I'm stumped, and can't work out where it's getting a duplicate key from.
The "htuser" class that it refers to is a Proxy model on the auth User class.
I'm using South for my apps' migrations and it sounds like https://stackoverflow.com/a/2888916/250962 may be the answer, but I don't understand how to "Remove the explicitly-set primary key values from the datamigrations"?
I am relatively new to django. I have defined my db schema and validated it with no errors (manage.py validate reports 0 errors found).
Yet when I run ./manage.py syncdb
I get the following stack trace:
Creating table demo_foobar_one
Creating table demo_foobar_two
<snip>...</snip>
Traceback (most recent call last):
File "manage.py", line 11, in <module>
execute_manager(settings)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/__init__.py", line 438, in execute_manager
utility.execute()
File "/usr/local/lib/python2.6/dist-packages/django/core/management/__init__.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 191, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 218, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 347, in handle
return self.handle_noargs(**options)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/commands/syncdb.py", line 103, in handle_noargs
emit_post_sync_signal(created_models, verbosity, interactive, db)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/sql.py", line 185, in emit_post_sync_signal
interactive=interactive, db=db)
File "/usr/local/lib/python2.6/dist-packages/django/dispatch/dispatcher.py", line 162, in send
response = receiver(signal=self, sender=sender, **named)
File "/usr/local/lib/python2.6/dist-packages/django/contrib/auth/management/__init__.py", line 28, in create_permissions
defaults={'name': name, 'content_type': ctype})
File "/usr/local/lib/python2.6/dist-packages/django/db/models/manager.py", line 135, in get_or_create
return self.get_query_set().get_or_create(**kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 373, in get_or_create
obj.save(force_insert=True, using=self.db)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/base.py", line 435, in save
self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/base.py", line 528, in save_base
result = manager._insert(values, return_id=update_pk, using=using)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/manager.py", line 195, in _insert
return insert_query(self.model, values, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 1479, in insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/compiler.py", line 783, in execute_sql
cursor = super(SQLInsertCompiler, self).execute_sql(None)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/compiler.py", line 727, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.6/dist-packages/django/db/backends/util.py", line 15, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.6/dist-packages/django/db/backends/mysql/base.py", line 86, in execute
return self.cursor.execute(query, args)
File "/usr/local/lib/python2.6/dist-packages/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/cursors.py", line 176, in execute
if not self._defer_warnings: self._warning_check()
File "/usr/local/lib/python2.6/dist-packages/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/cursors.py", line 92, in _warning_check
warn(w[-1], self.Warning, 3)
_mysql_exceptions.Warning: Data truncated for column 'name' at row 1
I have checked (and double checked) my table schema. All name field are CharField type with maximum length = 64. The backend db I am using is MySQL, so I am sure that indexes can be created for strings of length 64.
What could be causing this error (I suspect it is a misleading error message - even though its coming from the db itself)...
The traceback is happening during the creation of a django.contrib.auth.Permission: some of these get created for your models automatically as part of syncdb.
Permission.name has max_length=50, so you probably have an application and/or model class with a really long name?
Try the following query in manage.py dbshell:
SELECT * FROM auth_permission WHERE LENGTH(name) = 50;
If you cannot change your model name, then you can fix this problem by reducing the length of the generated Permission.name by specifying verbose_name in the model Meta class (see here for more details):
class MyVeryLongModelNameThatIsBreakingMyMigration(models.Model):
class Meta:
verbose_name = 'my long model'
Update
There's an open (as of 2013) Django ticket to fix this:
#8162 (Increase Permission.name max_length)
As Piet Delport noted, the problem is that your model name is too long.
You're certainly going to have to shorten your model name, and then clean up your database. How you do that depends upon where you are in the development process.
If this is a brand new application, with a dedicated database, and no actual data, the simple answer is to drop and recreate the database, and then re-run python manage.py syncdb.
If you have other tables in the database that need to be left alone, but the tables for your Django have no 'real' data, and can thus be dropped without damage, then you can use manage.py sqlclear to generate SQL DDL to drop all of the Django-generated tables, constraints, and indexes.
Do the following:
apps="auth contenttypes sessions sites messages admin <myapp1> <myapp2>"
python manage.py sqlclear ${apps} > clear.sql
You can feed the generated script to mysql or python manage.py dbshell. Once that's done, you can re-run python manage.py syncdb.
If you have actual data in your database tables that can't be dropped or deleted: Slap yourself and repeat 100 times "I will never do development against a production database again. I will always back up my databases before changing them." Now you're going to have to hand-code SQL to change the affected table name, as well as anything else that references it and any references in the auth_permissions table and any other Django system tables. Your actual steps will depend entirely upon the state of your database and tables.
I also got error like this one using postgresql django 1.2, but the problem was not the length, but using ugettext_lazy for translating the names. ('can_purge', _("Can purge")) is evidently unacceptable, since the name is stored in the database as text