I am getting this error after upgrading to django 1.4 while running tests using sqlite:
Creating test database for alias 'default'...
Problem installing fixture '/home/devasia/pyserver/project/mysite/app/fixtures/test_data.json': Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/loaddata.py", line 196, in handle
obj.save(using=using)
File "/usr/local/lib/python2.7/dist-packages/django/core/serializers/base.py", line 165, in save
models.Model.save_base(self.object, using=using, raw=True)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 529, in save_base
rows = manager.using(using).filter(pk=pk_val)._update(values)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 557, in _update
return query.get_compiler(self.db).execute_sql(None)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 986, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 818, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/base.py", line 337, in execute
return Database.Cursor.execute(self, query, params)
IntegrityError: Could not load auth.Permission(pk=31): columns content_type_id, codename are not unique
----------------------------------------------------------------------
Ran 1 test in 0.011s
OK
Destroying test database for alias 'default'...
This is my sample test case:
class CheckTest(TestCase):
fixtures = ['test_data.json']
def test_check(self):
c = Client()
c.login(username = 'test', password = 'test')
The fixtures are not that of old version. I have created this fixture using django 1.4 starting from 'syncdb' command. The tests are running well if I use postgresql as my test database(the tests run really slow).
Did I miss something while upgrading? I found this https://code.djangoproject.com/ticket/14731 but couldn't resolve the issue. (Please note that the only code/data from old project is the models)
Thanks
Related
I am trying to migrate the database of one my django projects from sqlite to mysql. I first dumped the whole thing with ./manage.py dumpdata > dump.json and prepared the database with ./manage.py migrate and deleted any created data in the tables (This was necessary, as the dump holds all the data).
When I wanted to import the data into the new database with ./manage.py loaddata, there were a lot of errors I was able to resolve, but I can't find the source of this error:
Processed 330984 object(s).Traceback (most recent call last):
File "/app/manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.5/site-packages/django/core/management/__init__.py", line 363, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.5/site-packages/django/core/management/__init__.py", line 355, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.5/site-packages/django/core/management/base.py", line 283, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.5/site-packages/django/core/management/base.py", line 330, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.5/site-packages/django/core/management/commands/loaddata.py", line 69, in handle
self.loaddata(fixture_labels)
File "/usr/local/lib/python3.5/site-packages/django/core/management/commands/loaddata.py", line 109, in loaddata
self.load_label(fixture_label)
File "/usr/local/lib/python3.5/site-packages/django/core/management/commands/loaddata.py", line 175, in load_label
obj.save(using=self.using)
File "/usr/local/lib/python3.5/site-packages/django/core/serializers/base.py", line 205, in save
models.Model.save_base(self.object, using=using, raw=True, **kwargs)
File "/usr/local/lib/python3.5/site-packages/django/db/models/base.py", line 837, in save_base
updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File "/usr/local/lib/python3.5/site-packages/django/db/models/base.py", line 904, in _save_table
forced_update)
File "/usr/local/lib/python3.5/site-packages/django/db/models/base.py", line 954, in _do_update
return filtered._update(values) > 0
File "/usr/local/lib/python3.5/site-packages/django/db/models/query.py", line 664, in _update
return query.get_compiler(self.db).execute_sql(CURSOR)
File "/usr/local/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1199, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/usr/local/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 894, in execute_sql
raise original_exception
File "/usr/local/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 884, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.5/site-packages/django/db/backends/utils.py", line 60, in execute
self.db.validate_no_broken_transaction()
File "/usr/local/lib/python3.5/site-packages/django/db/backends/base/base.py", line 448, in validate_no_broken_transaction
"An error occurred in the current transaction. You can't "
django.db.transaction.TransactionManagementError: Problem installing fixture '/path/to/django/dump.json': Could not load auth.User(pk=1): An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.
I already tried to remove all signal receivers, so that none of my own code is run when executing loaddata.
Has anyone else experienced similar behaviour with django's loaddata and managed to get it working?
Context:
python v3.5
django v1.11.4
only stdlib django model fields used
This is more of a workaround than a solution, but it did solve my problem.
After further investigation I realized that the auth.user model somehow triggered the query and broke the import.
The solution was to write a simple script to split the dump into multiple parts (I did not have access to the database anymore) and then import them in the correct order: contenttypes, auth, and the rest.
You can find the script here.
This is how I split up the dump:
./investigate-django-dump.py dump.json extract contenttypes. > contenttypes.json
./investigate-django-dump.py dump.json extract auth. > auth.json
./investigate-django-dump.py dump.json extractnot auth.,contenttypes. > data.json
I have connected django with sql anywhere 11 database, I have follow the instructions on the page SQL Anywhere Django Driver from github.
I have created the database with this command:
C:\>dbinit -z UCA django.db --> (database created successful)
I have started the Database Server with command:
C:\>dbsrv11 django.db --> (database server started successful)
I have created my project with:
C:\>django-admin.py startproject mysite (at this point all it's ok)
I have edited the file mysite/mysite/settings.py and I have changed the DATABASES setting with this:
DATABASES = {
'default': {
'ENGINE': 'sqlany_django',
'NAME': 'django',
'USER': 'dba',
'PASSWORD': 'sql',
'OPTIONS': {'eng': 'django'}
}
}
When I want to sync database using this command:
c:/mysite>python manage.py syncdb
the prompt show me a error message (the error message is shown below)
I have connected Sybase Central to django.db I see the tables was created.
but when I try save data to the tables this data wasn't recorded
even when I have created the superuser with this command:
c:\mysite>python manage.py createsuperuser --username=joe --email=joe#example.com
Password:123
Password (again):123
Superuser created successfully.
I querying the datebase with:
select * from auth_user
No rows are shown
The data aren't saved
I have an environment for test with this features:
O.S.: Windows XP
DB Engine: SQL Anywhere 11 with EBF 3069
Python 2.7
Django 1.6.1
Setuptools installed
PIP installed
sqlanydb installed
sqlany-django installed
Here is the error message
C:\mysite>python manage.py syncdb
Creating tables ...
Creating table django_admin_log
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_groups
Creating table auth_user_user_permissions
Creating table auth_user
Creating table django_content_type
Creating table django_session
Creating table Personal_persona
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "C:\Python27\lib\site-packages\django\core\management\__init__.py", line
399, in execute_from_command_line
utility.execute()
File "C:\Python27\lib\site-packages\django\core\management\__init__.py", line
392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Python27\lib\site-packages\django\core\management\base.py", line 242,
in run_from_argv
self.execute(*args, **options.__dict__)
File "C:\Python27\lib\site-packages\django\core\management\base.py", line 285,
in execute
output = self.handle(*args, **options)
File "C:\Python27\lib\site-packages\django\core\management\base.py", line 415,
in handle
return self.handle_noargs(**options)
File "C:\Python27\lib\site-packages\django\core\management\commands\syncdb.py"
, line 112, in handle_noargs
emit_post_sync_signal(created_models, verbosity, interactive, db)
File "C:\Python27\lib\site-packages\django\core\management\sql.py", line 216,
in emit_post_sync_signal
interactive=interactive, db=db)
File "C:\Python27\lib\site-packages\django\dispatch\dispatcher.py", line 185,
in send
response = receiver(signal=self, sender=sender, **named)
File "C:\Python27\lib\site-packages\django\contrib\auth\management\__init__.py
", line 101, in create_permissions
auth_app.Permission.objects.using(db).bulk_create(perms)
File "C:\Python27\lib\site-packages\django\db\models\query.py", line 356, in b
ulk_create
self._batched_insert(objs_without_pk, fields, batch_size)
File "C:\Python27\lib\site-packages\django\db\models\query.py", line 835, in _
batched_insert
using=self.db)
File "C:\Python27\lib\site-packages\django\db\models\manager.py", line 232, in
_insert
return insert_query(self.model, objs, fields, **kwargs)
File "C:\Python27\lib\site-packages\django\db\models\query.py", line 1511, in
insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "C:\Python27\lib\site-packages\django\db\models\sql\compiler.py", line 90
3, in execute_sql
cursor.execute(sql, params)
File "C:\Python27\lib\site-packages\django\db\backends\util.py", line 69, in e
xecute
return super(CursorDebugWrapper, self).execute(sql, params)
File "C:\Python27\lib\site-packages\django\db\backends\util.py", line 53, in e
xecute
return self.cursor.execute(sql, params)
File "C:\Python27\lib\site-packages\django\db\utils.py", line 99, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "C:\Python27\lib\site-packages\django\db\backends\util.py", line 53, in e
xecute
return self.cursor.execute(sql, params)
File "C:\Python27\lib\site-packages\sqlany_django\base.py", line 87, in execut
e
ret = self.cursor.execute(trace(query), trace(args))
File "C:\Python27\lib\site-packages\sqlanydb.py", line 714, in execute
self.executemany(operation, [parameters])
File "C:\Python27\lib\site-packages\sqlanydb.py", line 685, in executemany
bind_count = self.api.sqlany_num_params(self.stmt)
File "C:\Python27\lib\site-packages\sqlanydb.py", line 619, in __stmt_get
self.handleerror(*self.parent.error())
File "C:\Python27\lib\site-packages\sqlanydb.py", line 613, in handleerror
eh(self.parent, self, errorclass, errorvalue)
File "C:\Python27\lib\site-packages\sqlanydb.py", line 342, in standardErrorHa
ndler
raise errorclass(errorvalue)
django.db.utils.OperationalError: Syntax error near ',' on line 1
Sorry, but the SQL Anywhere Django driver requires SQL Anywhere v12 or higher; you cannot use it with SQL Anywhere v11. I have confirmed that the driver uses features of the database server that were not supported in version 11.
Disclaimer: I work for SAP in SQL Anywhere engineering.
I have a Django site that's working fine. It has a small amount of data in the database which I want to use for testing. I've done dumpdata to generate a few .json fixtures. But, when I try to run a test on my app ("tagger") I get a Postgresql error, and I can't work out how to solve it:
(django-projectname)$ ./manage.py test tagger
Creating test database for alias 'default'...
Problem installing fixture '/Users/phil/Projects/RIG/projectname/django-projectname/projectname/tagger/fixtures/auth_testdata.json': Traceback (most recent call last):
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/core/management/commands/loaddata.py", line 174, in handle
obj.save(using=using)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/core/serializers/base.py", line 165, in save
models.Model.save_base(self.object, using=using, raw=True)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/models/base.py", line 526, in save_base
rows = manager.using(using).filter(pk=pk_val)._update(values)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/models/query.py", line 491, in _update
return query.get_compiler(self.db).execute_sql(None)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 869, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 735, in execute_sql
cursor.execute(sql, params)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
return self.cursor.execute(query, args)
IntegrityError: duplicate key value violates unique constraint "auth_permission_content_type_id_codename_key"
DETAIL: Key (content_type_id, codename)=(3, add_htuser) already exists.
I tried doing dumpdata for the auth app using the --natural flag too, but that didn't change anything. I'm stumped, and can't work out where it's getting a duplicate key from.
The "htuser" class that it refers to is a Proxy model on the auth User class.
I'm using South for my apps' migrations and it sounds like https://stackoverflow.com/a/2888916/250962 may be the answer, but I don't understand how to "Remove the explicitly-set primary key values from the datamigrations"?
I'm loading a fixture created with dumpdata, and getting the following exception:
Problem installing fixture 'db_dump.json': Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/django/core/management/commands/loaddata.py", line 174, in handle
obj.save(using=using)
File "/usr/lib/python2.6/site-packages/django/core/serializers/base.py", line 165, in save
models.Model.save_base(self.object, using=using, raw=True)
File "/usr/lib/python2.6/site-packages/django/db/models/base.py", line 526, in save_base
rows = manager.using(using).filter(pk=pk_val)._update(values)
File "/usr/lib/python2.6/site-packages/django/db/models/query.py", line 491, in _update
return query.get_compiler(self.db).execute_sql(None)
File "/usr/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 869, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/usr/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 735, in execute_sql
cursor.execute(sql, params)
File "/usr/lib/python2.6/site-packages/django/db/backends/sqlite3/base.py", line 234, in execute
return Database.Cursor.execute(self, query, params)
IntegrityError: columns app_label, model are not unique
This is with a sqlite3 backend.
Update: Using natural keys doesn't make a difference here.
What does it mean, and why is it happening?
Apparently one of the traps for the unwary is that one must exclude contenttypes when exporting fixtures. (Thanks to subsume on #django for the information).
To exclude content types use the -e option when running the dumpdata command.
$./manage.py dumpdata -e contenttypes > initial_data.json
python manage.py dumpdata --exclude=contenttypes --exclude=auth.Permission > initial_data.json
I am relatively new to django. I have defined my db schema and validated it with no errors (manage.py validate reports 0 errors found).
Yet when I run ./manage.py syncdb
I get the following stack trace:
Creating table demo_foobar_one
Creating table demo_foobar_two
<snip>...</snip>
Traceback (most recent call last):
File "manage.py", line 11, in <module>
execute_manager(settings)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/__init__.py", line 438, in execute_manager
utility.execute()
File "/usr/local/lib/python2.6/dist-packages/django/core/management/__init__.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 191, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 218, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 347, in handle
return self.handle_noargs(**options)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/commands/syncdb.py", line 103, in handle_noargs
emit_post_sync_signal(created_models, verbosity, interactive, db)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/sql.py", line 185, in emit_post_sync_signal
interactive=interactive, db=db)
File "/usr/local/lib/python2.6/dist-packages/django/dispatch/dispatcher.py", line 162, in send
response = receiver(signal=self, sender=sender, **named)
File "/usr/local/lib/python2.6/dist-packages/django/contrib/auth/management/__init__.py", line 28, in create_permissions
defaults={'name': name, 'content_type': ctype})
File "/usr/local/lib/python2.6/dist-packages/django/db/models/manager.py", line 135, in get_or_create
return self.get_query_set().get_or_create(**kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 373, in get_or_create
obj.save(force_insert=True, using=self.db)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/base.py", line 435, in save
self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/base.py", line 528, in save_base
result = manager._insert(values, return_id=update_pk, using=using)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/manager.py", line 195, in _insert
return insert_query(self.model, values, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 1479, in insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/compiler.py", line 783, in execute_sql
cursor = super(SQLInsertCompiler, self).execute_sql(None)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/compiler.py", line 727, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.6/dist-packages/django/db/backends/util.py", line 15, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.6/dist-packages/django/db/backends/mysql/base.py", line 86, in execute
return self.cursor.execute(query, args)
File "/usr/local/lib/python2.6/dist-packages/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/cursors.py", line 176, in execute
if not self._defer_warnings: self._warning_check()
File "/usr/local/lib/python2.6/dist-packages/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/cursors.py", line 92, in _warning_check
warn(w[-1], self.Warning, 3)
_mysql_exceptions.Warning: Data truncated for column 'name' at row 1
I have checked (and double checked) my table schema. All name field are CharField type with maximum length = 64. The backend db I am using is MySQL, so I am sure that indexes can be created for strings of length 64.
What could be causing this error (I suspect it is a misleading error message - even though its coming from the db itself)...
The traceback is happening during the creation of a django.contrib.auth.Permission: some of these get created for your models automatically as part of syncdb.
Permission.name has max_length=50, so you probably have an application and/or model class with a really long name?
Try the following query in manage.py dbshell:
SELECT * FROM auth_permission WHERE LENGTH(name) = 50;
If you cannot change your model name, then you can fix this problem by reducing the length of the generated Permission.name by specifying verbose_name in the model Meta class (see here for more details):
class MyVeryLongModelNameThatIsBreakingMyMigration(models.Model):
class Meta:
verbose_name = 'my long model'
Update
There's an open (as of 2013) Django ticket to fix this:
#8162 (Increase Permission.name max_length)
As Piet Delport noted, the problem is that your model name is too long.
You're certainly going to have to shorten your model name, and then clean up your database. How you do that depends upon where you are in the development process.
If this is a brand new application, with a dedicated database, and no actual data, the simple answer is to drop and recreate the database, and then re-run python manage.py syncdb.
If you have other tables in the database that need to be left alone, but the tables for your Django have no 'real' data, and can thus be dropped without damage, then you can use manage.py sqlclear to generate SQL DDL to drop all of the Django-generated tables, constraints, and indexes.
Do the following:
apps="auth contenttypes sessions sites messages admin <myapp1> <myapp2>"
python manage.py sqlclear ${apps} > clear.sql
You can feed the generated script to mysql or python manage.py dbshell. Once that's done, you can re-run python manage.py syncdb.
If you have actual data in your database tables that can't be dropped or deleted: Slap yourself and repeat 100 times "I will never do development against a production database again. I will always back up my databases before changing them." Now you're going to have to hand-code SQL to change the affected table name, as well as anything else that references it and any references in the auth_permissions table and any other Django system tables. Your actual steps will depend entirely upon the state of your database and tables.
I also got error like this one using postgresql django 1.2, but the problem was not the length, but using ugettext_lazy for translating the names. ('can_purge', _("Can purge")) is evidently unacceptable, since the name is stored in the database as text