I'm loading a fixture created with dumpdata, and getting the following exception:
Problem installing fixture 'db_dump.json': Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/django/core/management/commands/loaddata.py", line 174, in handle
obj.save(using=using)
File "/usr/lib/python2.6/site-packages/django/core/serializers/base.py", line 165, in save
models.Model.save_base(self.object, using=using, raw=True)
File "/usr/lib/python2.6/site-packages/django/db/models/base.py", line 526, in save_base
rows = manager.using(using).filter(pk=pk_val)._update(values)
File "/usr/lib/python2.6/site-packages/django/db/models/query.py", line 491, in _update
return query.get_compiler(self.db).execute_sql(None)
File "/usr/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 869, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/usr/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 735, in execute_sql
cursor.execute(sql, params)
File "/usr/lib/python2.6/site-packages/django/db/backends/sqlite3/base.py", line 234, in execute
return Database.Cursor.execute(self, query, params)
IntegrityError: columns app_label, model are not unique
This is with a sqlite3 backend.
Update: Using natural keys doesn't make a difference here.
What does it mean, and why is it happening?
Apparently one of the traps for the unwary is that one must exclude contenttypes when exporting fixtures. (Thanks to subsume on #django for the information).
To exclude content types use the -e option when running the dumpdata command.
$./manage.py dumpdata -e contenttypes > initial_data.json
python manage.py dumpdata --exclude=contenttypes --exclude=auth.Permission > initial_data.json
Related
I am trying to migrate the database of one my django projects from sqlite to mysql. I first dumped the whole thing with ./manage.py dumpdata > dump.json and prepared the database with ./manage.py migrate and deleted any created data in the tables (This was necessary, as the dump holds all the data).
When I wanted to import the data into the new database with ./manage.py loaddata, there were a lot of errors I was able to resolve, but I can't find the source of this error:
Processed 330984 object(s).Traceback (most recent call last):
File "/app/manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.5/site-packages/django/core/management/__init__.py", line 363, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.5/site-packages/django/core/management/__init__.py", line 355, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.5/site-packages/django/core/management/base.py", line 283, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.5/site-packages/django/core/management/base.py", line 330, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.5/site-packages/django/core/management/commands/loaddata.py", line 69, in handle
self.loaddata(fixture_labels)
File "/usr/local/lib/python3.5/site-packages/django/core/management/commands/loaddata.py", line 109, in loaddata
self.load_label(fixture_label)
File "/usr/local/lib/python3.5/site-packages/django/core/management/commands/loaddata.py", line 175, in load_label
obj.save(using=self.using)
File "/usr/local/lib/python3.5/site-packages/django/core/serializers/base.py", line 205, in save
models.Model.save_base(self.object, using=using, raw=True, **kwargs)
File "/usr/local/lib/python3.5/site-packages/django/db/models/base.py", line 837, in save_base
updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File "/usr/local/lib/python3.5/site-packages/django/db/models/base.py", line 904, in _save_table
forced_update)
File "/usr/local/lib/python3.5/site-packages/django/db/models/base.py", line 954, in _do_update
return filtered._update(values) > 0
File "/usr/local/lib/python3.5/site-packages/django/db/models/query.py", line 664, in _update
return query.get_compiler(self.db).execute_sql(CURSOR)
File "/usr/local/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1199, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/usr/local/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 894, in execute_sql
raise original_exception
File "/usr/local/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 884, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.5/site-packages/django/db/backends/utils.py", line 60, in execute
self.db.validate_no_broken_transaction()
File "/usr/local/lib/python3.5/site-packages/django/db/backends/base/base.py", line 448, in validate_no_broken_transaction
"An error occurred in the current transaction. You can't "
django.db.transaction.TransactionManagementError: Problem installing fixture '/path/to/django/dump.json': Could not load auth.User(pk=1): An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.
I already tried to remove all signal receivers, so that none of my own code is run when executing loaddata.
Has anyone else experienced similar behaviour with django's loaddata and managed to get it working?
Context:
python v3.5
django v1.11.4
only stdlib django model fields used
This is more of a workaround than a solution, but it did solve my problem.
After further investigation I realized that the auth.user model somehow triggered the query and broke the import.
The solution was to write a simple script to split the dump into multiple parts (I did not have access to the database anymore) and then import them in the correct order: contenttypes, auth, and the rest.
You can find the script here.
This is how I split up the dump:
./investigate-django-dump.py dump.json extract contenttypes. > contenttypes.json
./investigate-django-dump.py dump.json extract auth. > auth.json
./investigate-django-dump.py dump.json extractnot auth.,contenttypes. > data.json
I am new to django and trying to complete django following the tutorial. However, I cannot proceed to the next guide. As I type Poll.objects.all(). I receive several errors. Full information is provided below.
>>> Poll.objects.all()
Traceback (most recent call last):
File "<pyshell#11>", line 1, in <module>
Poll.objects.all()
File "C:\Python26\lib\site-packages\django\db\models\query.py", line 93, in __repr__
data = list(self[:REPR_OUTPUT_SIZE + 1])
File "C:\Python26\lib\site-packages\django\db\models\query.py", line 108, in __len__
self._result_cache.extend(self._iter)
File "C:\Python26\lib\site-packages\django\db\models\query.py", line 317, in iterator
for row in compiler.results_iter():
File "C:\Python26\lib\site-packages\django\db\models\sql\compiler.py", line 775, in results_iter
for rows in self.execute_sql(MULTI):
File "C:\Python26\lib\site-packages\django\db\models\sql\compiler.py", line 840, in execute_sql
cursor.execute(sql, params)
File "C:\Python26\lib\site-packages\django\db\backends\util.py", line 41, in execute
return self.cursor.execute(sql, params)
File "C:\Python26\lib\site-packages\django\db\backends\sqlite3\base.py", line 366, in execute
six.reraise(utils.DatabaseError, utils.DatabaseError(*tuple(e.args)), sys.exc_info()[2])
File "C:\Python26\lib\site-packages\django\db\backends\sqlite3\base.py", line 362, in execute
return Database.Cursor.execute(self, query, params)
DatabaseError: no such column: polls_poll.pub_date
Since the code given in the question is not sufficient enough, I will provide you the two possible faults:
1) The pub_date column was never created
2) The pub_date column was created, but after the table was created first. So, python manage.py syncdb does not add columns to the existing tables.
So, drop the table from SQL and then do syncdb again to solve this problem. If you do not want to drop the table, use south
I am getting this error after upgrading to django 1.4 while running tests using sqlite:
Creating test database for alias 'default'...
Problem installing fixture '/home/devasia/pyserver/project/mysite/app/fixtures/test_data.json': Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/loaddata.py", line 196, in handle
obj.save(using=using)
File "/usr/local/lib/python2.7/dist-packages/django/core/serializers/base.py", line 165, in save
models.Model.save_base(self.object, using=using, raw=True)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/base.py", line 529, in save_base
rows = manager.using(using).filter(pk=pk_val)._update(values)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/query.py", line 557, in _update
return query.get_compiler(self.db).execute_sql(None)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 986, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 818, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/base.py", line 337, in execute
return Database.Cursor.execute(self, query, params)
IntegrityError: Could not load auth.Permission(pk=31): columns content_type_id, codename are not unique
----------------------------------------------------------------------
Ran 1 test in 0.011s
OK
Destroying test database for alias 'default'...
This is my sample test case:
class CheckTest(TestCase):
fixtures = ['test_data.json']
def test_check(self):
c = Client()
c.login(username = 'test', password = 'test')
The fixtures are not that of old version. I have created this fixture using django 1.4 starting from 'syncdb' command. The tests are running well if I use postgresql as my test database(the tests run really slow).
Did I miss something while upgrading? I found this https://code.djangoproject.com/ticket/14731 but couldn't resolve the issue. (Please note that the only code/data from old project is the models)
Thanks
I have a Django site that's working fine. It has a small amount of data in the database which I want to use for testing. I've done dumpdata to generate a few .json fixtures. But, when I try to run a test on my app ("tagger") I get a Postgresql error, and I can't work out how to solve it:
(django-projectname)$ ./manage.py test tagger
Creating test database for alias 'default'...
Problem installing fixture '/Users/phil/Projects/RIG/projectname/django-projectname/projectname/tagger/fixtures/auth_testdata.json': Traceback (most recent call last):
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/core/management/commands/loaddata.py", line 174, in handle
obj.save(using=using)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/core/serializers/base.py", line 165, in save
models.Model.save_base(self.object, using=using, raw=True)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/models/base.py", line 526, in save_base
rows = manager.using(using).filter(pk=pk_val)._update(values)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/models/query.py", line 491, in _update
return query.get_compiler(self.db).execute_sql(None)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 869, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 735, in execute_sql
cursor.execute(sql, params)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
return self.cursor.execute(query, args)
IntegrityError: duplicate key value violates unique constraint "auth_permission_content_type_id_codename_key"
DETAIL: Key (content_type_id, codename)=(3, add_htuser) already exists.
I tried doing dumpdata for the auth app using the --natural flag too, but that didn't change anything. I'm stumped, and can't work out where it's getting a duplicate key from.
The "htuser" class that it refers to is a Proxy model on the auth User class.
I'm using South for my apps' migrations and it sounds like https://stackoverflow.com/a/2888916/250962 may be the answer, but I don't understand how to "Remove the explicitly-set primary key values from the datamigrations"?
I am relatively new to django. I have defined my db schema and validated it with no errors (manage.py validate reports 0 errors found).
Yet when I run ./manage.py syncdb
I get the following stack trace:
Creating table demo_foobar_one
Creating table demo_foobar_two
<snip>...</snip>
Traceback (most recent call last):
File "manage.py", line 11, in <module>
execute_manager(settings)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/__init__.py", line 438, in execute_manager
utility.execute()
File "/usr/local/lib/python2.6/dist-packages/django/core/management/__init__.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 191, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 218, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 347, in handle
return self.handle_noargs(**options)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/commands/syncdb.py", line 103, in handle_noargs
emit_post_sync_signal(created_models, verbosity, interactive, db)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/sql.py", line 185, in emit_post_sync_signal
interactive=interactive, db=db)
File "/usr/local/lib/python2.6/dist-packages/django/dispatch/dispatcher.py", line 162, in send
response = receiver(signal=self, sender=sender, **named)
File "/usr/local/lib/python2.6/dist-packages/django/contrib/auth/management/__init__.py", line 28, in create_permissions
defaults={'name': name, 'content_type': ctype})
File "/usr/local/lib/python2.6/dist-packages/django/db/models/manager.py", line 135, in get_or_create
return self.get_query_set().get_or_create(**kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 373, in get_or_create
obj.save(force_insert=True, using=self.db)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/base.py", line 435, in save
self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/base.py", line 528, in save_base
result = manager._insert(values, return_id=update_pk, using=using)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/manager.py", line 195, in _insert
return insert_query(self.model, values, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 1479, in insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/compiler.py", line 783, in execute_sql
cursor = super(SQLInsertCompiler, self).execute_sql(None)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/compiler.py", line 727, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.6/dist-packages/django/db/backends/util.py", line 15, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.6/dist-packages/django/db/backends/mysql/base.py", line 86, in execute
return self.cursor.execute(query, args)
File "/usr/local/lib/python2.6/dist-packages/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/cursors.py", line 176, in execute
if not self._defer_warnings: self._warning_check()
File "/usr/local/lib/python2.6/dist-packages/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/cursors.py", line 92, in _warning_check
warn(w[-1], self.Warning, 3)
_mysql_exceptions.Warning: Data truncated for column 'name' at row 1
I have checked (and double checked) my table schema. All name field are CharField type with maximum length = 64. The backend db I am using is MySQL, so I am sure that indexes can be created for strings of length 64.
What could be causing this error (I suspect it is a misleading error message - even though its coming from the db itself)...
The traceback is happening during the creation of a django.contrib.auth.Permission: some of these get created for your models automatically as part of syncdb.
Permission.name has max_length=50, so you probably have an application and/or model class with a really long name?
Try the following query in manage.py dbshell:
SELECT * FROM auth_permission WHERE LENGTH(name) = 50;
If you cannot change your model name, then you can fix this problem by reducing the length of the generated Permission.name by specifying verbose_name in the model Meta class (see here for more details):
class MyVeryLongModelNameThatIsBreakingMyMigration(models.Model):
class Meta:
verbose_name = 'my long model'
Update
There's an open (as of 2013) Django ticket to fix this:
#8162 (Increase Permission.name max_length)
As Piet Delport noted, the problem is that your model name is too long.
You're certainly going to have to shorten your model name, and then clean up your database. How you do that depends upon where you are in the development process.
If this is a brand new application, with a dedicated database, and no actual data, the simple answer is to drop and recreate the database, and then re-run python manage.py syncdb.
If you have other tables in the database that need to be left alone, but the tables for your Django have no 'real' data, and can thus be dropped without damage, then you can use manage.py sqlclear to generate SQL DDL to drop all of the Django-generated tables, constraints, and indexes.
Do the following:
apps="auth contenttypes sessions sites messages admin <myapp1> <myapp2>"
python manage.py sqlclear ${apps} > clear.sql
You can feed the generated script to mysql or python manage.py dbshell. Once that's done, you can re-run python manage.py syncdb.
If you have actual data in your database tables that can't be dropped or deleted: Slap yourself and repeat 100 times "I will never do development against a production database again. I will always back up my databases before changing them." Now you're going to have to hand-code SQL to change the affected table name, as well as anything else that references it and any references in the auth_permissions table and any other Django system tables. Your actual steps will depend entirely upon the state of your database and tables.
I also got error like this one using postgresql django 1.2, but the problem was not the length, but using ugettext_lazy for translating the names. ('can_purge', _("Can purge")) is evidently unacceptable, since the name is stored in the database as text