Django can't makemigrations after changing app name and db tables - django

I'm working with Django 1.11.5 and using PyCharm as my IDE. I've been trying to refactor my app name from "clinicaltrials" to "cancer_trials". PyCharm updated all of my project files accordingly. I then followed the steps in this SO answer to update the appropriate database tables. However, I'm getting the following error when I try run makemigration. I can't seem to figure out what this means and/or what part I'm missing here.
> python manage.py makemigrations
Traceback (most recent call last):
File "manage.py", line 22, in <module>
execute_from_command_line(sys.argv)
File "C:\Python_3.6.1\lib\site-packages\django\core\management\__init__.py", line 364, in execute_from_command_line
utility.execute()
File "C:\Python_3.6.1\lib\site-packages\django\core\management\__init__.py", line 356, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Python_3.6.1\lib\site-packages\django\core\management\base.py", line 283, in run_from_argv
self.execute(*args, **cmd_options)
File "C:\Python_3.6.1\lib\site-packages\django\core\management\base.py", line 330, in execute
output = self.handle(*args, **options)
File "C:\Python_3.6.1\lib\site-packages\django\core\management\commands\makemigrations.py", line 150, in handle
loader.project_state(),
File "C:\Python_3.6.1\lib\site-packages\django\db\migrations\loader.py", line 323, in project_state
return self.graph.make_state(nodes=nodes, at_end=at_end, real_apps=list(self.unmigrated_apps))
File "C:\Python_3.6.1\lib\site-packages\django\db\migrations\graph.py", line 409, in make_state
project_state = self.nodes[node].mutate_state(project_state, preserve=False)
File "C:\Python_3.6.1\lib\site-packages\django\db\migrations\migration.py", line 92, in mutate_state
operation.state_forwards(self.app_label, new_state)
File "C:\Python_3.6.1\lib\site-packages\django\db\migrations\operations\fields.py", line 142, in state_forwards
for name, instance in state.models[app_label, self.model_name_lower].fields:
KeyError: ('cancer_trials', 'cancer_trials')
Here's the function that's throwing the error
def state_forwards(self, app_label, state):
new_fields = []
old_field = None
for name, instance in state.models[app_label, self.model_name_lower].fields:
if name != self.name:
new_fields.append((name, instance))
else:
old_field = instance
state.models[app_label, self.model_name_lower].fields = new_fields
# Delay rendering of relationships if it's not a relational field
delay = not old_field.is_relation
state.reload_model(app_label, self.model_name_lower, delay=delay)

In my experience the easiest solution is to create new app and copy the code:
Create new app with the desired name and add it to settings
Copy/paste code from old app to new app, change references from old app to new app, run makemigrations and migrate
Open database and copy data from old tables to new tables
Check that everything works in new app
Search stackoverflow.com or google how to remove app from project or just leave in there. Unfortunately, I'm not 100 % positive about these steps, somebody please correct me if I'm wrong, but to my recollection:
run python manage.py migrate old_app zero (this unapplies all migrations for an app)
remove app from settings
delete files

Remove the all migrations files within your project
Go through each of your projects apps migration folder and remove everything inside, except the init.py file.
Drop the current database, or delete the db.sqlite3 if it is your case.
Create the initial migrations and generate the database schema
try run again migrationsa and migrate commans

OK
if you want to keep the existing database maybe this guide may help you:
https://simpleisbetterthancomplex.com/tutorial/2016/07/26/how-to-reset-migrations.html
(scenario 2)

A nice answer on how to properly move data between apps, can be found here.
What worked for me is the following:
Export the data to json
./manage.py dumpdata --exclude auth.permission --exclude contenttypes --exclude admin.LogEntry --exclude sessions --indent 2 > <path_out_of_the_project>/db.json
Open the db.json file using a capable editor and rename all the instances of the old app name to the new app name.
Rename your app and all the necessary references into your code.
Delete the database and recreate a new empty one applying all the migrations.
Load the data from the db.json file which include the new app name.
./manage.py loaddata <path_out_of_the_project>/db.json

simple approach:
Manually delete all migrations in the apps by going into each apps' 'Migrations' named directory.
Note: deleting init.py in 'Migrations' named directory will not cause any harm.
Above 'appname' is apps taken one-by-one and migrated using above steps. For new apps added will only show migrated table for the next two 'migrate' and 'sqlmigrate' command work.
After this:
$: python manage.py makemigrations appname
$: python manage.py migrate appname
$: python manage.py sqlmigrate appname 0001

Related

Django - django.core.exceptions.FieldDoesNotExist - has no field named

On our production server we got the following error when restarting django or try to run 'python manage.py makemigrations'
django.core.exceptions.FieldDoesNotExist: pricing.pricing has no field named 'price_per_hour'
What is strange is that the field price_per_hour was renamed long time ago to price and the migration when well.
But now I got this error every time and it is preventing to make any other model modification (in any app) and migrations.
What I checked :
If I run 'python manage.py showmigrations' every migration is flagged with an X, which if I'm right, means all the migration were done
price_per_hour is no longer find/used in any of the django app / class
class Pricing(models.Model):
price = models.DecimalField(default=5,max_digits=10, decimal_places=2)
class Meta:
ordering = ['-price',]
def __str__(self):
return "{}".format(self.price)
I also exported the matching./current database in sql and we well see that it contains price column and not price_per_hour. And no reference anywhere to price_per_hour
CREATE TABLE public.pricing_pricing (
id integer NOT NULL,
price numeric(10,2) NOT NULL,
);
I also tried to rename the filed price to price_per_hour just in case but it didn't help
For me it seems the error comes from Django rather than the PostgreSQL database but I'm not sure.
Here is the complete traceback
python manage.py makemigrations
Traceback (most recent call last):
File "manage.py", line 21, in <module>
main()
File "manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 328, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 369, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 83, in wrapped
res = handle_func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/core/management/commands/makemigrations.py", line 141, in handle
loader.project_state(),
File "/usr/local/lib/python3.8/site-packages/django/db/migrations/loader.py", line 324, in project_state
return self.graph.make_state(nodes=nodes, at_end=at_end, real_apps=list(self.unmigrated_apps))
File "/usr/local/lib/python3.8/site-packages/django/db/migrations/graph.py", line 315, in make_state
project_state = self.nodes[node].mutate_state(project_state, preserve=False)
File "/usr/local/lib/python3.8/site-packages/django/db/migrations/migration.py", line 87, in mutate_state
operation.state_forwards(self.app_label, new_state)
File "/usr/local/lib/python3.8/site-packages/django/db/migrations/operations/fields.py", line 326, in state_forwards
raise FieldDoesNotExist(
django.core.exceptions.FieldDoesNotExist: pricing.pricing has no field named 'price_per_hour'
I don't know what other things to look for so, any idea or suggestion will be really appreciated
PS : I thought of removing all the migration files and re-running them but as this is a production server I'm afraid to lose the database content or break something.
find . -path "*/migrations/*.py" -not -name "__init__.py" -delete
find . -path "*/migrations/*.pyc" -delete
So what Boris described was the issue I had. There were incoherences in migration files and django couldn't make the migrations.
To be more precise in my case the error 'the pricing.pricing has no field named' was due to the fact that in the initial migration file the field was named price, but in the second migration file it was asked to rename price_per_hour (which didn't exist) to price and so the error
0001_initial.py
migrations.CreateModel(
name='Pricing',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('price', models.DecimalField(decimal_places=2, default=5, max_digits=10)),
0002_auto_20200304_1344.py
operations = [
migrations.RenameField(
model_name='pricing',
old_name='price_per_hour',
new_name='price',
),
So both options described by Boris are those to follow. First you can try to go through all migrations files and hopefully find and fix manually the error.
To find out which one caused trouble I did this https://stackoverflow.com/a/53135777/20025351
And if you cannot fix it manually (that was my case) I made sure to have the database model matching the Django models, then I removed all the migrations files and re-run migration/migrate.
The error is found on the row migrations/operations/fields. Somehow a discrepancy between the database and the migrations files has occurred, or perhaps you applied a RunPython command in your migration file that references to this field.
I also faced a similar issues, and in my case it appeared to be a creation and deletion of a specific field in two old migration files that seemed to be the issue.
The solution for you is to 'fix' the old migration files, or squash your history.
Option 1) Fixing the error would require you to go through all migrations files and manually edit the migrations where this error stems from. If you can run the makemigration command in your local editor in debug mode you can track where this error occurs. Sadly Django's errorhanding on these errors are not that detailed.
Option 2) An easier option, the option which I would go for, is to squash the migration files. You thus remove all the history of the migration files, and reduce it to a single migration step. See docs. A downside is that you lose all your history, so make sure your local, staging and production environment is all synced up. Specifically on production, make sure that the database model (if you use postgres you can use pgadmin) is exactly the same as your Django model seen in the code.
Squashing the migration files actually removes the whole history. In your example, if you have a migration to add a field price_per_hour, and one to rename that field later to price, squashing will merge those migration files into a single action that makes the field price.

Setting up dj-stripe (initial migrations) ProgrammingError: relation "djstripe_customer" does not exist

dj-stripe looks like a very attractive way to manage stripe subscriptions with django.
I installed dj-stripe following the instructions here.
Installation seemed to work, in that the installation includes running these three commands, and doing so created and populated new tables in my Postgres DB as well as creating new customer objects in my stripe account.
python manage.py migrate
python manage.py djstripe_init_customers
python manage.py djstripe_init_plans
For some reason, I decided dj-stripe was overkill (stripe's api seems easy enough to use without so much boilerplate) so I uninstalled using (pip uninstall dj-stripe), removed the mention from INSTALLED_APPS and the entry in urls.py.
To finish the clean-up, I deleted the customers which had been created in my stripe.com account, then I used DROP TABLE to manually delete the leftover tables in my Postgres DB (all tables starting with djstripe_: tables like djstripe_customer, djstripe_invoice, etc.)
Now, I've decided I actually do want to use dj-stripe, but this time installation is not working. The second of those three commands throws a ProgrammingError
requirements.txt includes:
stripe==1.22.3
dj-stripe==0.5.0
Installed dj-stripe using pip...
pip install -r requirements.txt
...and received success messages.
Then following on instructions as before, I ran into a ProgrammingError.
(awe01)MoriartyMacBookAir13:awesomeapp macuser$ python manage.py migrate
Operations to perform:
Synchronize unmigrated apps: django_user_agents, evernote, twitter, polls, widget_tweaks, djrill, sitemaps, facebook, django_slack, storages, hello
Apply all migrations: account, djstripe, admin, sessions, sites, auth, audiotracks, contenttypes, socialaccount
Synchronizing apps without migrations:
Creating tables...
Installing custom SQL...
Installing indexes...
Running migrations:
No migrations to apply.
(I'm not sure what the above was supposed to achieve...
At this point I checked my postgres db: no extra tables have been created. There is no table with name starting with djstripe_)
(awe01)MoriartyMacBookAir13:awesomeapp macuser$ python manage.py djstripe_init_customers
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
utility.execute()
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/core/management/__init__.py", line 377, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/core/management/base.py", line 288, in run_from_argv
self.execute(*args, **options.__dict__)
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute
output = self.handle(*args, **options)
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/djstripe/management/commands/djstripe_init_customers.py", line 15, in handle
for subscriber in get_subscriber_model().objects.filter(customer__isnull=True):
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/models/query.py", line 141, in __iter__
self._fetch_all()
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/models/query.py", line 966, in _fetch_all
self._result_cache = list(self.iterator())
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/models/query.py", line 265, in iterator
for row in compiler.results_iter():
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 700, in results_iter
for rows in self.execute_sql(MULTI):
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
cursor.execute(sql, params)
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/backends/utils.py", line 81, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "djstripe_customer" does not exist
LINE 1: ...r"."date_joined" FROM "auth_user" LEFT OUTER JOIN "djstripe_...
The error
for subscriber in get_subscriber_model().objects.filter(customer__isnull=True):
referrs to line 15 of djstripe_init_customers.py
I don't understand why the normal migration steps are failing (why it expects "djstripe_customer" to have been created before it has been). Grateful for steps I can take to further trouble-shoot. (I'm assuming I've created the gotcha due to dirty uninstall of initial attempt?)
In case relevant:
My auth system is django-allauth
AUTH_USER_MODEL is not explicitly defined in my settings.py file and has been working fine. (I didn't make a custom user model.)
#kavanaugh-development solved this:
To completely remove remnant of the initial install, I had to remove the relevant djstripe rows from the migrations table. "If you don't delete those rows, django will ignore the migration commands the second time around."
DELETE FROM django_migrations WHERE app = 'djstripe';
Once I had done this, a fresh install of dj-stripe worked perfectly as it did the first time round:
I ran python manage.py migrate, which
re-created a few djstripe rows in django_migrations table
also created (empty) tables required for dj-stripe:
djstripe_charge
djstripe_charge_id_seq
djstripe_currentsubscription
djstripe_currentsubscription_id_seq
djstripe_customer
djstripe_customer_id_seq
djstripe_event
djstripe_event_id_seq
djstripe_eventprocessingexception
djstripe_eventprocessingexception_id_seq
djstripe_invoice
djstripe_invoice_id_seq
djstripe_invoiceitem
djstripe_invoiceitem_id_seq
djstripe_plan
djstripe_plan_id_seq
djstripe_transfer
djstripe_transfer_id_seq
djstripe_transferchargefee
djstripe_transferchargefee_id_seq
So the required tables are ready to be populated by the subsequent commands
(such as python manage.py djstripe_init_customers)
So that's a general (basic) lesson learned about migrations; hope this helps someone.
Thanks

django 1.8 fails to django.db.utils.ProgrammingError: relation "auth_user" does not exist

I had a working project with django 1.7, and now I moved it to django 1.8.
I can do syncdb and run the app with sqlite, but when I switch to postgres, it fails to do syncdb:
Creating tables...
Creating table x
Creating table y
Running deferred SQL...
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "~/venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
utility.execute()
File "~/venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 330, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "~/venv/lib/python2.7/site-packages/django/core/management/base.py", line 390, in run_from_argv
self.execute(*args, **cmd_options)
File "~/venv/lib/python2.7/site-packages/django/core/management/base.py", line 441, in execute
output = self.handle(*args, **options)
File "~/venv/lib/python2.7/site-packages/django/core/management/commands/syncdb.py", line 25, in handle
call_command("migrate", **options)
File "~/venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 120, in call_command
return command.execute(*args, **defaults)
File "~/venv/lib/python2.7/site-packages/django/core/management/base.py", line 441, in execute
output = self.handle(*args, **options)
File "~/venv/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 179, in handle
created_models = self.sync_apps(connection, executor.loader.unmigrated_apps)
File "~/venv/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 317, in sync_apps
cursor.execute(statement)
File "~/venv/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "~/venv/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "~/venv/lib/python2.7/site-packages/django/db/utils.py", line 97, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "~/venv/lib/python2.7/site-packages/django/db/backends/utils.py", line 62, in execute
return self.cursor.execute(sql)
django.db.utils.ProgrammingError: relation "auth_user" does not exist
I tried deleting the database and recreating it.
Also, I tried:
python manage.py migrate auth
which also fails:
django.db.utils.ProgrammingError: relation "django_site" does not exist
LINE 1: SELECT (1) AS "a" FROM "django_site" LIMIT 1
Please help get this fixed.
I didn't like the idea of commenting/uncommenting code, so I tried a different approach: I migrated "manually" some apps, and then run django-admin.py migrate for the remaining ones. After deleting all the *.pyc files, my sequence of commands was:
$ django-admin.py migrate auth
$ django-admin.py migrate contentypes
$ django-admin.py migrate sites
$ django-admin.py migrate MY_CUSTOM_USER_APP
$ django-admin.py migrate
where MY_CUSTOM_USER_APP is the name of the application containing the model I set AUTH_USER_MODEL to in my settings file.
Hope it can help. Btw, it seems strange that the best way to synchronize your db in Django 1.8 is so complicated. I wonder if I'm missing something (I'm not very familiar with Django 1.8, I used to work with older versions)
Working on Django 1.10 I found out another solution:
My application is named "web", and first I call:
python manage.py makemigrations web
then I call:
python manage.py makemigrations auth
then I call:
python manage.py migrate
Amazed: IT'S WORKING! :)
It seems auth was searching for the AUTH_USER_MODEL "web.UserProfile" and a relation named web_user_profile, and it didn't find it, hence the error.
On the other hand, calling makemigrations web first creates the required relation first, before auth is able to check and alert it's not there.
Always migrate db with python manage.py makemigrations and then python manage.py migrate in newer versions. For the error above if first time your are migrating your database then use python manage.py migrate --fake-initial. See docs https://docs.djangoproject.com/en/1.9/ref/django-admin/#django-admin-migrate
I had the same problem, and I spent hours banging my head trying to find a solution, which was hidden in the comments. My problem was that CircleCI couldn't run tests because of this error. And I thought I would need to start fresh with a new and empty DB. But I got the same errors. Everything was seemingly related to 'auth', 'contenttypes' and 'sites'.
I read this, and this, as well as this and also this. None were solutions for me.
So after having destroyed my DB and created a new one, the only solution I found to entirely avoid these django.db.utils.ProgrammingError was to:
Comment out all code related to the User model.
Delete all .pyc files in my project! find . -name "*.pyc" -exec rm -- {} + Thanks #max!
run ./manage.py migrate (no fake, no fake-initial, no migration of 'auth' or 'contenttypes' before, juste plain migrate.
Uncomment the above code, and run migrate again!
My INSTALLED_APP is the following:
INSTALLED_APPS = (
'django.contrib.admin',
'django.contrib.contenttypes',
'django.contrib.sites',
'django.contrib.auth',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'mptt',
'djangobower',
'honeypot',
'django_hosts',
'leaflet',
'multiselectfield',
'corsheaders',
'rest_framework_swagger',
'allauth',
'allauth.account',
# 'allauth.socialaccount',
# 'allauth.socialaccount.providers.twitter',
# 'allauth.socialaccount.providers.facebook',
'project.<app_name>',
)
Deleting migration files, associated .pyc files, and just to be safe all .pyc files with the following commands did not solve my issue.
$ find . -path "*/migrations/*.py" -not -name "__init__.py" -delete
$ find . -path "*/migrations/*.pyc" -delete
$ find . -name "*.pyc" -exec rm -- {} +
What ended up solving my issue wasn't clearing caches, it was because I had a function that performed a query as a default function parameter. On init, which is what commands like makemigrations and migrate do before executing it seems like django (perhaps a python attribute?) initializes all default parameters.
As my database was completely empty (I needed to perform migrate --run-syncdb to recreate the tables) when the below default parameter was initialized, it ran a query against the empty database that subsequently failed.
change this:
def generate_new_addresses(number=1, index=None, wallet=get_active_wallet()):
...
...
return
to:
def generate_new_addresses(number=1, index=None, wallet=None):
if not wallet:
wallet = get_active_wallet()
...
...
return
In my case, this error was appearing when the postgresql driver was able to connect to the database, but the provided user does not have access to the schema or the tables, etc. Instead of saying permission denied, the error being shown is saying that the database table being queried is not being found. Typically in such a situation, the migrate command will also fail with a similar error when it tries to create the django_migrations table.
Check for access being granted on the user you're using in database connection in Django.
I had the same issue, but my underlying cause was the __init__.py file in one of the migrations folders had been deleted from source code but not locally (causing 'Not on my machine' errors).
Migrations folders still need __init__.py files, even with Python 3.
I had this issues with a forms.ChoiceForm queryset. I was able to switch to using forms.ModelChoiceForm which are lazily evaluated and this fixed the problem for me.
In My Case
I made migrations using some other similar looking migration file(1).
Then I deleted it, and made 2 new migration files for replacement(2&3).
And then I was getting this error.
In my case the table was renamed using migration file 1
But django was searching for old table name in migration file 3
So I renamed table manually to old name, and applied migration and It was successful
Error is basically because db (postgres or sqlite) have not found the relation, for which you are inserting or else performing CRUD.
The solution is to make migrations
python manage.py makemigrations <app_name>
python manage.py migrate

Switching to PostgreSQL fails loading datadump

Using sqlite3 and Django I want to change to PostgreSQL and keep all data intact. I used ./manage.py dumpdata > dump.json to dump the data, and changed settings to use PostgreSQL.
With an empty database ./manage.py loaddata dump.json resulted in errors about tables not existing, so I ran ./manage.py syncdb and tried again. That results in this error:
Problem installing fixture 'dump.json': Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/django/core/management/commands/loaddata.py", line 163, in handle
obj.save()
File "/usr/lib/python2.6/site-packages/django/core/serializers/base.py", line 163, in save
models.Model.save_base(self.object, raw=True)
File "/usr/lib/python2.6/site-packages/django/db/models/base.py", line 495, in save_base
rows = manager.filter(pk=pk_val)._update(values)
File "/usr/lib/python2.6/site-packages/django/db/models/query.py", line 448, in _update
return query.execute_sql(None)
File "/usr/lib/python2.6/site-packages/django/db/models/sql/subqueries.py", line 124, in execute_sql
cursor = super(UpdateQuery, self).execute_sql(result_type)
File "/usr/lib/python2.6/site-packages/django/db/models/sql/query.py", line 2347, in execute_sql
cursor.execute(sql, params)
File "/usr/lib/python2.6/site-packages/django/db/backends/util.py", line 19, in execute
return self.cursor.execute(sql, params)
IntegrityError: duplicate key value violates unique constraint "django_content_type_app_label_key"
Is this not the correct way to move data from one database to another? How should I switch database backend safely?
The problem is simply that you're getting the content types defined twice - once when you do syncdb, and once from the exported data you're trying to import. Since you may well have other items in your database that depend on the original content type definitions, I would recommend keeping those.
So, after running syncdb, do manage.py dbshell and in your database do TRUNCATE django_content_type; to remove all the newly-defined content types. Then you shouldn't get any conflicts - on that part of the process, in any case.
There is a big discussion about it on the Django ticket 7052. The right way now is to use the --natural parameter, example: ./manage.py dumpdata --natural --format=xml --indent=2 > fixture.xml
In order for --natural to work with your models, they must implement natural_key and get_by_natural_key, as described on the Django documentation regarding natural keys.
Having said that, you might still need to edit the data before importing it with ./manage.py loaddata. For instance, if your applications changed, syncdb will populate the table django_content_type and you might want to delete the respective entries from the xml-file before loading it.
This worked for me. You probably want to ensure the server is stopped so no new data is lost. Dump it:
$ python manage.py dumpdata --exclude auth.permission --exclude contenttypes --natural > db.json
Make sure your models don't have signals (e.g. post_save) or anything that creates models. If you do, comment it out momentarily.
Edit settings.py to point to the new database and set it up:
$ python manage.py syncdb
$ python manage.py migrate
Load the data:
./manage.py loaddata db.json
I used pgloader, just take a few seconds to migrate successfully:
$ pgloader project.load
project.load file with:
load database
from sqlite:////path/to/dev.db
into postgresql://user:pwd#localhost/db_name
with include drop, create tables, create indexes, reset sequences
set work_mem to '16MB', maintenance_work_mem to '512 MB';

Django beginner problem: manage.py dbsync

I'm running ubuntu 9.04 32b and got django from Synaptics.
My settings.py is configured for a sqlite3 database.
I've been through this tutorial and got the following error when trying to run the command python manage.py syncdb :
Traceback (most recent call last):
File "manage.py", line 11, in
execute_manager(settings)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/__init__.py", line 340, in execute_manager
utility.execute()
File "/usr/local/lib/python2.6/dist-packages/django/core/management/__init__.py", line 295, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 192, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 219, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 348, in handle
return self.handle_noargs(**options)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/commands/syncdb.py", line 51, in handle_noargs
cursor = connection.cursor()
File "/usr/local/lib/python2.6/dist-packages/django/db/backends/__init__.py", line 56, in cursor
cursor = self._cursor(settings)
File "/usr/local/lib/python2.6/dist-packages/django/db/backends/sqlite3/base.py", line 145, in _cursor
self.connection = Database.connect(**kwargs)
sqlite3.OperationalError: unable to open database file
Do anyone have a clue on my problem ?
In settings.py are you using a relative path to the sqlite file?
If you are, try changing that to an absolute path.
i.e. instead of:
~/project/mydata.db
use
/home/user/project/mydata.db
This can also happen if your database name is the same as your project name. To fix it just change the name of your db e.g. by adding a .db to the NAME. So if your DATABASE NAME in settings.py was 'epic_project' change it to 'epic_project.db'
Long and boring explanation:
If you start your project by running:
django startproject epic_project
your folder structure will be like this:
/path/to/epic_project/
manage.py
epic_project/
settings.py
if then in your settings.py you set your database as:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'epic_project',
...
}
}
when
python manage.py syncdb
runs it tries to open or create a sqlite db file at /path/to/epic_project/epic_project, but there is a directory there, so it says "oh ok the path exists already, let's open it as an sqlite db", unfortunately for sqlite it's a directory and not a DB, so it cries and django presents these tears to you as "sqlite3.OperationalError: unable to open database file"
might be a permission problem, does your user have sufficient right to write on the folder? for example if you do
sudo python manage.py syncdb
instead, does it work?
For Google's sake:
The path to the database must be the full path to the file --- not just the directory where the file lives.
I had the same problem on Windows then realized that syncdb would not create the specified folder if it didn't already exist. I had specified c:/mysite/db/sqlite3.db in settings but the /db/ folder didn't exist. Created it in terminal then re-ran syncdb successfully.
I had the same problem two days ago. I solved it by setting the 'NAME' in the DATABASE dictionary (settings.py) to the absolute path.
For example.
settings.py
DATABASES = {
'default' : {
'ENGINE' : 'django.db.backends.sqlite3',
'NAME' : DATABASE_PATH,
}
}
here you can set the DATABASE_PATH at the top of the settings.py as such
(Not sure if this will work for windows)
SETTINGS_DIR = os.path.dirname(__file__)
PROJECT_PATH = os.path.abspath(os.path.join(SETTINGS_DIR, os.pardir))
DATABASE_PATH = os.path.abspath(os.path.join(PROJECT_PATH, 'your-database-name'))
For Windows you might have to use the replace method. (Not sure .. But you can try it out as follows)
PROJECT_PATH = os.path.abspath(os.path.join(SETTINGS_DIR, os.pardir).replace('\\', '/'))
Same goes for the DATABASE_PATH.
PS. Correct me if I am wrong.
Similar to answer from user104264 - different perspective...
Currently following the YouTube tutorial I had the same error. Seems to me while running the manage.py in a virtualenv django project directory ...path to virtual env.../django_project/, the database name inside /myapp/settings.py was simply 'storage.db' so
(django-v)...path to virtual env project.../django_project/>python manage.py syncdb
created ...path to virtual env project.../django_project/storage.db
-Bill
If you are specifying a full path to db file and are still having issues, try putting an r before the string.
ie.
r'C:\home\usr\mysite\sqlite3.db'
The problem I was running into was a bootstrapping issue where some model lookups were being done at import (outside of any class or function). Per the docs, this is a bad idea.