On our production server we got the following error when restarting django or try to run 'python manage.py makemigrations'
django.core.exceptions.FieldDoesNotExist: pricing.pricing has no field named 'price_per_hour'
What is strange is that the field price_per_hour was renamed long time ago to price and the migration when well.
But now I got this error every time and it is preventing to make any other model modification (in any app) and migrations.
What I checked :
If I run 'python manage.py showmigrations' every migration is flagged with an X, which if I'm right, means all the migration were done
price_per_hour is no longer find/used in any of the django app / class
class Pricing(models.Model):
price = models.DecimalField(default=5,max_digits=10, decimal_places=2)
class Meta:
ordering = ['-price',]
def __str__(self):
return "{}".format(self.price)
I also exported the matching./current database in sql and we well see that it contains price column and not price_per_hour. And no reference anywhere to price_per_hour
CREATE TABLE public.pricing_pricing (
id integer NOT NULL,
price numeric(10,2) NOT NULL,
);
I also tried to rename the filed price to price_per_hour just in case but it didn't help
For me it seems the error comes from Django rather than the PostgreSQL database but I'm not sure.
Here is the complete traceback
python manage.py makemigrations
Traceback (most recent call last):
File "manage.py", line 21, in <module>
main()
File "manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 328, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 369, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 83, in wrapped
res = handle_func(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/django/core/management/commands/makemigrations.py", line 141, in handle
loader.project_state(),
File "/usr/local/lib/python3.8/site-packages/django/db/migrations/loader.py", line 324, in project_state
return self.graph.make_state(nodes=nodes, at_end=at_end, real_apps=list(self.unmigrated_apps))
File "/usr/local/lib/python3.8/site-packages/django/db/migrations/graph.py", line 315, in make_state
project_state = self.nodes[node].mutate_state(project_state, preserve=False)
File "/usr/local/lib/python3.8/site-packages/django/db/migrations/migration.py", line 87, in mutate_state
operation.state_forwards(self.app_label, new_state)
File "/usr/local/lib/python3.8/site-packages/django/db/migrations/operations/fields.py", line 326, in state_forwards
raise FieldDoesNotExist(
django.core.exceptions.FieldDoesNotExist: pricing.pricing has no field named 'price_per_hour'
I don't know what other things to look for so, any idea or suggestion will be really appreciated
PS : I thought of removing all the migration files and re-running them but as this is a production server I'm afraid to lose the database content or break something.
find . -path "*/migrations/*.py" -not -name "__init__.py" -delete
find . -path "*/migrations/*.pyc" -delete
So what Boris described was the issue I had. There were incoherences in migration files and django couldn't make the migrations.
To be more precise in my case the error 'the pricing.pricing has no field named' was due to the fact that in the initial migration file the field was named price, but in the second migration file it was asked to rename price_per_hour (which didn't exist) to price and so the error
0001_initial.py
migrations.CreateModel(
name='Pricing',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('price', models.DecimalField(decimal_places=2, default=5, max_digits=10)),
0002_auto_20200304_1344.py
operations = [
migrations.RenameField(
model_name='pricing',
old_name='price_per_hour',
new_name='price',
),
So both options described by Boris are those to follow. First you can try to go through all migrations files and hopefully find and fix manually the error.
To find out which one caused trouble I did this https://stackoverflow.com/a/53135777/20025351
And if you cannot fix it manually (that was my case) I made sure to have the database model matching the Django models, then I removed all the migrations files and re-run migration/migrate.
The error is found on the row migrations/operations/fields. Somehow a discrepancy between the database and the migrations files has occurred, or perhaps you applied a RunPython command in your migration file that references to this field.
I also faced a similar issues, and in my case it appeared to be a creation and deletion of a specific field in two old migration files that seemed to be the issue.
The solution for you is to 'fix' the old migration files, or squash your history.
Option 1) Fixing the error would require you to go through all migrations files and manually edit the migrations where this error stems from. If you can run the makemigration command in your local editor in debug mode you can track where this error occurs. Sadly Django's errorhanding on these errors are not that detailed.
Option 2) An easier option, the option which I would go for, is to squash the migration files. You thus remove all the history of the migration files, and reduce it to a single migration step. See docs. A downside is that you lose all your history, so make sure your local, staging and production environment is all synced up. Specifically on production, make sure that the database model (if you use postgres you can use pgadmin) is exactly the same as your Django model seen in the code.
Squashing the migration files actually removes the whole history. In your example, if you have a migration to add a field price_per_hour, and one to rename that field later to price, squashing will merge those migration files into a single action that makes the field price.
Related
I have two development machines for django projects - tower and laptop. I use a private git repo to keep the projects synchronized. I work on tower for awhile, commit the changes to my git repo (including the database), then do a git pull origin master, and git reset --hard origin/master, and then I work on the laptop when I travel.
I seem to have made a mistake somewhere, as when I updated laptop as above, I have an error in my migrations. On tower, all the migrations are current and applied. On laptop, I have several migrations that cannot be applied.
[X] 0044_remove_document_rotation
[ ] 0041_remove_collectiondocument_position
[ ] 0045_merge_20191023_1922
[X] 0045_auto_20191121_1536
[ ] 0046_merge_20200213_1523
[X] 0046_auto_20200213_1541
[ ] 0047_merge_20200213_1546
These migrations are all checked on tower. I get an error when I try to migrate on laptop:
Applying memorabilia.0041_remove_collectiondocument_position...Traceback (most recent call last):
File "./manage.py", line 15, in <module>
execute_from_command_line(sys.argv)
File "/home/mark/.virtualenvs/memorabilia-JSON/lib/python3.6/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/home/mark/.virtualenvs/memorabilia-JSON/lib/python3.6/site-packages/django/core/management/__init__.py", line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/mark/.virtualenvs/memorabilia-JSON/lib/python3.6/site-packages/django/core/management/base.py", line 328, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/mark/.virtualenvs/memorabilia-JSON/lib/python3.6/site-packages/django/core/management/base.py", line 369, in execute
output = self.handle(*args, **options)
File "/home/mark/.virtualenvs/memorabilia-JSON/lib/python3.6/site-packages/django/core/management/base.py", line 83, in wrapped
res = handle_func(*args, **kwargs)
File "/home/mark/.virtualenvs/memorabilia-JSON/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 233, in handle
fake_initial=fake_initial,
File "/home/mark/.virtualenvs/memorabilia-JSON/lib/python3.6/site-packages/django/db/migrations/executor.py", line 117, in migrate
state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
File "/home/mark/.virtualenvs/memorabilia-JSON/lib/python3.6/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
File "/home/mark/.virtualenvs/memorabilia-JSON/lib/python3.6/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
state = migration.apply(state, schema_editor)
File "/home/mark/.virtualenvs/memorabilia-JSON/lib/python3.6/site-packages/django/db/migrations/migration.py", line 114, in apply
operation.state_forwards(self.app_label, project_state)
File "/home/mark/.virtualenvs/memorabilia-JSON/lib/python3.6/site-packages/django/db/migrations/operations/fields.py", line 172, in state_forwards
delay = not old_field.is_relation
AttributeError: 'NoneType' object has no attribute 'is_relation'
The project runs on both tower and laptop.
How do I get these migrations applied on laptop, or remove the ones that are causing the issues?
Have you tried to take the migrations on your laptop?
This marks migrations as applied so you won't run into trouble with them again.
Fake a single migration file:
python manage.py migrate --fake <APP_NAME> <MIGRATION>
Fake all migrations for an app:
python manage.py migrate --fake <APP_NAME>
Fake all migrations:
python manage.py migrate --fake
When I run into problems with syncing like this, as a last resort, I often remove the migrations and begin from scratch. To do this:
Create a backup of all your migrations in your migrations folder (eg. 0001.iniital.py and 0002_auto_20191220_1357.py and others like it in myproject/myproject/myapp/migrations/), and then delete them.
Go to your Django database and delete all the entries in django.migrations.
Back up your tables and then delete them from your MySQL (or other) database.
The above will give you a completely clean slate.
Thereafter:
python manage.py makemigrations
Followed by:
python manage.py migrate
If your models are set up correctly this will recreate all your structure without errors. Finally, inspect the backed up tables and, where there are no differences, replace the existing tables. Where there are differences, make sure you make the necessary data structure modifications before syncing.
I find it's often faster to do this method when dealing with a manageable database size, than to figure out the chain of what went wrong with the migrations.
I'm working with Django 1.11.5 and using PyCharm as my IDE. I've been trying to refactor my app name from "clinicaltrials" to "cancer_trials". PyCharm updated all of my project files accordingly. I then followed the steps in this SO answer to update the appropriate database tables. However, I'm getting the following error when I try run makemigration. I can't seem to figure out what this means and/or what part I'm missing here.
> python manage.py makemigrations
Traceback (most recent call last):
File "manage.py", line 22, in <module>
execute_from_command_line(sys.argv)
File "C:\Python_3.6.1\lib\site-packages\django\core\management\__init__.py", line 364, in execute_from_command_line
utility.execute()
File "C:\Python_3.6.1\lib\site-packages\django\core\management\__init__.py", line 356, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Python_3.6.1\lib\site-packages\django\core\management\base.py", line 283, in run_from_argv
self.execute(*args, **cmd_options)
File "C:\Python_3.6.1\lib\site-packages\django\core\management\base.py", line 330, in execute
output = self.handle(*args, **options)
File "C:\Python_3.6.1\lib\site-packages\django\core\management\commands\makemigrations.py", line 150, in handle
loader.project_state(),
File "C:\Python_3.6.1\lib\site-packages\django\db\migrations\loader.py", line 323, in project_state
return self.graph.make_state(nodes=nodes, at_end=at_end, real_apps=list(self.unmigrated_apps))
File "C:\Python_3.6.1\lib\site-packages\django\db\migrations\graph.py", line 409, in make_state
project_state = self.nodes[node].mutate_state(project_state, preserve=False)
File "C:\Python_3.6.1\lib\site-packages\django\db\migrations\migration.py", line 92, in mutate_state
operation.state_forwards(self.app_label, new_state)
File "C:\Python_3.6.1\lib\site-packages\django\db\migrations\operations\fields.py", line 142, in state_forwards
for name, instance in state.models[app_label, self.model_name_lower].fields:
KeyError: ('cancer_trials', 'cancer_trials')
Here's the function that's throwing the error
def state_forwards(self, app_label, state):
new_fields = []
old_field = None
for name, instance in state.models[app_label, self.model_name_lower].fields:
if name != self.name:
new_fields.append((name, instance))
else:
old_field = instance
state.models[app_label, self.model_name_lower].fields = new_fields
# Delay rendering of relationships if it's not a relational field
delay = not old_field.is_relation
state.reload_model(app_label, self.model_name_lower, delay=delay)
In my experience the easiest solution is to create new app and copy the code:
Create new app with the desired name and add it to settings
Copy/paste code from old app to new app, change references from old app to new app, run makemigrations and migrate
Open database and copy data from old tables to new tables
Check that everything works in new app
Search stackoverflow.com or google how to remove app from project or just leave in there. Unfortunately, I'm not 100 % positive about these steps, somebody please correct me if I'm wrong, but to my recollection:
run python manage.py migrate old_app zero (this unapplies all migrations for an app)
remove app from settings
delete files
Remove the all migrations files within your project
Go through each of your projects apps migration folder and remove everything inside, except the init.py file.
Drop the current database, or delete the db.sqlite3 if it is your case.
Create the initial migrations and generate the database schema
try run again migrationsa and migrate commans
OK
if you want to keep the existing database maybe this guide may help you:
https://simpleisbetterthancomplex.com/tutorial/2016/07/26/how-to-reset-migrations.html
(scenario 2)
A nice answer on how to properly move data between apps, can be found here.
What worked for me is the following:
Export the data to json
./manage.py dumpdata --exclude auth.permission --exclude contenttypes --exclude admin.LogEntry --exclude sessions --indent 2 > <path_out_of_the_project>/db.json
Open the db.json file using a capable editor and rename all the instances of the old app name to the new app name.
Rename your app and all the necessary references into your code.
Delete the database and recreate a new empty one applying all the migrations.
Load the data from the db.json file which include the new app name.
./manage.py loaddata <path_out_of_the_project>/db.json
simple approach:
Manually delete all migrations in the apps by going into each apps' 'Migrations' named directory.
Note: deleting init.py in 'Migrations' named directory will not cause any harm.
Above 'appname' is apps taken one-by-one and migrated using above steps. For new apps added will only show migrated table for the next two 'migrate' and 'sqlmigrate' command work.
After this:
$: python manage.py makemigrations appname
$: python manage.py migrate appname
$: python manage.py sqlmigrate appname 0001
dj-stripe looks like a very attractive way to manage stripe subscriptions with django.
I installed dj-stripe following the instructions here.
Installation seemed to work, in that the installation includes running these three commands, and doing so created and populated new tables in my Postgres DB as well as creating new customer objects in my stripe account.
python manage.py migrate
python manage.py djstripe_init_customers
python manage.py djstripe_init_plans
For some reason, I decided dj-stripe was overkill (stripe's api seems easy enough to use without so much boilerplate) so I uninstalled using (pip uninstall dj-stripe), removed the mention from INSTALLED_APPS and the entry in urls.py.
To finish the clean-up, I deleted the customers which had been created in my stripe.com account, then I used DROP TABLE to manually delete the leftover tables in my Postgres DB (all tables starting with djstripe_: tables like djstripe_customer, djstripe_invoice, etc.)
Now, I've decided I actually do want to use dj-stripe, but this time installation is not working. The second of those three commands throws a ProgrammingError
requirements.txt includes:
stripe==1.22.3
dj-stripe==0.5.0
Installed dj-stripe using pip...
pip install -r requirements.txt
...and received success messages.
Then following on instructions as before, I ran into a ProgrammingError.
(awe01)MoriartyMacBookAir13:awesomeapp macuser$ python manage.py migrate
Operations to perform:
Synchronize unmigrated apps: django_user_agents, evernote, twitter, polls, widget_tweaks, djrill, sitemaps, facebook, django_slack, storages, hello
Apply all migrations: account, djstripe, admin, sessions, sites, auth, audiotracks, contenttypes, socialaccount
Synchronizing apps without migrations:
Creating tables...
Installing custom SQL...
Installing indexes...
Running migrations:
No migrations to apply.
(I'm not sure what the above was supposed to achieve...
At this point I checked my postgres db: no extra tables have been created. There is no table with name starting with djstripe_)
(awe01)MoriartyMacBookAir13:awesomeapp macuser$ python manage.py djstripe_init_customers
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
utility.execute()
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/core/management/__init__.py", line 377, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/core/management/base.py", line 288, in run_from_argv
self.execute(*args, **options.__dict__)
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute
output = self.handle(*args, **options)
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/djstripe/management/commands/djstripe_init_customers.py", line 15, in handle
for subscriber in get_subscriber_model().objects.filter(customer__isnull=True):
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/models/query.py", line 141, in __iter__
self._fetch_all()
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/models/query.py", line 966, in _fetch_all
self._result_cache = list(self.iterator())
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/models/query.py", line 265, in iterator
for row in compiler.results_iter():
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 700, in results_iter
for rows in self.execute_sql(MULTI):
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
cursor.execute(sql, params)
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/backends/utils.py", line 81, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/Users/macuser/Dropbox/code/hero/awes01/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "djstripe_customer" does not exist
LINE 1: ...r"."date_joined" FROM "auth_user" LEFT OUTER JOIN "djstripe_...
The error
for subscriber in get_subscriber_model().objects.filter(customer__isnull=True):
referrs to line 15 of djstripe_init_customers.py
I don't understand why the normal migration steps are failing (why it expects "djstripe_customer" to have been created before it has been). Grateful for steps I can take to further trouble-shoot. (I'm assuming I've created the gotcha due to dirty uninstall of initial attempt?)
In case relevant:
My auth system is django-allauth
AUTH_USER_MODEL is not explicitly defined in my settings.py file and has been working fine. (I didn't make a custom user model.)
#kavanaugh-development solved this:
To completely remove remnant of the initial install, I had to remove the relevant djstripe rows from the migrations table. "If you don't delete those rows, django will ignore the migration commands the second time around."
DELETE FROM django_migrations WHERE app = 'djstripe';
Once I had done this, a fresh install of dj-stripe worked perfectly as it did the first time round:
I ran python manage.py migrate, which
re-created a few djstripe rows in django_migrations table
also created (empty) tables required for dj-stripe:
djstripe_charge
djstripe_charge_id_seq
djstripe_currentsubscription
djstripe_currentsubscription_id_seq
djstripe_customer
djstripe_customer_id_seq
djstripe_event
djstripe_event_id_seq
djstripe_eventprocessingexception
djstripe_eventprocessingexception_id_seq
djstripe_invoice
djstripe_invoice_id_seq
djstripe_invoiceitem
djstripe_invoiceitem_id_seq
djstripe_plan
djstripe_plan_id_seq
djstripe_transfer
djstripe_transfer_id_seq
djstripe_transferchargefee
djstripe_transferchargefee_id_seq
So the required tables are ready to be populated by the subsequent commands
(such as python manage.py djstripe_init_customers)
So that's a general (basic) lesson learned about migrations; hope this helps someone.
Thanks
I had a working project with django 1.7, and now I moved it to django 1.8.
I can do syncdb and run the app with sqlite, but when I switch to postgres, it fails to do syncdb:
Creating tables...
Creating table x
Creating table y
Running deferred SQL...
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "~/venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
utility.execute()
File "~/venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 330, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "~/venv/lib/python2.7/site-packages/django/core/management/base.py", line 390, in run_from_argv
self.execute(*args, **cmd_options)
File "~/venv/lib/python2.7/site-packages/django/core/management/base.py", line 441, in execute
output = self.handle(*args, **options)
File "~/venv/lib/python2.7/site-packages/django/core/management/commands/syncdb.py", line 25, in handle
call_command("migrate", **options)
File "~/venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 120, in call_command
return command.execute(*args, **defaults)
File "~/venv/lib/python2.7/site-packages/django/core/management/base.py", line 441, in execute
output = self.handle(*args, **options)
File "~/venv/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 179, in handle
created_models = self.sync_apps(connection, executor.loader.unmigrated_apps)
File "~/venv/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 317, in sync_apps
cursor.execute(statement)
File "~/venv/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "~/venv/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "~/venv/lib/python2.7/site-packages/django/db/utils.py", line 97, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "~/venv/lib/python2.7/site-packages/django/db/backends/utils.py", line 62, in execute
return self.cursor.execute(sql)
django.db.utils.ProgrammingError: relation "auth_user" does not exist
I tried deleting the database and recreating it.
Also, I tried:
python manage.py migrate auth
which also fails:
django.db.utils.ProgrammingError: relation "django_site" does not exist
LINE 1: SELECT (1) AS "a" FROM "django_site" LIMIT 1
Please help get this fixed.
I didn't like the idea of commenting/uncommenting code, so I tried a different approach: I migrated "manually" some apps, and then run django-admin.py migrate for the remaining ones. After deleting all the *.pyc files, my sequence of commands was:
$ django-admin.py migrate auth
$ django-admin.py migrate contentypes
$ django-admin.py migrate sites
$ django-admin.py migrate MY_CUSTOM_USER_APP
$ django-admin.py migrate
where MY_CUSTOM_USER_APP is the name of the application containing the model I set AUTH_USER_MODEL to in my settings file.
Hope it can help. Btw, it seems strange that the best way to synchronize your db in Django 1.8 is so complicated. I wonder if I'm missing something (I'm not very familiar with Django 1.8, I used to work with older versions)
Working on Django 1.10 I found out another solution:
My application is named "web", and first I call:
python manage.py makemigrations web
then I call:
python manage.py makemigrations auth
then I call:
python manage.py migrate
Amazed: IT'S WORKING! :)
It seems auth was searching for the AUTH_USER_MODEL "web.UserProfile" and a relation named web_user_profile, and it didn't find it, hence the error.
On the other hand, calling makemigrations web first creates the required relation first, before auth is able to check and alert it's not there.
Always migrate db with python manage.py makemigrations and then python manage.py migrate in newer versions. For the error above if first time your are migrating your database then use python manage.py migrate --fake-initial. See docs https://docs.djangoproject.com/en/1.9/ref/django-admin/#django-admin-migrate
I had the same problem, and I spent hours banging my head trying to find a solution, which was hidden in the comments. My problem was that CircleCI couldn't run tests because of this error. And I thought I would need to start fresh with a new and empty DB. But I got the same errors. Everything was seemingly related to 'auth', 'contenttypes' and 'sites'.
I read this, and this, as well as this and also this. None were solutions for me.
So after having destroyed my DB and created a new one, the only solution I found to entirely avoid these django.db.utils.ProgrammingError was to:
Comment out all code related to the User model.
Delete all .pyc files in my project! find . -name "*.pyc" -exec rm -- {} + Thanks #max!
run ./manage.py migrate (no fake, no fake-initial, no migration of 'auth' or 'contenttypes' before, juste plain migrate.
Uncomment the above code, and run migrate again!
My INSTALLED_APP is the following:
INSTALLED_APPS = (
'django.contrib.admin',
'django.contrib.contenttypes',
'django.contrib.sites',
'django.contrib.auth',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'mptt',
'djangobower',
'honeypot',
'django_hosts',
'leaflet',
'multiselectfield',
'corsheaders',
'rest_framework_swagger',
'allauth',
'allauth.account',
# 'allauth.socialaccount',
# 'allauth.socialaccount.providers.twitter',
# 'allauth.socialaccount.providers.facebook',
'project.<app_name>',
)
Deleting migration files, associated .pyc files, and just to be safe all .pyc files with the following commands did not solve my issue.
$ find . -path "*/migrations/*.py" -not -name "__init__.py" -delete
$ find . -path "*/migrations/*.pyc" -delete
$ find . -name "*.pyc" -exec rm -- {} +
What ended up solving my issue wasn't clearing caches, it was because I had a function that performed a query as a default function parameter. On init, which is what commands like makemigrations and migrate do before executing it seems like django (perhaps a python attribute?) initializes all default parameters.
As my database was completely empty (I needed to perform migrate --run-syncdb to recreate the tables) when the below default parameter was initialized, it ran a query against the empty database that subsequently failed.
change this:
def generate_new_addresses(number=1, index=None, wallet=get_active_wallet()):
...
...
return
to:
def generate_new_addresses(number=1, index=None, wallet=None):
if not wallet:
wallet = get_active_wallet()
...
...
return
In my case, this error was appearing when the postgresql driver was able to connect to the database, but the provided user does not have access to the schema or the tables, etc. Instead of saying permission denied, the error being shown is saying that the database table being queried is not being found. Typically in such a situation, the migrate command will also fail with a similar error when it tries to create the django_migrations table.
Check for access being granted on the user you're using in database connection in Django.
I had the same issue, but my underlying cause was the __init__.py file in one of the migrations folders had been deleted from source code but not locally (causing 'Not on my machine' errors).
Migrations folders still need __init__.py files, even with Python 3.
I had this issues with a forms.ChoiceForm queryset. I was able to switch to using forms.ModelChoiceForm which are lazily evaluated and this fixed the problem for me.
In My Case
I made migrations using some other similar looking migration file(1).
Then I deleted it, and made 2 new migration files for replacement(2&3).
And then I was getting this error.
In my case the table was renamed using migration file 1
But django was searching for old table name in migration file 3
So I renamed table manually to old name, and applied migration and It was successful
Error is basically because db (postgres or sqlite) have not found the relation, for which you are inserting or else performing CRUD.
The solution is to make migrations
python manage.py makemigrations <app_name>
python manage.py migrate
I am trying to run Selenium tests on a Django project (1.5.4), which uses South. I think South is conflicting with my tests when I try to inject initial data with fixtures, but I'm not sure why; I appreciate any help.
According to the Django documentation, fixtures are supposed to be loaded after the first syncdb and then all migrations are applied.
Question 1) Does this take into account South migrations?? Do I need to run those separately somehow?
The error I get when I run my tests makes it seem like my South migrations are still present in the test database after the first test...but I thought each test has its own database (and migrations / fixtures) created? The first test passes / fails, but each subsequent test raises this IntegrityError:
IntegrityError: Problem installing fixture '<PROJECT_PATH>/fixtures/toy_course.json': Could not load contenttypes.ContentType(pk=8): (1062, "Duplicate entry 'south-migrationhistory' for key 'app_label'")
This South documentation and SO question seem to indicate that I need to override some type of forwards method in order to get fixtures working, but I'm not entirely sure how to apply that to a testing situation instead of production (or if that is the solution I need).
Question 2) Am I supposed to override forwards in my test setup? Where would I do it?
My relevant test code:
from django.conf import settings
from selenium import webdriver
from functional_tests.test import SeleniumTestCase
class Resources(SeleniumTestCase):
fixtures = ['toy_course.json']
def setUp(self):
self.browser = webdriver.Chrome(settings.SELENIUM_WEBDRIVER)
self.browser.implicitly_wait(3)
def tearDown(self):
self.browser.quit()
def test_main_page_renders_correctly(self):
"""
User sees a properly formatted main page
"""
self.open('/RDB/')
h3_headers = self.browser.find_elements_by_tag_name('h3')
self.assertIn(
'Complete List of Resources',
[header.text for header in h3_headers])
self.assertTrue(self.check_exists_by_id('main_table'))
self.assertTrue(self.check_exists_by_id('searchDiv'))
self.assertTrue(self.check_exists_by_class_name('tablesorter'))
Thanks!
UPDATE
So per Alex's suggestion below and this South doc, I added this line to my settings.py:
SOUTH_TESTS_MIGRATE = False
But I am now getting 8 of 8 Errors (before I was getting 1 pass/fail on the first test, and then 7 Errors). The full error for a single test is below:
======================================================================
ERROR: test_table_sorts_on_click (functional_tests.tests.main_resources.Resources)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/test/testcases.py", line 259, in __call__
self._pre_setup()
File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/test/testcases.py", line 479, in _pre_setup
self._fixture_setup()
File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/test/testcases.py", line 518, in _fixture_setup
**{'verbosity': 0, 'database': db_name, 'skip_validation': True})
File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/core/management/__init__.py", line 161, in call_command
return klass.execute(*args, **defaults)
File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/core/management/base.py", line 255, in execute
output = self.handle(*args, **options)
File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/core/management/commands/loaddata.py", line 193, in handle
obj.save(using=using)
File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/core/serializers/base.py", line 165, in save
models.Model.save_base(self.object, using=using, raw=True)
File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/db/models/base.py", line 626, in save_base
rows = manager.using(using).filter(pk=pk_val)._update(values)
File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/db/models/query.py", line 605, in _update
return query.get_compiler(self.db).execute_sql(None)
File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 1014, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 840, in execute_sql
cursor.execute(sql, params)
File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 122, in execute
six.reraise(utils.IntegrityError, utils.IntegrityError(*tuple(e.args)), sys.exc_info()[2])
File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 120, in execute
return self.cursor.execute(query, args)
File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/MySQLdb/cursors.py", line 201, in execute
self.errorhandler(self, exc, value)
File "/<PATH TO VIRTUAL ENV>/virtual_environments/relate/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
IntegrityError: Problem installing fixture '/<PATH TO PROJECT>/RDB/fixtures/toy_course.json': Could not load contenttypes.ContentType(pk=8): (1062, "Duplicate entry 'south-migrationhistory' for key 'app_label'")
The command I ran:
$ python manage.py test functional_tests
I'm not quite sure if I made the problem better, worse, or the same, but I seem to be more in-line with the documentation...
Thanks!
UPDATE #2 -- with solution
So a couple of other pages that helped me figure it out (in addition to Alex's pointer to the South doc). First, this person had a similar issue, and solved it using the SOUTH_TESTS_MIGRATE = False statement. So half my solution was to include that.
The second half of my solution was to fix my fixture document. I was dumping everything into my fixture with:
$ python manage.py datadump > RDB/fixtures/toy-course.json
This is, apparently, a bad way to do fixtures it with South--because it also dumps the South migration tables into the fixture. The post above shows the blogger using app-specific fixtures (which are also talked about in this SO post), and that was the key to getting my fixtures to work. The Django docs on fixtures do show the optional parameters to dump just an app, but I didn't know ignoring them would cause South to conflict. So the second half of my solution was to create my fixture to be app-specific:
$ python manage.py datadump RDB > RDB/fixtures/toy-course.json
And my tests now run fine (slow, but probably a different issue)!
Your test database is created using South migrations by default. Set SOUTH_TESTS_MIGRATE = False in your settings.py, quote from docs:
If this is False, South’s test runner integration will make the test
database be created using syncdb, rather than via migrations (the
default).