When i run my app test django don't create my app tables and throw an error.
My test file:
from django.test import TestCase
class MyTest(TestCase):
fixtures = ['initial_data.json']
def test_my_stuff(self):
[...]
When i run test:
DatabaseError: Problem installing fixture
'/home/.../djStock/stock/fixtures/initial_data.json': Could
not load stock.Provider(pk=1): (1146, "Table
'test_djstock.stock_provider' doesn't exist")
My app is correctly added in INSTALLED_APPS. What i miss ?
Must be have south migration files. Check if myapp/migration/ contain files for migration.
You have to comment out 'south' in INSTALLED_APPS before running any tests, otherwise if models are updated, you Django won't have the update reflected in the table creation, thus you get the error that table does not exist.
Assuming you are using Django 1.6.
Related
Is there a way to exclude database from django migrations?
I have a sphinxsearch database in my django project:
DATABASES['sphinxsearch'] = {
'ENGINE': 'sphinxsearch.backend.sphinx',
...
}
And when I try to run manage.py makemigrations command, Django tries to run
SHOW FULL TABLES query against it
which leads to an error, because this is wrong syntax for sphinxql
File "C:\Anaconda\lib\site-packages\django\db\backends\mysql\introspection.py", line 56, in get_table_list
cursor.execute("SHOW FULL TABLES")
...
django.db.utils.ProgrammingError: (1064, "sphinxql: syntax error, unexpected IDENT, expecting VARIABLES near 'FULL TABLES'")
An exception to this rule is the makemigrations command.
It validates the migration history in the databases to catch problems with the existing migration files (which could be caused by editing them) before creating new migrations.
By default, it checks only the default database, but it consults the allow_migrate() method of routers if any are installed.
makemigrations always creates migrations for model changes, but if allow_migrate() returns False, any migration operations for the model_name will be silently skipped when running migrate on the db.
Changing the behavior of allow_migrate() for models that already have migrations may result in broken foreign keys, extra tables, or missing tables. When makemigrations verifies the migration history, it skips databases where no app is allowed to migrate.
class DBMigrateRouter(object):
"""
A router to control all database operations on models in the
auth application.
"""
def allow_migrate(self, db, app_label, model_name=None, **hints):
"""
Allows migration for default DB
"""
return db == 'default'
Add this class to settings file
DATABASE_ROUTERS = ['path.to.DBMigrateRouter']
This will skip migrations on databases other than "default".
I created an app under edx-platform/djangoapps.
Under that I created a model.
After that I ran
paver update_db -s devstack
to update database and make migrations.
I login to django shell via
./manage.py lms --settings aws shell
and import my model via
from myapp.models import MyModel
it imports successfully without errors but when i ran
MyModel.objects.all()
I encounter this error:
DatabaseError: (1146, "Table 'edxapp.myapp_mymodel' doesn't exist")
What am I missing here?
Make sure you have added your new Django application to the INSTALLED_APPS list in the LMS settings:
INSTALLED_APPS = (
...
'yourapp',
Then re-run the lms migrations:
$ paver update_db -s devstack
I want to load a fixture for my selenium testing. Using fixtures was successful in my initial tests, so I know I am capable of loading the fixtures in my test setup and using them in my tests. I have attempted several approaches.
First, I generated fixtures specific to the models I was testing using dumpdata. An example is below:
python manage.py dumpdata protocols.Step --indent=2 > functional_tests/fixtures/step.json
When used in my test as so:
class SignInTest(FunctionalTest):
fixtures = ['admin_user.json', 'protocol.json', 'step.json',
'protocol_element.json']
def test_login_and_view_user_data(self):
...
I get error:
django.db.utils.IntegrityError: Problem installing fixtures: The row in table 'protocols_protocolelement' with primary key '37' has an invalid foreign key: protocols_protocolelement.element_content_type_id contains a value '41' that does not have a corresponding value in django_content_type.id.
Second attempt involved using all the test data in my tables, but excluding contenttypes:
python manage.py dumpdata --indent=2 -e contenttypes > functional_tests/fixtures/initial_data.json
class SignInTest(FunctionalTest):
fixtures = ['initial_data.json']
...
Getting the error:
django.db.utils.OperationalError: Problem installing fixture '.../mike/mike/functional_tests/fixtures/initial_data.json': Could not load auth.Permission(pk=103): no such table: auth_permission
Next, I tried using natural to show the natural keys:
python manage.py dumpdata --natural -e contenttypes -e auth.Permission --indent=2 > functional_tests/fixtures/initial_data2.json
Only to get the error:
django.db.utils.OperationalError: Problem installing fixture '.../mike/mike/functional_tests/fixtures/initial_data.json': Could not load auth.User(pk=1): no such table: auth_user
Noticing natural was depreciated I tried --natural-foreign and wanted to include user and permission models (I need contenttypes for my models anyway):
python manage.py dumpdata --natural-foreign --indent=2 > functional_tests/fixtures/initial_data3.json
Only to get the error:
django.db.utils.IntegrityError: Problem installing fixture '.../mike/mike/functional_tests/fixtures/initial_data3.json': Could not load contenttypes.ContentType(pk=35): UNIQUE constraint failed: django_content_type.app_label, django_content_type.model
So, any ideas on how to load the fixture so I can run my tests? Is there something simple I'm missing? Thanks!
After some more reading about how Django maintains its own models and such, it is my understanding that Django caches the contenttype, auth.Permission, etc and uses them in testing frameworks (I was using StaticLiveServerTestCase). This means that when I was loading my fixture, it was clashing with the data Django had stored for its own uses causing the integrity error. This is what worked for me:
python manage.py dumpdata -e contenttypes -e admin -e auth.Permission --natural-foreign --indent=2 > functional_tests/fixtures/initial_data4.json
This post has some additional helpful information to help me solve the problem:
Problems with contenttypes when loading a fixture in Django.
All this exercises around dumpdata didn't bring me any result.
But I found this advise:
https://stackoverflow.com/a/3855764/12191031
For some reason reset doesn't work in my code, but it was enough load fixture before every test and flush DB after each one.
So, my solution is this:
from django.core import management
from django.contrib.staticfiles.testing import StaticLiveServerTestCase
from selenium import webdriver
class login_page_test(StaticLiveServerTestCase):
#classmethod
def setUpClass(cls):
cls.selenium = webdriver.Firefox()
cls.selenium.implicitly_wait(5)
super(login_page_test, cls).setUpClass()
#classmethod
def tearDownClass(cls):
cls.selenium.quit()
super(login_page_test, cls).tearDownClass()
def setUp(self):
# for every test I load fixture
management.call_command("loaddata", "user-data.yaml", verbosity=0)
super(login_page_test, self).setUp()
def tearDown(self):
# after each test I flush all DB
management.call_command("flush", verbosity=0, interactive=False)
super(login_page_test, self).setUp()
I'm happy )))
Django 1.7 introduced database migrations.
When running the unit tests in Django 1.7, it forces a migrate, that takes a long time. So I would like to skip the django migrations, and create the database in the final state.
I know that ignoring the migrations can be a bad practice, as that part of the code would not be tested. But that's not the case: I'm running the full migrations in the CI test server (jenkins). I only want to skip the migrations in my local tests, where the speed matters.
Some context:
Until Django 1.6, when using South, I used the SOUTH_TESTS_MIGRATE setting:
By default, South’s syncdb command will also apply migrations if it’s run in non-interactive mode, which includes when you’re running tests - it will run every migration every time you run your tests.
If you want the test runner to use syncdb instead of migrate - for example, if your migrations are taking way too long to apply - simply set SOUTH_TESTS_MIGRATE = False in settings.py.
However, syncdb does not exist anymore, now it's migrate.
And from Django 1.8 I'll use the --keepdb parameter:
The --keepdb option can be used to preserve the test database between test runs. This has the advantage of skipping both the create and destroy actions which greatly decreases the time to run tests, especially those in a large test suite. If the test database does not exist, it will be created on the first run and then preserved for each subsequent run. Any unapplied migrations will also be applied to the test database before running the test suite.
So this question is limited to Django 1.7.
Look at this workaround, posted by Bernie Sumption to the Django developers mailing list:
If makemigrations has not yet been run, the "migrate" command treats
an app as unmigrated, and creates tables directly from the models just
like syncdb did in 1.6. I defined a new settings module just for unit
tests called "settings_test.py", which imports * from the main
settings module and adds this line:
MIGRATION_MODULES = {"myapp": "myapp.migrations_not_used_in_tests"}
Then I run tests like this:
DJANGO_SETTINGS_MODULE="myapp.settings_test" python manage.py test
This fools migrate into thinking that the app is unmigrated, and so
every time a test database is created it reflects the current
structure of models.py.
In Django 1.9, this situation is improved somewhat, and you can set the value to None:
MIGRATION_MODULES = {"myapp": None}
Here is the end of my settings file :
class DisableMigrations(object):
def __contains__(self, item):
return True
def __getitem__(self, item):
return None
TESTS_IN_PROGRESS = False
if 'test' in sys.argv[1:] or 'jenkins' in sys.argv[1:]:
logging.disable(logging.CRITICAL)
PASSWORD_HASHERS = (
'django.contrib.auth.hashers.MD5PasswordHasher',
)
DEBUG = False
TEMPLATE_DEBUG = False
TESTS_IN_PROGRESS = True
MIGRATION_MODULES = DisableMigrations()
based on this snippet
I disabled migrations only when tests are running
django-test-without-migrations adds a --nomigrations flag to manage.py test. Works like a charm.
From Django version 3.1 and up the correct way to do this is to use the MIGRATE setting in the database settings dict. See also the documentation.
#settings.py
DATABASES = {
'TEST': {
'NAME': 'Foo',
'MIGRATE': False
}
}
Update: Never mind, this change was reverted before 1.10 final was released. Hopefully it will return in a future version.
Note that as of Django 1.10 this can be controlled by a test database setting.
MIGRATE
Default: True
If set to False, Django won’t use migrations to create the test database.
I just figure out how to disable migrations after django 1.10,may be it could help for somebody. Here is link at git
class DisableMigrations(dict):
def __contains__(self, item):
return True
def __getitem__(self, item):
return None
DATABASES = DisableMigrations()
MIGRATION_MODULES = DisableMigrations()
Migrations for django 1.10 has two part,please look at load_disk and recorder
The part of load_disk for migrations model of app that be added at INSTALL_APP
And the part of recorder for database connection
For the version before 1.9 we need set MIGRATION_MODULES={'do.not.migrate':'notmigrations'} when you are running test
Now we need set it None like MIGRATION_MODULES={'do.not.migrate':None}
So if we do not want make migrations for any app, just extend a dict and return None for getitem function , and do the same at DATABASES, that is the right thing you need to do
PS: For command, you need to specify --setting=module.path.settings_test_snippet after test
PPS If you are working with pycharm ,do not set --settings options at Run/Debug configurations, just add path of settings_test_snippet.py at Custom setting. That just be fine!!
enjoy
https://gist.github.com/apollovy/22826f493ad2d06d9a9a22464730ce0b
MIGRATION_MODULES = {
app[app.rfind('.') + 1:]: 'my_app.migrations_not_used_in_tests'
for app in INSTALLED_APPS
}
For django 1.9 and up the answer of Guillaume Vincent does not work anymore, so here's a new solution:
I'm using this snippet in my settings file, after the definition of the INSTALLED_APPS
if os.environ.get('TESTS_WITHOUT_MIGRATIONS', False):
MIGRATION_MODULES = {
app.split('.')[-1]: None for app in INSTALLED_APPS
}
It iterates over all installed apps and marks each as having no migration module. See the django docs for more information.
Using this snippet you can run your tests, setting the environment variable TESTS_WITHOUT_MIGRATIONS, e.g.:
TESTS_WITHOUT_MIGRATIONS=1 ./manage.py test
I am unable to use syncdb because my app uses some MySQL views. I have run manage.py sqlall <app>, but this does not output the SQL for django_content_type table or the auth_permission tables. I have also had a look into south and django evolution, but they both require syncdb, and I'm not sure they would help anyway.
I have manually added some models to the tables, but this is getting frustrating, and having installed the dbsettings app I am unsure of what I now need to enter.
Does anyone know of a way to get manage.py (or something else) to output the SQL for these tables and their contents?
Thanks.
Having done a bit more digging, I found these:
Fixing the auth_permission table after renaming a model in Django and manage.py sql command for django models - Django.
These output the tables, but not the data:
python manage.py sql auth
python manage.py sql admin
But this gets a lot closer. In the end I managed it with the following:
from django.contrib.auth.management import create_permissions
from django.db.models import get_apps
for app in get_apps():
create_permissions(app, None, 2)
from django.contrib.contenttypes.management import update_all_contenttypes
update_all_contenttypes(interactive=True)
This adds all the permissions and then all the content types which are needed. interactive=True means that it asks you if you want to remove stale content types.
#hajamie solution works for older supported version, taking a hint, below is what worked for me!
django = 1.9.7
from django.contrib.auth.management import create_permissions
from django.contrib.auth.models import Permission
from django.apps import apps
def fix_user_permission():
"""
run this method via shell whenever any amendments in any of the tables is made
"""
print "fixing user permissions"
# delete pre-existing user permission
Permission.objects.all().delete()
apps.models_module = True
create_permissions(apps, verbosity=0)
apps.models_module = None
print "process completed - fixed user permissions"
The easiest solution I found is to install Django Extensions, add it to settings.INSTALLED_APPS and run:
manage.py update_permissions