Under my Django project there are a few apps and all of them have unit tests. One of them that I'm working right now is supposed to be included only in dev/stage environments, so I'm enabling it using a environment variable.
When this variable is present it is added to INSTALLED_APPS and it is working just fine, the problem is that Django is executing the tests for this app even when it is not in INSTALLED_APPS, and it fails with the following message:
ImportError: Failed to import test module: debug.tests.unit.test_services`
...(traceback information)...
RuntimeError: Model class debug.models.Email doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.
When I define the app_label in the class Meta of models in this app the error is different, it says it can't find a table, I assume that this is because the app is not in INSTALLED_APPS, so it's migrations are not executed.
OperationalError: no such table: debug_email
I'm not sure why Django executes the tests for all apps, but not it's migrations.
Am I missing something from Django configuration for tests?
https://docs.python.org/3/library/unittest.html#unittest.TestLoader.discover says:
If load_tests exists then discovery does not recurse into the package, load_tests is responsible for loading all tests in the package.
So in the lowest __init__.py in your app which you don't always want run:
from django.apps import apps
def load_tests(loader, tests, pattern):
from django.conf import settings
if apps.is_installed("your_dev_app"):
# Actually load the tests - thanks to #barney-szabolcs
return loader.discover(start_dir=dirname(abspath(__file__)), pattern=pattern)
You need to return the discovered tests in load_tests.
So, adding to #DaveLawrence's answer, the complete code is:
# your_dev_app/__init__.py
from django.apps import apps
from os.path import dirname, abspath
def load_tests(loader, tests, pattern):
"""
loads tests for your_dev_app if it is installed.
"""
from django.conf import settings
if apps.is_installed("your_dev_app"):
return loader.discover(start_dir=dirname(abspath(__file__)), pattern=pattern)
When you run:
python manage.py test
the command will look per default recursive for all files with the pattern test*.py in the working directory. It isn't affected by INSTALLED_APPS in settings.py.
You can specify a certain app to test it:
python manage.py test app_label
or specify a path:
python manage.py test myapp/tests
If you want to exclude some tests you can tag them and use the option --exclude-tag.
Run python manage.py test --help to get information on all options.
The official documentation gives a lot of information on the different possibilities how to run the tests.
EDIT:
If you have apps that are required only in the development environment, but not in the production, you could split your settings.py. One possible solution would be to outsource all development settings into a file local_settings.py and exclude it from versioning or from the production branch, i.e. don't push it in the production environment.
local_settings.py
DEBUG = True
INSTALLED_APPS += (
# Django Debug Toolbar would be for example
# used only in development
'debug_toolbar',
'your dev app',
)
settings.py
try:
from .local_settings import *
except ImportError:
pass
I have been struggling for this issue for the whole days while no solutions at all. so I post it here.
I am trying to set up a blog website in Heroku via Django 1.8 which uses Python 3.4.3. I follows the instructions from Heroku website here.
I use "foreman start" to run Django project in my Mac and I already installed all dependence.
Part of my setting.py file involving the database initially looks like:
import dj_database_url
DATABASES = {}
DATABASES['default'] = dj_database_url.config()
Then I got error: ImproperlyConfigured at /settings.DATABASES is improperly configured. Please supply the ENGINE value.
Then I modify the files by adding one line supplying the ENGINE value:
import dj_database_url
DATABASES = {}
DATABASES['default'] = dj_database_url.config()
DATABASES['default']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'
Based on this post answered by Or Arbel, it should work. But I got another error: ImproperlyConfigured at /settings.DATABASES is improperly configured. Please supply the NAME value.
What should I do next? Actually my Django project is very simple and does not involve any database operations(may need in the future). I just want to make it works on Heroku. Thanks!
Do I need to create a database to continue? I just want to make the webpage works.
Thanks for your guys help, specially souldeux.
Update:
I have fixed the issue by using souldeux's method by providing more informations about the database. Here I want to emphasis that it seems the code from the original Heroku tutorial does not work for Django 1.8:
import dj_database_url ####not working for my case
DATABASES = {}
DATABASES['default'] = dj_database_url.config()
Initially I did not create a database because I think it is not necessary for simple projects, based on my understanding obtained from Heroku tutorial. Actually it does need to create a database in Heroku to make it works. The tutorial is here. You need run "heroku config -s | grep HEROKU_POSTGRESQL" to get the database information. The format is like:
scheme://username:password#host:port/database
So you can get 'database', 'username', 'password', etc.
Afterwards, modify the 'settings.py' according to souldeux, then run following codes:
git add .
git commit -m "Ready to go to Heroku"
git push heroku master
heroku run python manage.py syncdb
Now it works. But other issues arise like my webpages do not show images... Anyway it solved. Please confirm my solutions, thanks.
I think you need to add more information to your database definition. For instance, here's what my own database entry looks like in my settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'django',
'USER': 'redacted',
'PASSWORD': 'redacted',
'HOST': '127.0.0.1',
'PORT': '5432',
}
}
If you don't have a database user and password to enter in the fields marked redacted then you need to make sure you have the actual database created and psycopg2 installed.
I am building a Django website and I am using Selenium to test my pages. My problem is that when I run the tests, the browser is launched, but no page is loaded or even attempted to be loaded. It just opens blank and the tests hang. It seems to me that the liveserver doesn't get started. I am running on Apache2 and WSGI, but my understanding is that the Selenium tests are run by the the Django's built-in web server. Any idea what could be wrong? The relevant files are below:
tests.py:
from selenium.webdriver.firefox.webdriver import WebDriver
class MyProjectLiveServerTestCase(LiveServerTestCase):
#classmethod
def initSeleniumDriver(cls):
cls.driver = WebDriver()
#classmethod
def closeSeleniumDriver(cls):
cls.driver.quit()
def testIndexShouldLoad(self):
self.driver.get('%s%s' % (self.live_server_url, '/nd5/mybook/'))
self.assertEqual(len(self.driver.find_elements(
By.CSS_SELECTOR,
'span#copyright'
)), 1)
settings.py:
# Test database runs on SQLite
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(os.path.realpath(os.path.dirname(__file__)), '..', 'myprojectdb'),
}
}
TEST_RUNNER = 'django_nose.NoseTestSuiteRunner'
I am using django-nose, so I execute the tests this way:
python manage.py test --exe
Please, let me know if you need to see any other parts of the code.
UPDATE:
Here is update: I found out that the reason Firefox doesn't load the page is because my version of Firefox is newer than the latest version supported by Selenium. So I switched to Chrome and now the URL in the browser is requested. However, the page isn't found (404 error). This must mean that the liveserver is still not running. My tests don't turn on the liveserver when they get run. Any idea why? The port isn't blocked - I checked.
I think you forgot to select webdriver to use:
class SomethingTest(LiveServerTestCase):
def setUp(self):
self.browser = webdriver.Firefox()
self.browser.implicitly_wait(2)
def tearDown(self):
self.browser.quit()
def test_user_can_log_in(self):
self.browser.get(self.live_server_url + reverse('something'))
self.fail('write rest of the test')
This probably wasn't your problem, but what bit me was that LiveServerTestCase starts the server thread from setUpClass, which I'd defined without calling super(MyProjectLiveServerTestCase, self).setUpClass().
In Django I'm using multple databases, but by testing the functionality 1 database of the multiple database should not create a test database if you run manage.py test **. How can I solve this issue.
In your settings.py you can tell Django what database to use when testing. Here is the code to do so if you would like to use SQLite for example:
settings.py:
if 'test' in sys.argv:
DATABASES['default'] = {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'tests.db',
}
Django creates a new database every time you run your manage.py test, so you would want to use fixtures in your test.py file to preload you database with data before running tests (if you need it preloaded with data).
To generate a fixture for the User model for example, use this code:
python manage.py dumpdata auth.User --indent=2 > app_name/fixtures/admin.json
To use fixtures you would structure your tests.py like so:
tests.py:
from django.test import TestCase
class MyTests(TestCase):
fixtures = [admin.json]
def setUp(self):
# other setup code here
def test_the_obvious(self):
assert True == True
Is there any simple mechanism for overriding Django settings for a unit test? I have a manager on one of my models that returns a specific number of the latest objects. The number of objects it returns is defined by a NUM_LATEST setting.
This has the potential to make my tests fail if someone were to change the setting. How can I override the settings on setUp() and subsequently restore them on tearDown()? If that isn't possible, is there some way I can monkey patch the method or mock the settings?
EDIT: Here is my manager code:
class LatestManager(models.Manager):
"""
Returns a specific number of the most recent public Articles as defined by
the NEWS_LATEST_MAX setting.
"""
def get_query_set(self):
num_latest = getattr(settings, 'NEWS_NUM_LATEST', 10)
return super(LatestManager, self).get_query_set().filter(is_public=True)[:num_latest]
The manager uses settings.NEWS_LATEST_MAX to slice the queryset. The getattr() is simply used to provide a default should the setting not exist.
EDIT: This answer applies if you want to change settings for a small number of specific tests.
Since Django 1.4, there are ways to override settings during tests:
https://docs.djangoproject.com/en/stable/topics/testing/tools/#overriding-settings
TestCase will have a self.settings context manager, and there will also be an #override_settings decorator that can be applied to either a test method or a whole TestCase subclass.
These features did not exist yet in Django 1.3.
If you want to change settings for all your tests, you'll want to create a separate settings file for test, which can load and override settings from your main settings file. There are several good approaches to this in the other answers; I have seen successful variations on both hspander's and dmitrii's approaches.
You can do anything you like to the UnitTest subclass, including setting and reading instance properties:
from django.conf import settings
class MyTest(unittest.TestCase):
def setUp(self):
self.old_setting = settings.NUM_LATEST
settings.NUM_LATEST = 5 # value tested against in the TestCase
def tearDown(self):
settings.NUM_LATEST = self.old_setting
Since the django test cases run single-threaded, however, I'm curious about what else may be modifying the NUM_LATEST value? If that "something else" is triggered by your test routine, then I'm not sure any amount of monkey patching will save the test without invalidating the veracity of the tests itself.
You can pass --settings option when running tests
python manage.py test --settings=mysite.settings_local
Although overriding settings configuration on runtime might help, in my opinion you should create a separate file for testing. This saves lot of configuration for testing and this would ensure that you never end up doing something irreversible (like cleaning staging database).
Say your testing file exists in 'my_project/test_settings.py', add
settings = 'my_project.test_settings' if 'test' in sys.argv else 'my_project.settings'
in your manage.py. This will ensure that when you run python manage.py test you use test_settings only. If you are using some other testing client like pytest, you could as easily add this to pytest.ini
Update: the solution below is only needed on Django 1.3.x and earlier. For >1.4 see slinkp's answer.
If you change settings frequently in your tests and use Python ≥2.5, this is also handy:
from contextlib import contextmanager
class SettingDoesNotExist:
pass
#contextmanager
def patch_settings(**kwargs):
from django.conf import settings
old_settings = []
for key, new_value in kwargs.items():
old_value = getattr(settings, key, SettingDoesNotExist)
old_settings.append((key, old_value))
setattr(settings, key, new_value)
yield
for key, old_value in old_settings:
if old_value is SettingDoesNotExist:
delattr(settings, key)
else:
setattr(settings, key, old_value)
Then you can do:
with patch_settings(MY_SETTING='my value', OTHER_SETTING='other value'):
do_my_tests()
You can override setting even for a single test function.
from django.test import TestCase, override_settings
class SomeTestCase(TestCase):
#override_settings(SOME_SETTING="some_value")
def test_some_function():
or you can override setting for each function in class.
#override_settings(SOME_SETTING="some_value")
class SomeTestCase(TestCase):
def test_some_function():
#override_settings is great if you don't have many differences between your production and testing environment configurations.
In other case you'd better just have different settings files. In this case your project will look like this:
your_project
your_app
...
settings
__init__.py
base.py
dev.py
test.py
production.py
manage.py
So you need to have your most of your settings in base.py and then in other files you need to import all everything from there, and override some options. Here's what your test.py file will look like:
from .base import *
DEBUG = False
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'app_db_test'
}
}
PASSWORD_HASHERS = (
'django.contrib.auth.hashers.MD5PasswordHasher',
)
LOGGING = {}
And then you either need to specify --settings option as in #MicroPyramid answer, or specify DJANGO_SETTINGS_MODULE environment variable and then you can run your tests:
export DJANGO_SETTINGS_MODULE=settings.test
python manage.py test
For pytest users.
The biggest issue is:
override_settings doesn't work with pytest.
Subclassing Django's TestCase will make it work but then you can't use pytest fixtures.
The solution is to use the settings fixture documented here.
Example
def test_with_specific_settings(settings):
settings.DEBUG = False
settings.MIDDLEWARE = []
..
And in case you need to update multiple fields
def override_settings(settings, kwargs):
for k, v in kwargs.items():
setattr(settings, k, v)
new_settings = dict(
DEBUG=True,
INSTALLED_APPS=[],
)
def test_with_specific_settings(settings):
override_settings(settings, new_settings)
I created a new settings_test.py file which would import everything from settings.py file and modify whatever is different for testing purpose.
In my case I wanted to use a different cloud storage bucket when testing.
settings_test.py:
from project1.settings import *
import os
CLOUD_STORAGE_BUCKET = 'bucket_name_for_testing'
manage.py:
def main():
# use seperate settings.py for tests
if 'test' in sys.argv:
print('using settings_test.py')
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project1.settings_test')
else:
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project1.settings')
try:
from django.core.management import execute_from_command_line
except ImportError as exc:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
) from exc
execute_from_command_line(sys.argv)
Found this while trying to fix some doctests... For completeness I want to mention that if you're going to modify the settings when using doctests, you should do it before importing anything else...
>>> from django.conf import settings
>>> settings.SOME_SETTING = 20
>>> # Your other imports
>>> from django.core.paginator import Paginator
>>> # etc
I'm using pytest.
I managed to solve this the following way:
import django
import app.setting
import modules.that.use.setting
# do some stuff with default setting
setting.VALUE = "some value"
django.setup()
import importlib
importlib.reload(app.settings)
importlib.reload(modules.that.use.setting)
# do some stuff with settings new value
You can override settings in test in this way:
from django.test import TestCase, override_settings
test_settings = override_settings(
DEFAULT_FILE_STORAGE='django.core.files.storage.FileSystemStorage',
PASSWORD_HASHERS=(
'django.contrib.auth.hashers.UnsaltedMD5PasswordHasher',
)
)
#test_settings
class SomeTestCase(TestCase):
"""Your test cases in this class"""
And if you need these same settings in another file you can just directly import test_settings.
If you have multiple test files placed in a subdirectory (python package), you can override settings for all these files based on condition of presence of 'test' string in sys.argv
app
tests
__init__.py
test_forms.py
test_models.py
__init__.py:
import sys
from project import settings
if 'test' in sys.argv:
NEW_SETTINGS = {
'setting_name': value,
'another_setting_name': another_value
}
settings.__dict__.update(NEW_SETTINGS)
Not the best approach. Used it to change Celery broker from Redis to Memory.
One setting for all tests in a testCase
class TestSomthing(TestCase):
def setUp(self, **kwargs):
with self.settings(SETTING_BAR={ALLOW_FOO=True})
yield
override one setting in the testCase
from django.test import override_settings
#override_settings(SETTING_BAR={ALLOW_FOO=False})
def i_need_other_setting(self):
...
Important
Even though you are overriding these settings this will not apply to settings that your server initialize stuff with because it is already initialized, to do that you will need to start django with another setting module.