I'm trying to start writing unit tests for django and I'm having some questions about fixtures:
I made a fixture of my whole project db (not certain application) and I want to load it for each test, because it looks like loading only the fixture for certain app won't be enough.
I'd like to have the fixture stored in /proj_folder/fixtures/proj_fixture.json.
I've set the FIXTURE_DIRS = ('/fixtures/',) in my settings.py.
Then in my testcase I'm trying
fixtures = ['proj_fixture.json']
but my fixtures don't load.
How can this be solved?
How to add the place for searching fixtures?
In general, is it ok to load the fixture for the whole test_db for each test in each app (if it's quite small)?
Thanks!
I've specified path relative to project root in the TestCase like so:
from django.test import TestCase
class MyTestCase(TestCase):
fixtures = ['/myapp/fixtures/dump.json',]
...
and it worked without using FIXTURE_DIRS
Good practice is using PROJECT_ROOT variable in your settings.py:
import os.path
PROJECT_ROOT = os.path.dirname(os.path.realpath(__file__))
FIXTURE_DIRS = (os.path.join(PROJECT_ROOT, 'fixtures'),)
Do you really have a folder /fixtures/ on your hard disk?
You probably intended to use:
FIXTURE_DIRS = ('/path/to/proj_folder/fixtures/',)
Instead of creating fixures folder and placing fixtures in them (in every app), a better and neater way to handle this would be to put all fixtures in one folder at the project level and load them.
from django.core.management import call_command
class TestMachin(TestCase):
def setUp(self):
# Load fixtures
call_command('loaddata', 'fixtures/myfixture', verbosity=0)
Invoking call_command is equivalent to running :
manage.py loaddata /path/to/fixtures
Saying you have a project named hello_django with api app.
Following are steps to create fixtures for it:
Optional step: create fixture file from database: python manage.py dumpdata --format=json > api/fixtures/testdata.json
Create test directory: api/tests
Create empty file __init__.py in api/tests
Create test file: test_fixtures.py
from django.test import TestCase
class FixturesTestCase(TestCase):
fixtures = ['api/api/fixtures/testdata.json']
def test_it(self):
# implement your test here
Run the test to load fixtures into the database: python manage.py test api.tests
I did this and I didn't have to give a path reference, the fixture file name was enough for me.
class SomeTest(TestCase):
fixtures = ('myfixture.json',)
You have two options, depending on whether you have a fixture, or you have a set of Python code to populate the data.
For fixtures, use cls.fixtures, like shown in an answer to this question,
class MyTestCase(django.test.TestCase):
fixtures = ['/myapp/fixtures/dump.json',]
For Python, use cls.setUpTestData:
class MyTestCase(django.test.TestCase):
#classmethod
def setUpTestData(cls):
cls.create_fixture() # create_fixture is a custom function
setUpTestData is called by the TestCase.setUpClass.
You can use both, in which case fixtures is loaded first because setUpTestData is called after loading the fixtures.
You need to import from django.test import TestCase and NOT from unittest import TestCase. That fixed the problem for me.
If you have overridden setUpClass method, make sure you call super().setUpClass() method as the first line in the method. The code to load fixtures is in TestCase class.
Related
I am using django_nose to run my tests. I have a fixture initial_data.json which is an app's folder : my_app/fixtures/initial.json.
The fixture loads fine when using python manage.py syncdb, but doesn't load at all when running the tests python manage.py test my_app (while I guess it should load !?). Any pointer to what could be wrong ?
I quote the official documentation:
Once you've created a fixture and placed it in a fixtures directory in one of your INSTALLED_APPS, you can use it in your unit tests by specifying a fixtures class attribute on your django.test.TestCase subclass:
from django.test import TestCase
from myapp.models import Animal
class AnimalTestCase(TestCase):
fixtures = ['mammals.json', 'birds']
def setUp(self):
# Test definitions as before.
call_setup_methods()
def testFluffyAnimals(self):
# A test that uses the fixtures.
call_some_test_code()
I had similar symptoms but a different solution. I am running my tests through py.test instead of manage.py test.
In appname/fixtures I had initial_data.json and foo.json. My test looked like this:
from django.test import TestCase
class TestFoo(TestCase):
fixtures = ['initial_data.json', 'foo.json']
...
However, when I ran the test initial_data.json was not being loaded, but foo.json was.
For me, the solution was to rename initial_data.json to base.json, and then the fixture was loaded.
I don't think this is the original poster's problem, but posting this so the next person that has the same problem as me can find something. :)
I wrote a unit test checking whether initial data is loaded correctly. However the Node.objects.all().count() always returns 0, thus it seems as the fixtures are not loaded at all. There is no output/error msg in the command line that fixtures are not loaded.
from core.models import Node
class NodeTableTestCase(unittest.TestCase):
fixtures = ['core/core_fixture.json']
def setUp(self):
print "nothing to prepare..."
def testFixture(self):
"""Check if initial data can be loaded correctly"""
self.assertEqual(Node.objects.all().count(), 14)
the fixture core_fixture.json contains 14 nodes and I'm using this fixture as a initial data load into the db using the following command:
python manage.py loaddata core/core_fixture.json
They are located in the folder I provided in the settings.py setting FIXTURE_DIRS.
Found the solution in another thread, answer from John Mee
# Import the TestCase from django.test:
# Bad: import unittest
# Bad: import django.utils.unittest
# Good: import django.test
from django.test import TestCase
class test_something(TestCase):
fixtures = ['one.json', 'two.json']
...
Doing this I got a proper error message, saying that foreign key is violated and I had to also include the fixtures for the app "auth". I exported the needed data with this command:
manage.py dumpdata auth.User auth.Group > usersandgroups.json
Using Unittest I got only the message that loading of fixture data failed, which was not very helpful.
Finally my working test looks like this:
from django.test import TestCase
class NodeTableTestCase2(TestCase):
fixtures = ['auth/auth_usersandgroups_fixture.json','core/core_fixture.json']
def setUp(self):
# Test definitions as before.
print "welcome in setup: while..nothing to setup.."
def testFixture2(self):
"""Check if initial data can be loaded correctly"""
self.assertEqual(Node.objects.all().count(), 11)
When loading fixtures in test cases, I don't think Django allows you to include the directory name. Try changing your fixtures setting to:
fixtures = ['core_fixture.json',]
You might have to change your FIXTURE_DIRS setting as well, to include the core directory.
If you run your tests in verbose mode, you will see the fixture files that Django attempts to load. This should help you debug your configuration.
python manage.py test -v 2
Make sure you have your app listed in INSTALLED_APPS and that your app contains models.py file.
Im writing some code which retrieves info about installed apps, especially defined models, and then does stuff based on that information, but Im having some problems writing a clean, nice unittest. Is there a way to emulate or add an app in unittests without have to run manage.py startproject, manage.py startapp in my testsfolder to have a test app available for unittests?
Sure, try this on for size:
from django.conf import settings
from django.core.management import call_command
from django.test.testcases import TestCase
from django.db.models import loading
class AppTestCase(TestCase):
'''
Adds apps specified in `self.apps` to `INSTALLED_APPS` and
performs a `syncdb` at runtime.
'''
apps = ()
_source_installed_apps = ()
def _pre_setup(self):
super(AppTestCase, self)._pre_setup()
if self.apps:
self._source_installed_apps = settings.INSTALLED_APPS
settings.INSTALLED_APPS = settings.INSTALLED_APPS + self.apps
loading.cache.loaded = False
call_command('syncdb', verbosity=0)
def _post_teardown(self):
super(AppTestCase, self)._post_teardown()
if self._source_installed_apps:
settings.INSTALLED_APPS = self._source_installed_apps
self._source_installed_apps = ()
loading.cache.loaded = False
Your test case would look something like this:
class SomeAppTestCase(AppTestCase):
apps = ('someapp',)
In case you were wondering why, I did an override of _pre_setup() and _post_teardown() so I don't have to bother with calling super() in setUp() and tearDown() in my final test case. Otherwise, this is what I pulled out of Django's test runner. I whipped it up and it worked, although I'm sure that, with closer inspection, you can further optimize it and even avoid calling syncdb every time if it won't conflict with future tests.
EDIT:
So I seem to have gotten out of my way, thinking you need to dynamically add new models. If you've created an app for testing purposes only, here's what you can do to have it discovered during your tests.
In your project directory, create a test.py file that will contain your test settings. It should look something like this:
from settings import *
# registers test app for discovery
INSTALLED_APPS += ('path.to.test.app',)
You can now run your tests with python manage.py test --settings=myproject.test and your app will be in the installed apps.
How can I detect whether a view is being called in a test environment (e.g., from manage.py test)?
#pseudo_code
def my_view(request):
if not request.is_secure() and not TEST_ENVIRONMENT:
return HttpResponseForbidden()
Put this in your settings.py:
import sys
TESTING = len(sys.argv) > 1 and sys.argv[1] == 'test'
This tests whether the second commandline argument (after ./manage.py) was test. Then you can access this variable from other modules, like so:
from django.conf import settings
if settings.TESTING:
...
There are good reasons to do this: suppose you're accessing some backend service, other than Django's models and DB connections. Then you might need to know when to call the production service vs. the test service.
Create your own TestSuiteRunner subclass and change a setting or do whatever else you need to for the rest of your application. You specify the test runner in your settings:
TEST_RUNNER = 'your.project.MyTestSuiteRunner'
In general, you don't want to do this, but it works if you absolutely need it.
from django.conf import settings
from django.test.simple import DjangoTestSuiteRunner
class MyTestSuiteRunner(DjangoTestSuiteRunner):
def __init__(self, *args, **kwargs):
settings.IM_IN_TEST_MODE = True
super(MyTestSuiteRunner, self).__init__(*args, **kwargs)
Just look at request.META['SERVER_NAME']
def my_view(request):
if request.META['SERVER_NAME'] == "testserver":
print "This is test environment!"
There's also a way to temporarily overwrite settings in a unit test in Django. This might be a easier/cleaner solution for certain cases.
You can do this inside a test:
with self.settings(MY_SETTING='my_value'):
# test code
Or add it as a decorator on the test method:
#override_settings(MY_SETTING='my_value')
def test_my_test(self):
# test code
You can also set the decorator for the whole test case class:
#override_settings(MY_SETTING='my_value')
class MyTestCase(TestCase):
# test methods
For more info check the Django docs: https://docs.djangoproject.com/en/1.11/topics/testing/tools/#django.test.override_settings
I think the best approach is to run your tests using their own settings file (i.e. settings/tests.py). That file can look like this (the first line imports settings from a local.py settings file):
from local import *
TEST_MODE = True
Then do ducktyping to check if you are in test mode.
try:
if settings.TEST_MODE:
print 'foo'
except AttributeError:
pass
If you are multiple settings file for different environment, all you need to do is to create one settings file for testing.
For instance, your setting files are:
your_project/
|_ settings/
|_ __init__.py
|_ base.py <-- your original settings
|_ testing.py <-- for testing only
In your testing.py, add a TESTING flag:
from .base import *
TESTING = True
In your application, you can access settings.TESTING to check if you're in testing environment.
To run tests, use:
python manage.py test --settings your_project.settings.testing
While there's no official way to see whether we're in a test environment, django actually leaves some clues for us.
By default Django’s test runner automatically redirects all Django-sent email to a dummy outbox. This is accomplished by replacing EMAIL_BACKEND in a function called setup_test_environment, which in turn is called by a method of DiscoverRunner. So, we can check whether settings.EMAIL_BACKEND is set to 'django.core.mail.backends.locmem.EmailBackend'. That mean we're in a test environment.
A less hacky solution would be following the devs lead by adding our own setting by subclassing DisoverRunner and then overriding setup_test_environment method.
Piggybacking off of #Tobia's answer, I think it is better implemented in settings.py like this:
import sys
try:
TESTING = 'test' == sys.argv[1]
except IndexError:
TESTING = False
This will prevent it from catching things like ./manage.py loaddata test.json or ./manage.py i_am_not_running_a_test
I wanted to exclude some data migrations from being run in tests, and came up with this solution on a Django 3.2 project:
class Migration(migrations.Migration):
def apply(self, project_state, schema_editor, collect_sql=False):
import inspect
if 'create_test_db' in [i.function for i in inspect.stack()]:
return project_state
else:
return super().apply(project_state, schema_editor, collect_sql=collect_sql)
I haven't seen this suggested elsewhere, and for my purposes it's pretty clean. Of course, it might break if Django changes the name of the create_test_db method (or the return value of the apply method) at some point in time, but modifying this to work should be reasonably simple, since it's likely that some method exists in the stack that doesn't exist during non-test migration runs.
Is there any simple mechanism for overriding Django settings for a unit test? I have a manager on one of my models that returns a specific number of the latest objects. The number of objects it returns is defined by a NUM_LATEST setting.
This has the potential to make my tests fail if someone were to change the setting. How can I override the settings on setUp() and subsequently restore them on tearDown()? If that isn't possible, is there some way I can monkey patch the method or mock the settings?
EDIT: Here is my manager code:
class LatestManager(models.Manager):
"""
Returns a specific number of the most recent public Articles as defined by
the NEWS_LATEST_MAX setting.
"""
def get_query_set(self):
num_latest = getattr(settings, 'NEWS_NUM_LATEST', 10)
return super(LatestManager, self).get_query_set().filter(is_public=True)[:num_latest]
The manager uses settings.NEWS_LATEST_MAX to slice the queryset. The getattr() is simply used to provide a default should the setting not exist.
EDIT: This answer applies if you want to change settings for a small number of specific tests.
Since Django 1.4, there are ways to override settings during tests:
https://docs.djangoproject.com/en/stable/topics/testing/tools/#overriding-settings
TestCase will have a self.settings context manager, and there will also be an #override_settings decorator that can be applied to either a test method or a whole TestCase subclass.
These features did not exist yet in Django 1.3.
If you want to change settings for all your tests, you'll want to create a separate settings file for test, which can load and override settings from your main settings file. There are several good approaches to this in the other answers; I have seen successful variations on both hspander's and dmitrii's approaches.
You can do anything you like to the UnitTest subclass, including setting and reading instance properties:
from django.conf import settings
class MyTest(unittest.TestCase):
def setUp(self):
self.old_setting = settings.NUM_LATEST
settings.NUM_LATEST = 5 # value tested against in the TestCase
def tearDown(self):
settings.NUM_LATEST = self.old_setting
Since the django test cases run single-threaded, however, I'm curious about what else may be modifying the NUM_LATEST value? If that "something else" is triggered by your test routine, then I'm not sure any amount of monkey patching will save the test without invalidating the veracity of the tests itself.
You can pass --settings option when running tests
python manage.py test --settings=mysite.settings_local
Although overriding settings configuration on runtime might help, in my opinion you should create a separate file for testing. This saves lot of configuration for testing and this would ensure that you never end up doing something irreversible (like cleaning staging database).
Say your testing file exists in 'my_project/test_settings.py', add
settings = 'my_project.test_settings' if 'test' in sys.argv else 'my_project.settings'
in your manage.py. This will ensure that when you run python manage.py test you use test_settings only. If you are using some other testing client like pytest, you could as easily add this to pytest.ini
Update: the solution below is only needed on Django 1.3.x and earlier. For >1.4 see slinkp's answer.
If you change settings frequently in your tests and use Python ≥2.5, this is also handy:
from contextlib import contextmanager
class SettingDoesNotExist:
pass
#contextmanager
def patch_settings(**kwargs):
from django.conf import settings
old_settings = []
for key, new_value in kwargs.items():
old_value = getattr(settings, key, SettingDoesNotExist)
old_settings.append((key, old_value))
setattr(settings, key, new_value)
yield
for key, old_value in old_settings:
if old_value is SettingDoesNotExist:
delattr(settings, key)
else:
setattr(settings, key, old_value)
Then you can do:
with patch_settings(MY_SETTING='my value', OTHER_SETTING='other value'):
do_my_tests()
You can override setting even for a single test function.
from django.test import TestCase, override_settings
class SomeTestCase(TestCase):
#override_settings(SOME_SETTING="some_value")
def test_some_function():
or you can override setting for each function in class.
#override_settings(SOME_SETTING="some_value")
class SomeTestCase(TestCase):
def test_some_function():
#override_settings is great if you don't have many differences between your production and testing environment configurations.
In other case you'd better just have different settings files. In this case your project will look like this:
your_project
your_app
...
settings
__init__.py
base.py
dev.py
test.py
production.py
manage.py
So you need to have your most of your settings in base.py and then in other files you need to import all everything from there, and override some options. Here's what your test.py file will look like:
from .base import *
DEBUG = False
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'app_db_test'
}
}
PASSWORD_HASHERS = (
'django.contrib.auth.hashers.MD5PasswordHasher',
)
LOGGING = {}
And then you either need to specify --settings option as in #MicroPyramid answer, or specify DJANGO_SETTINGS_MODULE environment variable and then you can run your tests:
export DJANGO_SETTINGS_MODULE=settings.test
python manage.py test
For pytest users.
The biggest issue is:
override_settings doesn't work with pytest.
Subclassing Django's TestCase will make it work but then you can't use pytest fixtures.
The solution is to use the settings fixture documented here.
Example
def test_with_specific_settings(settings):
settings.DEBUG = False
settings.MIDDLEWARE = []
..
And in case you need to update multiple fields
def override_settings(settings, kwargs):
for k, v in kwargs.items():
setattr(settings, k, v)
new_settings = dict(
DEBUG=True,
INSTALLED_APPS=[],
)
def test_with_specific_settings(settings):
override_settings(settings, new_settings)
I created a new settings_test.py file which would import everything from settings.py file and modify whatever is different for testing purpose.
In my case I wanted to use a different cloud storage bucket when testing.
settings_test.py:
from project1.settings import *
import os
CLOUD_STORAGE_BUCKET = 'bucket_name_for_testing'
manage.py:
def main():
# use seperate settings.py for tests
if 'test' in sys.argv:
print('using settings_test.py')
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project1.settings_test')
else:
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project1.settings')
try:
from django.core.management import execute_from_command_line
except ImportError as exc:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
) from exc
execute_from_command_line(sys.argv)
Found this while trying to fix some doctests... For completeness I want to mention that if you're going to modify the settings when using doctests, you should do it before importing anything else...
>>> from django.conf import settings
>>> settings.SOME_SETTING = 20
>>> # Your other imports
>>> from django.core.paginator import Paginator
>>> # etc
I'm using pytest.
I managed to solve this the following way:
import django
import app.setting
import modules.that.use.setting
# do some stuff with default setting
setting.VALUE = "some value"
django.setup()
import importlib
importlib.reload(app.settings)
importlib.reload(modules.that.use.setting)
# do some stuff with settings new value
You can override settings in test in this way:
from django.test import TestCase, override_settings
test_settings = override_settings(
DEFAULT_FILE_STORAGE='django.core.files.storage.FileSystemStorage',
PASSWORD_HASHERS=(
'django.contrib.auth.hashers.UnsaltedMD5PasswordHasher',
)
)
#test_settings
class SomeTestCase(TestCase):
"""Your test cases in this class"""
And if you need these same settings in another file you can just directly import test_settings.
If you have multiple test files placed in a subdirectory (python package), you can override settings for all these files based on condition of presence of 'test' string in sys.argv
app
tests
__init__.py
test_forms.py
test_models.py
__init__.py:
import sys
from project import settings
if 'test' in sys.argv:
NEW_SETTINGS = {
'setting_name': value,
'another_setting_name': another_value
}
settings.__dict__.update(NEW_SETTINGS)
Not the best approach. Used it to change Celery broker from Redis to Memory.
One setting for all tests in a testCase
class TestSomthing(TestCase):
def setUp(self, **kwargs):
with self.settings(SETTING_BAR={ALLOW_FOO=True})
yield
override one setting in the testCase
from django.test import override_settings
#override_settings(SETTING_BAR={ALLOW_FOO=False})
def i_need_other_setting(self):
...
Important
Even though you are overriding these settings this will not apply to settings that your server initialize stuff with because it is already initialized, to do that you will need to start django with another setting module.