I am writing a Flask application that uses SQLAlchemy for its database backend.
The Flask application is created with an app factory called create_app.
from flask import Flask
def create_app(config_filename = None):
app = Flask(__name__)
if config_filename is None:
app.config.from_pyfile('config.py', silent=True)
else:
app.config.from_mapping(config_filename)
from .model import db
db.init_app(app)
db.create_all(app=app)
return app
The database model consists of a single object called Document.
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class Document(db.Model):
id = db.Column(db.Integer, primary_key=True)
document_uri = db.Column(db.String, nullable=False, unique=True)
I am using pytest to do unit testing. I create a pytest fixture called app_with_documents that calls the application factory to create an application and adds some Document objects to the database before the test is run, then empties out the database after the unit test has completed.
import pytest
from model import Document, db
from myapplication import create_app
#pytest.fixture
def app():
config = {
'SQLALCHEMY_DATABASE_URI': f"sqlite:///:memory:",
'TESTING': True,
'SQLALCHEMY_TRACK_MODIFICATIONS': False
}
app = create_app(config)
yield app
with app.app_context():
db.drop_all()
#pytest.fixture
def app_with_documents(app):
with app.app_context():
document_1 = Document(document_uri='Document 1')
document_2 = Document(document_uri='Document 2')
document_3 = Document(document_uri='Document 3')
document_4 = Document(document_uri='Document 4')
db.session.add_all([document_1, document_2, document_3, document_4])
db.session.commit()
return app
I have multiple unit tests that use this fixture.
def test_unit_test_1(app_with_documents):
...
def test_unit_test_2(app_with_documents):
...
If I run a single unit test everything works. If I run more than one test, subsequent unit tests crash at the db.session.commit() line in the test fixture setup with "no such table: document".
def do_execute(self, cursor, statement, parameters, context=None):
> cursor.execute(statement, parameters)
E sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: document [SQL: 'INSERT INTO document (document_uri) VALUES (?)'] [parameters: ('Document 1',)] (Background on this error at: http://sqlalche.me/e/e3q8)
What I expect is that each unit test gets its own brand-new identical prepopulated database so that all the tests would succeed.
(This is an issue with the database tables, not the unit tests. I see the bug even if my unit tests consist of just pass.)
The fact that the error message mentions a missing table makes it look like the db.create_all(app=app) in create_app is not being called after the first unit test runs. However, I have verified in the debugger that this application factory function is called once for every unit test as expected.
It is possible that my call to db.drop_all() is an incorrect way to clear out the database. So instead of an in-memory database, I tried creating one on disk and then deleting it as part of the test fixture cleanup. (This is the technique recommended in the Flask documentation.)
#pytest.fixture
def app():
db_fd, db_filename = tempfile.mkstemp(suffix='.sqlite')
config = {
'SQLALCHEMY_DATABASE_URI': f"sqlite:///{db_filename}",
'TESTING': True,
'SQLALCHEMY_TRACK_MODIFICATIONS': False
}
yield create_app(config)
os.close(db_fd)
os.unlink(db_filename)
This produces the same error.
Is this a bug in Flask and/or SQLAlchemy?
What is the correct way to write Flask test fixtures that prepopulate an application's database?
This is Flask 1.0.2, Flask-SQLAlchemy 2.3.2, and pytest 3.6.0, which are all the current latest versions.
In my conftest.py I was importing the contents of model.py in my application like so.
from model import Document, db
I was running the unit tests in Pycharm using Pycharm's pytest runner. If instead I run tests from the command line with python -m pytest I see the following error
ModuleNotFoundError: No module named 'model'
ERROR: could not load /Users/wmcneill/src/FlaskRestPlus/test/conftest.py
I can get my tests running from the command line by fully-qualifying the import path in conftest.py.
from myapplication.model import Document, db
When I do this all the unit tests pass. They also pass when I run the unit tests from inside Pycharm.
So it appears that I had incorrectly written an import statement in my unit tests. However, when I ran those unit tests via Pycharm, instead of seeing an error message about the import, the scripts launched but then had weird SQL errors.
I still haven't figured out why I saw the strange SQL errors I did. Presumably something subtle about the way global state is being handled. But changing the import line fixes my problem.
Using the shell, I can do this:
>>> from django.test.client import Client
>>> from django.test.utils import setup_test_environment
>>> setup_test_environment()
>>> c = Client()
>>> c.login(username="dev", password="password")
True
>>> r = c.get('/')
>>> r.status_code
200
Now with this in the test.py file:
from django.test.client import Client
__test__ = {"doctest": """
>>> c = Client()
>>> c.login(username='dev', password='password')
True
>>> r = c.get('/')
>>> r.status_code
200
"""}
I get this output:
Failed example:
c.login(username="dev", password="password")
Expected:
True
Got:
False
------------------------------------------------------
Failed example:
r.status_code
Expected:
200
Got:
302
I've looked all over the internet and I can't find anything that helps with this situation. Any ideas?
On a similar note, I've commented out: from django.views.decorators.debug import sensitive_post_parameters and all #sensitive_post_parameters() decorators from my code because each time I run ./manage.py test app django complains:
Could not import app.views. Error was: No module named debug
Removing this decorator and import statement allows it to move forward.
Im very much lost and I need StackOverflow! Thanks everyone.
sensitive_post_parameters is a new feature in Django 1.4, so if you're running Django 1.3 or earlier then the import will fail.
I believe that the commands you tried in the shell were run on the normal database. When you run your doc tests, Django will set up a test database. It looks like your user dev isn't in the test database when you run the doc tests, so the login attempt fails. One option is to create the User with User.objects.create_user before you attempt the login. Another option is to use fixtures.
With Django, I would recommend writing unit tests instead of doc tests. One big advantage is that it's easy to include fixtures to load initial data (e.g. users) into the test database. Another is that Django takes care of refreshing the database between unit tests.
When I run tests on my remote server using fabric, I get an error saying:
File "/usr/local/lib/python2.7/dist-packages/django/template/loader.py", line 138, in find_template
raise TemplateDoesNotExist(name)
TemplateDoesNotExist: index.html
I am trying to render the template as a string using "render_to_string()"
If I login to the server and run tests manually (python manage.py test app), it is working properly. This error occurs while running through fabric.
Here is my fabric code:
from __future__ import with_statement
from fabric.api import local
import os
from fabric.api import *
env.hosts = ['server.com']
production_project_path = '/path/to/production/app/'
def run_remote_test():
run("python %s/manage.py test app"%production_project_path)
Did I miss something?
Note: I am not using virtual environment
Then let's make this official. ;)
In this case, the problem was the fact that manage.py expects to be ran from the project directory, so rewriting the abovestanding as:
from __future__ import with_statement
from fabric.api import local
import os
from fabric.api import *
env.hosts = ['server.com']
production_project_path = '/path/to/production/app/'
def run_remote_test():
with cd(production_project_path):
run("python manage.py test app")
has fixed the issue.
Im writing some code which retrieves info about installed apps, especially defined models, and then does stuff based on that information, but Im having some problems writing a clean, nice unittest. Is there a way to emulate or add an app in unittests without have to run manage.py startproject, manage.py startapp in my testsfolder to have a test app available for unittests?
Sure, try this on for size:
from django.conf import settings
from django.core.management import call_command
from django.test.testcases import TestCase
from django.db.models import loading
class AppTestCase(TestCase):
'''
Adds apps specified in `self.apps` to `INSTALLED_APPS` and
performs a `syncdb` at runtime.
'''
apps = ()
_source_installed_apps = ()
def _pre_setup(self):
super(AppTestCase, self)._pre_setup()
if self.apps:
self._source_installed_apps = settings.INSTALLED_APPS
settings.INSTALLED_APPS = settings.INSTALLED_APPS + self.apps
loading.cache.loaded = False
call_command('syncdb', verbosity=0)
def _post_teardown(self):
super(AppTestCase, self)._post_teardown()
if self._source_installed_apps:
settings.INSTALLED_APPS = self._source_installed_apps
self._source_installed_apps = ()
loading.cache.loaded = False
Your test case would look something like this:
class SomeAppTestCase(AppTestCase):
apps = ('someapp',)
In case you were wondering why, I did an override of _pre_setup() and _post_teardown() so I don't have to bother with calling super() in setUp() and tearDown() in my final test case. Otherwise, this is what I pulled out of Django's test runner. I whipped it up and it worked, although I'm sure that, with closer inspection, you can further optimize it and even avoid calling syncdb every time if it won't conflict with future tests.
EDIT:
So I seem to have gotten out of my way, thinking you need to dynamically add new models. If you've created an app for testing purposes only, here's what you can do to have it discovered during your tests.
In your project directory, create a test.py file that will contain your test settings. It should look something like this:
from settings import *
# registers test app for discovery
INSTALLED_APPS += ('path.to.test.app',)
You can now run your tests with python manage.py test --settings=myproject.test and your app will be in the installed apps.
Is there any simple mechanism for overriding Django settings for a unit test? I have a manager on one of my models that returns a specific number of the latest objects. The number of objects it returns is defined by a NUM_LATEST setting.
This has the potential to make my tests fail if someone were to change the setting. How can I override the settings on setUp() and subsequently restore them on tearDown()? If that isn't possible, is there some way I can monkey patch the method or mock the settings?
EDIT: Here is my manager code:
class LatestManager(models.Manager):
"""
Returns a specific number of the most recent public Articles as defined by
the NEWS_LATEST_MAX setting.
"""
def get_query_set(self):
num_latest = getattr(settings, 'NEWS_NUM_LATEST', 10)
return super(LatestManager, self).get_query_set().filter(is_public=True)[:num_latest]
The manager uses settings.NEWS_LATEST_MAX to slice the queryset. The getattr() is simply used to provide a default should the setting not exist.
EDIT: This answer applies if you want to change settings for a small number of specific tests.
Since Django 1.4, there are ways to override settings during tests:
https://docs.djangoproject.com/en/stable/topics/testing/tools/#overriding-settings
TestCase will have a self.settings context manager, and there will also be an #override_settings decorator that can be applied to either a test method or a whole TestCase subclass.
These features did not exist yet in Django 1.3.
If you want to change settings for all your tests, you'll want to create a separate settings file for test, which can load and override settings from your main settings file. There are several good approaches to this in the other answers; I have seen successful variations on both hspander's and dmitrii's approaches.
You can do anything you like to the UnitTest subclass, including setting and reading instance properties:
from django.conf import settings
class MyTest(unittest.TestCase):
def setUp(self):
self.old_setting = settings.NUM_LATEST
settings.NUM_LATEST = 5 # value tested against in the TestCase
def tearDown(self):
settings.NUM_LATEST = self.old_setting
Since the django test cases run single-threaded, however, I'm curious about what else may be modifying the NUM_LATEST value? If that "something else" is triggered by your test routine, then I'm not sure any amount of monkey patching will save the test without invalidating the veracity of the tests itself.
You can pass --settings option when running tests
python manage.py test --settings=mysite.settings_local
Although overriding settings configuration on runtime might help, in my opinion you should create a separate file for testing. This saves lot of configuration for testing and this would ensure that you never end up doing something irreversible (like cleaning staging database).
Say your testing file exists in 'my_project/test_settings.py', add
settings = 'my_project.test_settings' if 'test' in sys.argv else 'my_project.settings'
in your manage.py. This will ensure that when you run python manage.py test you use test_settings only. If you are using some other testing client like pytest, you could as easily add this to pytest.ini
Update: the solution below is only needed on Django 1.3.x and earlier. For >1.4 see slinkp's answer.
If you change settings frequently in your tests and use Python ≥2.5, this is also handy:
from contextlib import contextmanager
class SettingDoesNotExist:
pass
#contextmanager
def patch_settings(**kwargs):
from django.conf import settings
old_settings = []
for key, new_value in kwargs.items():
old_value = getattr(settings, key, SettingDoesNotExist)
old_settings.append((key, old_value))
setattr(settings, key, new_value)
yield
for key, old_value in old_settings:
if old_value is SettingDoesNotExist:
delattr(settings, key)
else:
setattr(settings, key, old_value)
Then you can do:
with patch_settings(MY_SETTING='my value', OTHER_SETTING='other value'):
do_my_tests()
You can override setting even for a single test function.
from django.test import TestCase, override_settings
class SomeTestCase(TestCase):
#override_settings(SOME_SETTING="some_value")
def test_some_function():
or you can override setting for each function in class.
#override_settings(SOME_SETTING="some_value")
class SomeTestCase(TestCase):
def test_some_function():
#override_settings is great if you don't have many differences between your production and testing environment configurations.
In other case you'd better just have different settings files. In this case your project will look like this:
your_project
your_app
...
settings
__init__.py
base.py
dev.py
test.py
production.py
manage.py
So you need to have your most of your settings in base.py and then in other files you need to import all everything from there, and override some options. Here's what your test.py file will look like:
from .base import *
DEBUG = False
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'app_db_test'
}
}
PASSWORD_HASHERS = (
'django.contrib.auth.hashers.MD5PasswordHasher',
)
LOGGING = {}
And then you either need to specify --settings option as in #MicroPyramid answer, or specify DJANGO_SETTINGS_MODULE environment variable and then you can run your tests:
export DJANGO_SETTINGS_MODULE=settings.test
python manage.py test
For pytest users.
The biggest issue is:
override_settings doesn't work with pytest.
Subclassing Django's TestCase will make it work but then you can't use pytest fixtures.
The solution is to use the settings fixture documented here.
Example
def test_with_specific_settings(settings):
settings.DEBUG = False
settings.MIDDLEWARE = []
..
And in case you need to update multiple fields
def override_settings(settings, kwargs):
for k, v in kwargs.items():
setattr(settings, k, v)
new_settings = dict(
DEBUG=True,
INSTALLED_APPS=[],
)
def test_with_specific_settings(settings):
override_settings(settings, new_settings)
I created a new settings_test.py file which would import everything from settings.py file and modify whatever is different for testing purpose.
In my case I wanted to use a different cloud storage bucket when testing.
settings_test.py:
from project1.settings import *
import os
CLOUD_STORAGE_BUCKET = 'bucket_name_for_testing'
manage.py:
def main():
# use seperate settings.py for tests
if 'test' in sys.argv:
print('using settings_test.py')
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project1.settings_test')
else:
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project1.settings')
try:
from django.core.management import execute_from_command_line
except ImportError as exc:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
) from exc
execute_from_command_line(sys.argv)
Found this while trying to fix some doctests... For completeness I want to mention that if you're going to modify the settings when using doctests, you should do it before importing anything else...
>>> from django.conf import settings
>>> settings.SOME_SETTING = 20
>>> # Your other imports
>>> from django.core.paginator import Paginator
>>> # etc
I'm using pytest.
I managed to solve this the following way:
import django
import app.setting
import modules.that.use.setting
# do some stuff with default setting
setting.VALUE = "some value"
django.setup()
import importlib
importlib.reload(app.settings)
importlib.reload(modules.that.use.setting)
# do some stuff with settings new value
You can override settings in test in this way:
from django.test import TestCase, override_settings
test_settings = override_settings(
DEFAULT_FILE_STORAGE='django.core.files.storage.FileSystemStorage',
PASSWORD_HASHERS=(
'django.contrib.auth.hashers.UnsaltedMD5PasswordHasher',
)
)
#test_settings
class SomeTestCase(TestCase):
"""Your test cases in this class"""
And if you need these same settings in another file you can just directly import test_settings.
If you have multiple test files placed in a subdirectory (python package), you can override settings for all these files based on condition of presence of 'test' string in sys.argv
app
tests
__init__.py
test_forms.py
test_models.py
__init__.py:
import sys
from project import settings
if 'test' in sys.argv:
NEW_SETTINGS = {
'setting_name': value,
'another_setting_name': another_value
}
settings.__dict__.update(NEW_SETTINGS)
Not the best approach. Used it to change Celery broker from Redis to Memory.
One setting for all tests in a testCase
class TestSomthing(TestCase):
def setUp(self, **kwargs):
with self.settings(SETTING_BAR={ALLOW_FOO=True})
yield
override one setting in the testCase
from django.test import override_settings
#override_settings(SETTING_BAR={ALLOW_FOO=False})
def i_need_other_setting(self):
...
Important
Even though you are overriding these settings this will not apply to settings that your server initialize stuff with because it is already initialized, to do that you will need to start django with another setting module.