I use Django ORM inside async code. Everything works fine and all tests pass. However, DB connections do not close properly after tests. Here is an example:
from asgiref.sync import sync_to_async, async_to_sync
#sync_to_async
def count_books():
return Book.objects.count()
class FooTest(TestCase):
def setUp(self):
Book.objects.create(title='Haha')
def test1(self):
import asyncio
c = asyncio.run(count_books())
self.assertEqual(1, c)
def test2(self):
c = async_to_sync(count_books)()
self.assertEqual(1, c)
Postgres error:
django.db.utils.OperationalError: database "test_mydbname" is being accessed by other users
Sqlite error:
sqlite3.OperationalError: database table is locked: test_mydbname
I've tried swapping sync_to_async with database_sync_to_async from django-channels, but this didn't change anything.
How can I fix this?
The issue comes with how your async runloops are interacting with the main thread, handling this yourself can get quite complex.
For testing django-channels I suggest using pytest with pytest-asyncio for testing channels. And of course pytest-django.
This will provide a few useful tools for testing async code.
#pytest.mark.django_db(transaction=True)
#pytest.mark.asyncio
async def test1():
count = await database_sync_to_async(Book.objects.count)
....
for some examples of how to test channels code take a look here.
Using an edited version of the example from Django's own doc, lets say my code looks like this:
from django.test import TestCase
from myapp.models import Animal
class AnimalTestCase(TestCase):
def __init__(self, animal_family):
self.animal_family = animal_family
def setUp(self):
Animal.objects.create(name="lion", sound="roar", family=self.animal_family)
def test_animals_can_speak(self):
"""Animals that can speak are correctly identified"""
lion = Animal.objects.get(name="lion")
self.assertEqual(lion.speak(), 'The mammal lion says "roar"')
Basically, I want to pass in the animal_family parameter into my test, or something similar at least from my terminal. Something like this
python manage.py test myapp.tests.AnimalTestCase 'mammal'
Obviously, the above command does not work. I want to send the 'mammal' as the animal_family to the __init__ method in my test class.
Help would be greatly appreciated.
Whilst self-contained tests should be the best practice, if you really wanted to do this you could set an environment variable when you execute the test command.
For example:
import os
from django.test import TestCase
from myapp.models import Animal
class AnimalTestCase(TestCase):
def setUp(self):
# Retrieve the animal family from the environment variable
animal_family = os.environ['animal_family']
Animal.objects.create(name="lion", sound="roar", family=animal_family)
def test_animals_can_speak(self):
"""Animals that can speak are correctly identified"""
lion = Animal.objects.get(name="lion")
self.assertEqual(lion.speak(), 'The mammal lion says "roar"')
And then call the test command such as:
export animal_family=mammal;python manage.py test myapp.tests.AnimalTestCase
I am trying to mock out redis in my Django application. I have tried several different methods but none seem to work. What am I doing wrong?
My primary redis instance is called with:
redis_client = redis.from_url(os.environ.get("REDIS_URL"))
That instance is imported in other parts of the app in order to add and retrieve data.
In my tests I tried doing:
import fakeredis
from mock import patch
class TestViews(TestCase):
def setUp(self):
redis_patcher = patch('redis.Redis', fakeredis.FakeRedis)
self.redis = redis_patcher.start()
self.redis.set('UPDATE', 'Spring')
print(redis_client.get('UPDATE'))
def tearDown(self):
self.redis_patcher.stop
When running the tests I want the 'UPDATE' variable to be set. But instead every instance of redis_client fails saying the server is not available. How can I mock out redis and set values, so that they are available when testing my app?
You should mock an item where it is used, not where it came from.
So if redis_client is used in a view like this:
myapp/views.py
from somemodule import redis_client
def some_view_that_uses_redis(request):
result = redis_client(...)
Then in your TestViews you should patch redis_client like this:
class TestViews(TestCase):
def setUp(self):
redis_patcher = patch('myapp.views.redis_client', fakeredis.FakeRedis)
self.redis = redis_patcher.start()
I am having a problem where my functional test is not isolating from the main database, there is data left over from the test. I ran python manage.py flush and the test works, but when I run the test again the data created by the test that should have been deleted is still there making my test fail. I am currently only using SQLite.
functional_test.py
from django.test import LiveServerTestCase
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
class NewUserTest(LiveServerTestCase):
def setUp(self):
self.browser = webdriver.Firefox()
self.browser.implicitly_wait(15)
def tearDown(self):
self.browser.quit()
def test_user(self):
#Some test
You forgot to indent your method definitions (the statements that begin with "def"). As a result, the Python interpreter thinks that you've defined a new class, called NewUserTest, that contains no attributes, and three top-level functions.
To solve the problem, simply indent the "def" statements by an appropriate amount (4 spaces is the standard convention amongst the Python community). It should look like this:
class NewUserTest(LiveServerTestCase):
def setUp(self):
self.browser = webdriver.Firefox()
self.browser.implicitly_wait(15)
def tearDown(self):
self.browser.quit()
...
I'm working on a Django project and am writing unittests for it. However, in a test, when I try and log a user in, I get this error:
MessageFailure: You cannot add messages without installing django.contrib.messages.middleware.MessageMiddleware
Logging in on the actual site works fine -- and a login message is displayed using the MessageMiddleware.
In my tests, if I do this:
from django.conf import settings
print settings.MIDDLEWARE_CLASSES
Then it outputs this:
('django.middleware.cache.UpdateCacheMiddleware',
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.cache.FetchFromCacheMiddleware',
'debug_toolbar.middleware.DebugToolbarMiddleware')
Which appears to show the MessageMiddleware is installed when tests are run.
Is there an obvious step I'm missing?
UPDATE
After suggestions below, it does look like it's a settings thing.
I currently have settings/__init__.py like this:
try:
from settings.development import *
except ImportError:
pass
and settings/defaults.py containing most of the standard settings (including MIDDLEWARE_CLASSES). And then settings.development.py overrides some of those defaults like this:
from defaults import *
DEBUG = True
# etc
It looks like my dev site itself works fine, using the development settings. But although the tests seem to load the settings OK (both defaults and development) settings.DEBUG is set to False. I don't know why, or whether that's the cause of the problem.
Django 1.4 has a expected behavior when you create the request with RequestFactory that can trigger this error.
To resolve this issue, create your request with RequestFactory and do this:
from django.contrib.messages.storage.fallback import FallbackStorage
setattr(request, 'session', 'session')
messages = FallbackStorage(request)
setattr(request, '_messages', messages)
Works for me!
A way to solve this quite elegant is to mock the messages module using mock
Say you have a class based view named FooView in app named myapp
from django.contrib import messages
from django.views.generic import TemplateView
class FooView(TemplateView):
def post(self, request, *args, **kwargs):
...
messages.add_message(request, messages.SUCCESS, '\o/ Profit \o/')
...
You now can test it with
def test_successful_post(self):
mock_messages = patch('myapp.views.FooView.messages').start()
mock_messages.SUCCESS = success = 'super duper'
request = self.rf.post('/', {})
view = FooView.as_view()
response = view(request)
msg = _(u'\o/ Profit \o/')
mock_messages.add_message.assert_called_with(request, success, msg)
In my case (django 1.8) this problem occurs in when unit-test calls signal handler for user_logged_in signal, looks like messages app has not been called, i.e. request._messages is not yet set. This fails:
from django.contrib.auth.signals import user_logged_in
...
#receiver(user_logged_in)
def user_logged_in_handler(sender, user, request, **kwargs):
...
messages.warning(request, "user has logged in")
the same call to messages.warning in normal view function (that is called after) works without any issues.
A workaround I based on one of the suggestions from https://code.djangoproject.com/ticket/17971, use fail_silently argument only in signal handler function, i.e. this solved my problem:
messages.warning(request, "user has logged in",
fail_silently=True )
Do you only have one settings.py?
Tests create custom (tests) database. Maybe you have no messages there or something... Maybe you need setUp() fixtures or something?
Need more info to answer properly.
Why not simply do something like ? You sure run tests in debug mode right?
# settings.py
DEBUG = True
from django.conf import settings
# where message is sent:
if not settings.DEBUG:
# send your message ...
This builds on Tarsis Azevedo's answer by creating a MessagingRequest helper class below.
Given say a KittenAdmin I'd want to get 100% test coverage for:
from django.contrib import admin, messages
class KittenAdmin(admin.ModelAdmin):
def warm_fuzzy_method(self, request):
messages.warning(request, 'Can I haz cheezburger?')
I created a MessagingRequest helper class to use in say a test_helpers.py file:
from django.contrib.messages.storage.fallback import FallbackStorage
from django.http import HttpRequest
class MessagingRequest(HttpRequest):
session = 'session'
def __init__(self):
super(MessagingRequest, self).__init__()
self._messages = FallbackStorage(self)
def get_messages(self):
return getattr(self._messages, '_queued_messages')
def get_message_strings(self):
return [str(m) for m in self.get_messages()]
Then in a standard Django tests.py:
from django.contrib.admin.sites import AdminSite
from django.test import TestCase
from cats.kitten.admin import KittenAdmin
from cats.kitten.models import Kitten
from cats.kitten.test_helpers import MessagingRequest
class KittenAdminTest(TestCase):
def test_kitten_admin_message(self):
admin = KittenAdmin(model=Kitten, admin_site=AdminSite())
expect = ['Can I haz cheezburger?']
request = MessagingRequest()
admin.warm_fuzzy_method(request)
self.assertEqual(request.get_message_strings(), expect)
Results:
coverage run --include='cats/kitten/*' manage.py test; coverage report -m
Creating test database for alias 'default'...
.
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
Destroying test database for alias 'default'...
Name Stmts Miss Cover Missing
----------------------------------------------------------------------
cats/kitten/__init__.py 0 0 100%
cats/kitten/admin.py 4 0 100%
cats/kitten/migrations/0001_initial.py 5 0 100%
cats/kitten/migrations/__init__.py 0 0 100%
cats/kitten/models.py 3 0 100%
cats/kitten/test_helpers.py 11 0 100%
cats/kitten/tests.py 12 0 100%
----------------------------------------------------------------------
TOTAL 35 0 100%
This happened to me in the login_callback signal receiver function when called from a unit test, and the way around the problem was:
from django.contrib.messages.storage import default_storage
#receiver(user_logged_in)
def login_callback(sender, user, request, **kwargs):
if not hasattr(request, '_messages'): # fails for tests
request._messages = default_storage(request)
Django 2.0.x
I found when I had a problem patching messages the solution was to patch the module from within the class under test (obsolete Django version BTW, YMMV). Pseudocode follows.
my_module.py:
from django.contrib import messages
class MyClass:
def help(self):
messages.add_message(self.request, messages.ERROR, "Foobar!")
test_my_module.py:
from unittest import patch, MagicMock
from my_module import MyClass
class TestMyClass(TestCase):
def test_help(self):
with patch("my_module.messages") as mock_messages:
mock_messages.add_message = MagicMock()
MyClass().help() # shouldn't complain about middleware
If you're seeing a problem in your Middleware, then you're not doing "Unit Test". Unit tests test a unit of functionality. If you interact with other parts of your system, you're making something called "integration" testing.
You should try to write better tests, and this kind of problems shouldn't arise. Try RequestFactory. ;)
def test_some_view(self):
factory = RequestFactory()
user = get_mock_user()
request = factory.get("/my/view")
request.user = user
response = my_view(request)
self.asssertEqual(status_code, 200)