Mocking Django Storages Model ImageField backend S3 - django

Question

I have a model with an ImageField that is backed by django-storages' S3Boto. I have a test the exercises the "upload image" view, but the fact that it is uploading the image to S3 is slowing down my test suite.
In the interest of speeding up my tests, what is the best practice for dealing with this issue? Should I mock out S3Boto? Perhaps there is a memory backed storage backend that works well for testing (automatic cleanup would be nice!)?

Answer 1

I just had this problem too. I got much faster tests by using dj-inmemorystorage.
The quick way of setting this up is by creating a test_settings.py in the same folder as your settings:
from settings import *
DEFAULT_FILE_STORAGE = 'inmemorystorage.InMemoryStorage'
...and calling ./manage.py test --settings=project.test_settings to run the tests.
My preferred way is to set up a custom test runner:
In project/test_runner.py:
from django.conf import settings
from django.test.runner import DiscoverRunner
class FastTestRunner(DiscoverRunner):
def setup_test_environment(self):
super(FastTestRunner, self).setup_test_environment()
# Don't write files
settings.DEFAULT_FILE_STORAGE = 'inmemorystorage.InMemoryStorage'
# Bonus: Use a faster password hasher. This REALLY helps.
settings.PASSWORD_HASHERS = (
'django.contrib.auth.hashers.MD5PasswordHasher',
)
Note: This also sets the PASSWORD_HASHER, because it significantly improves User creation time. This should NOT be set in production.
In project/settings.py:
TEST_RUNNER = 'project.test_runner.FastTestRunner'
The requirements:
pip install dj-inmemorystorage
UPDATE: changed from django-inmemorystorage to dj-inmemorystorage.
UPDATE 2: Removed django-discover-runner, as it's now the default test runner in django, and fixed the link to the PASSWORD_HASHER related blog post.

Answer 2

I also use S3Boto but for testing, I prefer having custom settings which include using the file system storage. You can have your custom settings declared in a file which you can then import and use in your test cases. Even so, you can mock the file storage so that the files are not actually written to disk.
Here's a sample test_settings.py
# myproject/myproject/test_settings.py
from django.test import override_settings
common_settings = override_settings(
DEFAULT_FILE_STORAGE='django.core.files.storage.FileSystemStorage',
PASSWORD_HASHERS=(
'django.contrib.auth.hashers.UnsaltedMD5PasswordHasher',
),
)
Usage:
from django.test import TestCase
from myproject.test_settings import common_settings
#common_settings
class MyTestCase(TestCase):
"""Tests go here"""
On mocking the file system storage you can check out my answer here on SO.

Answer 3

Just ran into this as well so I thought I'd put my solution up. My solution uses Mock
import mock
from django.core.files.storage import FileSystemStorage
from django.test import TestCase
class ATestCase(TestCase):
def setUp(self):
# Stuff Happens
def tearDown(self):
# more Stuff
#mock.patch('storages.backends.s3boto.S3BotoStorage', FileSystemStorage)
def test_file_stuff(self):
self.assertMagicPonies(True)
Some gotchas - make sure you have a sane MEDIA_ROOT setup in the settings. as of django 1.4, you can't use the testing context manager to override MEDIA_ROOT, so you need a separate settings config for it (https://code.djangoproject.com/ticket/17787) This was fixed in 1.6. Also, make sure your upload_to works in normal filesystem, or you will get permission errors.

Answer 4

I would propose to use the standard Django Storage for testing, where you can a define custom path for storage and cleanup that path in your test suite once your done. Both the storage and the path can be set in the settings and overwritten for testing.