Can I specific a different configuration for the test database? Or alternatives simply use a different user in production?(how to manage that as settings file needs to be updated as well?) The testing requirements for postgresql require in my opinion too many privs like create DB and in my case I also need to create an extension upon db creation which means superuser privileges.
This is very messy, but a valid workaround nonetheless. We use a snippet of code like this in settings.py to run tests using a local sqlite3 database.
import sys
if 'test' in sys.argv or 'test_coverage' in sys.argv:
DATABASES['default']['ENGINE'] = 'django.db.backends.sqlite3'
DATABASES['default']['NAME'] = os.path.join(BASE_DIR, 'db.sqlite3')
Related
I wonder where I should initialize my MongoDB connection in my Django projects.
Currently I am initializing the client and db before my view functions in views.py:
import pymongo
from django.conf import settings
client = pymongo.MongoClient(settings.MONGO_URI)
db = client.get_default_database()
def some_view(request):
pass
However I also need to use MongoDB in my models.py in conjunction with Django signals. What do you suggest?
Maybe settings.py? Or even root __init__.py? Then you can import client and db everywhere you need it.
I've decided to use project/mongodb.py (same folder as settings.py)
import pymongo
from django.conf import settings
client = pymongo.MongoClient(settings.MONGO_URI)
mongodb = client.get_default_database()
I am using two different settings files for local and production. Therefore, this approach makes it possible to use environment dependent settings, while enabling me to access mongodb variable from anywhere in the project.
I have specified multiple databases in settings.py. One of the databases is a local Oracle database which is only available on my computer at work. So when I open my site on a computer which does not have access to this Oracle database, I see the error A server error occurred. Please contact the administrator..
So my code works as it should when I'm on my work computer, but fails when I use my page online from another computer which does not have access to the local Oracle database.
I don't want to access the local database remotely. I just want to ignore the error when I don't have access to the local Oracle database.
You can use a local.py file in your settings module that is unique for the environment you're working on. Given a settings module, you can put the following in your __init__.py:
try:
import .local
except ImportError:
pass
Then on the machine where the Oracle database is available, set the right settings in the local settings file, but don't check it in to your version control system (e.g. put it in .gitignore).
Also watch for the order in your settings, after you import your local settings, you shouldn't override them if you want to keep the changes. You can move the import down in the file, or just add the settings to the existing dict.
You can use if statements in your settings.py, since it's just a Python script. So you can add a function to detect whether you are at work, and then use that function to decide whether the at work-only database should be added to the DATABASES variable.
# in settings.py
def at_work():
'Determines whether the project runs at work'
if at_work():
DATABASES = {...}
else:
DATABASES = {...}
Of course, if your models depend on the database that is only available at work, you will need to add a mock database to enable use of the models elsewhere, even if there is no data available.
Im trying to write tests for my module. When i run:
python manage.py test my_module
Im getting message:
django.core.exceptions.ImproperlyConfigured: Please fill out the database NAME in the settings module before using the database.
I dont have any test yet, just BaseTest where im creating users, groups and assigning permissions.
Where could be the problem? Server normally works, configuration seems to be good. Do i need to define settings for test?
Ok. I think i know what was the problem :] I had lists with permissions stored in other module. So i were writting from module.perms import perms (normal python list). Its seems, that python is doing something more than just import that list from other module to my module. And that was the cause of failing.
Solution:
Surround code after list definition with:
if __name__ == "__main__":
...
Then everything should be working good.
Ok. I think i know what was the problem :] I had lists with permissions stored in other module. So i were writting from module.perms import perms (normal python list). Its seems, that python is doing something more than just import that list from other module to my module. And that was the cause of failing.
Solution: Surround code after list definition with:
if __name__ == "__main__":
...
Then everything should be working good.
You need to have your DATABASE setting in place in settings.py, and your app needs to be in installed apps. For example:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'database.sqlite',
python manage.py test doesn't use the NAME database defined in settings.py DATABASES. From the docs -
Tests that require a database (namely, model tests) will not use your
"real" (production) database. Separate, blank databases are created
for the tests.
Regardless of whether the tests pass or fail, the test databases are
destroyed when all the tests have been executed.
By default the test databases get their names by prepending test_ to
the value of the NAME settings for the databases defined in DATABASES.
When using the SQLite database engine the tests will by default use an
in-memory database (i.e., the database will be created in memory,
bypassing the filesystem entirely!). If you want to use a different
database name, specify TEST_NAME in the dictionary for any given
database in DATABASES.
To answer your question. You can define a test database with the TEST_NAME setting. But you shouldn't have to.
Here's another potential solution for people finding this through search:
I was trying to define my own TestRunner class inside my settings file, which meant it had to import DjangoTestSuiteRunner. This import at the top of the file caused the ImproperlyConfigured error. Moving the custom TestRunner class to a separate file resolved the issue.
I have a model with an ImageField that is backed by django-storages' S3Boto. I have a test the exercises the "upload image" view, but the fact that it is uploading the image to S3 is slowing down my test suite.
In the interest of speeding up my tests, what is the best practice for dealing with this issue? Should I mock out S3Boto? Perhaps there is a memory backed storage backend that works well for testing (automatic cleanup would be nice!)?
I just had this problem too. I got much faster tests by using dj-inmemorystorage.
The quick way of setting this up is by creating a test_settings.py in the same folder as your settings:
from settings import *
DEFAULT_FILE_STORAGE = 'inmemorystorage.InMemoryStorage'
...and calling ./manage.py test --settings=project.test_settings to run the tests.
My preferred way is to set up a custom test runner:
In project/test_runner.py:
from django.conf import settings
from django.test.runner import DiscoverRunner
class FastTestRunner(DiscoverRunner):
def setup_test_environment(self):
super(FastTestRunner, self).setup_test_environment()
# Don't write files
settings.DEFAULT_FILE_STORAGE = 'inmemorystorage.InMemoryStorage'
# Bonus: Use a faster password hasher. This REALLY helps.
settings.PASSWORD_HASHERS = (
'django.contrib.auth.hashers.MD5PasswordHasher',
)
Note: This also sets the PASSWORD_HASHER, because it significantly improves User creation time. This should NOT be set in production.
In project/settings.py:
TEST_RUNNER = 'project.test_runner.FastTestRunner'
The requirements:
pip install dj-inmemorystorage
UPDATE: changed from django-inmemorystorage to dj-inmemorystorage.
UPDATE 2: Removed django-discover-runner, as it's now the default test runner in django, and fixed the link to the PASSWORD_HASHER related blog post.
I also use S3Boto but for testing, I prefer having custom settings which include using the file system storage. You can have your custom settings declared in a file which you can then import and use in your test cases. Even so, you can mock the file storage so that the files are not actually written to disk.
Here's a sample test_settings.py
# myproject/myproject/test_settings.py
from django.test import override_settings
common_settings = override_settings(
DEFAULT_FILE_STORAGE='django.core.files.storage.FileSystemStorage',
PASSWORD_HASHERS=(
'django.contrib.auth.hashers.UnsaltedMD5PasswordHasher',
),
)
Usage:
from django.test import TestCase
from myproject.test_settings import common_settings
#common_settings
class MyTestCase(TestCase):
"""Tests go here"""
On mocking the file system storage you can check out my answer here on SO.
Just ran into this as well so I thought I'd put my solution up. My solution uses Mock
import mock
from django.core.files.storage import FileSystemStorage
from django.test import TestCase
class ATestCase(TestCase):
def setUp(self):
# Stuff Happens
def tearDown(self):
# more Stuff
#mock.patch('storages.backends.s3boto.S3BotoStorage', FileSystemStorage)
def test_file_stuff(self):
self.assertMagicPonies(True)
Some gotchas - make sure you have a sane MEDIA_ROOT setup in the settings. as of django 1.4, you can't use the testing context manager to override MEDIA_ROOT, so you need a separate settings config for it (https://code.djangoproject.com/ticket/17787) This was fixed in 1.6. Also, make sure your upload_to works in normal filesystem, or you will get permission errors.
I would propose to use the standard Django Storage for testing, where you can a define custom path for storage and cleanup that path in your test suite once your done. Both the storage and the path can be set in the settings and overwritten for testing.
Django settings includes a list of python variables that are used for a plethora of things from database settings to installed apps. Even many of the reusable apps make some of the settings required.
With a dozens of sites, it is hard to manage the settings of all the projects.
Fortunately settings is just a python module with variables, so you can do any magic to populate the variables you want.
What practices have you followed or you think can be used to separate various related settings into different files?
Apparently, the existing enterprisey practice is that a developer creates a war and the ops department slaps it to the bluefish and takes care of all the database(and such) ops stuff (according to Jacob's email).
What dynamic settings.py can you create that will aid the existing enterprise practices?
Often I've seen settings files with something like:
from localsettings import *
and in localsettings.py things like database connections and DEBUG values are defined. localsettings.py is (or may be) different for each deployment environment (dev/staging/production etc), and doesn't live in source control with everything else.
Something I've found helpful lately is putting this in my settings.py:
try:
from localsettings import *
except ImportError:
from default_localsettings import *
in default_localsettings.py I define a bunch of defaults (DEBUG = True, use a sqlite database in the same directory as default_localsettings.py etc).
This might not be useful once you've got things set up, but I've found it useful just to be able to check my project out of source control and have it work straightaway using runserver, without having to set anything up.
Follow this settings override example to handle dev, staging, and production environments.
http://djangodose.com/articles/2009/09/handling-development-staging-and-production-enviro/
(archived version at Wayback Machine)
Personally, I like it to make a module out of settings like
project/
settings/
__init__.py
settings.py
local.py
The file __init__.py looks like:
from .settings import *
from .local import *
You have adopt the variable BASE_DIR to point to the directory one level higher.
Using this procedure, you don't have to adopt anything else like the import mechanism. Just use from django.conf import settings. All settings in local.py overwrite the ones in settings.py.
However, this possible solution may be combined with the other answers given above.