Different db for testing in Django? - django

DATABASES = {
# 'default': {
# 'ENGINE': 'postgresql_psycopg2',
# ...
# }
# for unit tests
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'mydatabase'
}
}
I have two databases: one I'd like to use for unit tests, and one for everything else. Is it possible to configure this in Django 1.2.4?
(The reason I ask is because with postgresql I'm getting the following error:
foo#bar:~/path/$ python manage.py test
Creating test database 'default'...
Got an error creating the test database: permission denied to create database
Type 'yes' if you would like to try deleting the test database 'test_baz', or 'no' to cancel: yes
Destroying old test database...
Got an error recreating the test database: database "test_baz" does not exist
Why could I be getting this error? I guess I don't really care if I can always use SQLite for unit tests, as that works fine.)

In your settings.py (or local_settings.py):
import sys
if 'test' in sys.argv:
DATABASES['default'] = {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'mydatabase'
}

The way I handle this is through having multiple settings files, since I use that to maintain a set of common settings with modifications for each instance. It's a little more complicated to set up than some of the other solutions, but I needed to do it anyway because I was managing slightly different settings for local development, remote development, staging and production.
https://code.djangoproject.com/wiki/SplitSettings has a number of options for managing settings, and I've chosen a practice similar to the one described at https://code.djangoproject.com/wiki/SplitSettings#SimplePackageOrganizationforEnvironments
So, in my Django project directory, I have a settings folder that looks like this:
$ tree settings
settings
├── defaults.py
├── dev.py
├── dev.pyc
├── __init__.py
├── lettuce.py
├── travis.py
├── unittest.py
The common settings are in settings/defaults.py and I import these in my instance settings files. So settings/unittest.py looks like this:
from defaults import *
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'my_database',
}
}
Then, when I want to run tests, I just execute:
$ ./manage.py test --settings=settings.unittest
to use sqlite for testing. I'll use a different settings module if I want to use a different test runner or database configuration.

You can specify test database in settings.py. See
https://docs.djangoproject.com/en/3.0/topics/testing/overview/#the-test-database
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'USER': 'mydatabaseuser',
'NAME': 'mydatabase',
'TEST': {
'NAME': 'mytestdatabase',
},
},
}

I solved this issue simply creating other settings constant DATABASES_AVAILABLE.
DATABASES_AVAILABLE = {
'main': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'nep',
'USER': 'user',
'PASSWORD': 'passwd',
'HOST': 'localhost',
},
'remote': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'nes_dev',
'USER': 'usr',
'PASSWORD': 'passwd',
'HOST': '200.144.254.136',
},
'sqlite': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
},
}
# This solves the problem with tests
# Define a system variable called DJANGO_DATABASE_TEST and set it to the
# the database you want
database = os.environ.get('DJANGO_DATABASE_TEST', 'main')
DATABASES = {
'default': DATABASES_AVAILABLE[database]
}

This accelerated dramatically test execution.
import sys
if 'test' in sys.argv:
DATABASES['default'] = {
'ENGINE': 'django.db.backends.sqlite3',
'TEST_CHARSET': 'UTF8', # if your normal db is utf8
'NAME': ':memory:', # in memory
'TEST_NAME': ':memory:', # in memory
}
DEBUG = False # might accelerate a bit
TEMPLATE_DEBUG = False
from django.core.management import call_command
call_command('syncdb', migrate=True) # tables don't get created automatically for me

Though this is already solved...
If your database for tests is just a normal DB:
I think you are not doing unit test since you rely in the database. Anyway, django contains a test type for that (not unitary): django.test.TestCase
You need to derive from django.test.TestCase instead of unittest.TestCase that will create a fresh rehershal database for you that will be destroyed when the test end.
There are interesting explanations/tips about testing with db in the following link
Testing Django Applications

If you have access to manually create the database, you could use django-nose as your TEST_RUNNER. Once installed, if you pass the following environment variable, it will not delete and re-create the database.
REUSE_DB=1 ./manage.py test
You can also add the following to settings.py so you don't have to write REUSE_DB=1 every time you want to run tests:
os.environ['REUSE_DB'] = "1"
Note: this will also leave all your tables in the databases which means test setup will be a little quicker, but you will have to manually update the tables (or delete and re-create the database yourself) when you change your models.

You can mirror your db by editing settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'USER': 'mydatabaseuser',
'NAME': 'mydatabase',
'TEST': {
'MIRROR': 'default',
},
},
}

Why could I be getting this error?
Because of insufficient permissions. You can alter the user permissions by ALTER USER username CREATEDB; after running psql with superuser priviliges.
Example,
$ sudo su - postgres
$ psql
psql (9.3.18)
Type "help" for help.
postgres=# ALTER USER username CREATEDB;
ALTER ROLE

I had your same issue. I resolved it just adding in settings.py the TEST value in the database, available in my Django version 4.0.2, in this way:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'HOST': host,
'NAME': name,
'TEST': {'NAME': 'test_db'},
'USER': user,
'PASSWORD': your_password,
}
}.
This temporary creates db with different name and the conflict is resolved.

Related

Heroku Posgres' DATABASE_URL cannot be loaded in Django's setting.py and application error occurs

When building an API server using Django+REST framework and uploading it to Heroku, hard-coding Heroku Postgres' DATABASE_URL in the DATABASES of setting.py as follows will work on Heroku without problems. (xxx is originally the information of DATABASE_URL, but we dare to hide it this time)
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'xxx',
'HOST': 'xxx',
'PORT': '5432',
'USER': 'xxx',
'PASSWORD': 'xxx,
}
}
However, you can add a .env file to your Django project, stating the following (where xxx is the information originally entered for the DATABASE_URL, but we've dared to hide it this time)
DATABASE_URL=postgres://xxxx
In setting.py
import dj_database_url
from dotenv import (
find_dotenv,
load_dotenv,
)
load_dotenv(find_dotenv())
DATABASES = {
'default': dj_database_url.config(conn_max_age=600),
}
If I put something like this, it works in the local environment, but when I upload it to heroku, I get an application error.
Do you know the cause?

Sync all tables from sqlite to postgres

I have a django application and deployed it on DigitalOcean. But the only problem is, new models, admin models, tables are not showing in django admin dashboard which is on running on server. Although I pushed al changes to github, pulled them from, and made migrations, again nothing changes. How can migrate all tables from db.sqlite3 to postgresql ?
I would bet your settings.py database is still configured to the default sqlite.
# default settings.py using SQLlite
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
}
Change that to your actual postgresql database address
Since you're deploying, it is best practice to configure those parameters as separate environnment variables, as such in your settings.py. Environnment variables would then have the actual values.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': os.environ['POSTGRE_NAME'],
'USER': os.environ['POSTGRE_USER'],
'PASSWORD': os.environ['POSTGRE_PASSWORD'],
'HOST': os.environ['POSTGRE_HOST'],
'PORT': os.environ['POSTGRE_PORT'],
}
}

Remove Database settings of django using bash script

I am trying to create a bash script to create a boilerplate django project that suits for my company. I need to delete the DATABASES in settings and append new one.
The DATABASES is a python dictionary with structure
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
}
For this I tried something like this
sed -i -e "/DATABASES = {/,/}/d" settings.py
But it resulted in a trailing }.
The output is
}
I understood that the pattern it matches is for the first curly braces but not the second. What should be the approach for this.
Why not just append a new DATABASE Configuration at the end of settings? It will automatically override previous DATABASE configuration. I use like this:
DBCONIG="DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': '${dbName}',
'USER': '${dbUName}',
'PASSWORD': '${dbPass}',
'HOST': '127.0.0.1',
'PORT': '3306',
}
}"
echo $DBCONIG >> settings.py
Here I collection dbName, dbUName etc from shell input.
For cleaner implementation approach, I keep a local_settings.py which has user machine specific setups. Inside in local_settings.py, I put the user's DB configurations. and I import this file in settings.py like this:
try:
from .local_settings import *
except:
pass
Also, I usually put local_settings.py file in .gitignore, so that these settings are not pushed to repository.

Create a separate clean database for tests in django

Hi I would like to create a separate empty database for tests. I read on django docs (https://docs.djangoproject.com/en/2.1/topics/testing/overview/)
that :
The default test database names are created by prepending test_ to the value of each NAME in DATABASES. When using SQLite, the tests will use an in-memory database by default (i.e., the database will be created in memory, bypassing the filesystem entirely!). The TEST dictionary in DATABASES offers a number of settings to configure your test database. For example, if you want to use a different database name, specify NAME in the TEST dictionary for any given database in DATABASES.
So I tried:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
},
'test_db': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'test_db.sqlite3'),
}
}
but tests still use the default database when running them with
./manage.py test
How can I create and specify a new, empty database for tests purposes?
I think you misread the documentation. Django automatically uses a separate database.
Say your config file looks like:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
}
Then Django will not use the 'default' database, if the name of the database is 'FOO', it will create a database with the name test_FOO. Such that testing and running the Django project should - without of course "patching" this behaviour - not interfere (at least not the databases).
If you however want to specify a different NAME (or other attributes), you can add a 'TEST' key [Django-doc] in the databases, like:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
'TEST': {
'NAME': os.path.join(BASE_DIR, 'other_db.sqlite3'),
}
}
}

Can I set up a database just for django testing?

Creating a temp database when running django testing is really time-consuming, so I have a thought to set up a separate database just for django testing. Is it possible? If so, how can i achieve it?
to be more specific: I don't want django to create a new database and do migration and test and destroy every single time I run ./manage.py test. I want it to connect to an existing database(When i already created for testing) and do all the work.
You can use the --keepdb option when running the test, so that the database is not destroyed at the end of each run.
./manage.py test --keepdb
I'm not sure if I understand your question, but when I do testing on development, I just use SQLite. SQLite is my testing database. When I put it on production, I use PostgreSQL. This is my settings.py when I'm testing on development:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
}
I used to use PostgreSQL for testing as well (I switched to SQLite because it is quicker to set up).
If I am testing with PostgreSQL, my settings.py would look like this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'TESTDB',
'USER': 'mydatabaseuser',
'PASSWORD': 'mypassword',
'HOST': '127.0.0.1',
'PORT': '5432',
}
}
and when I'm not testing, I'd use this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'ProductionDB',
'USER': 'mydatabaseuser',
'PASSWORD': 'mypassword',
'HOST': '127.0.0.1',
'PORT': '5432',
}
}
Like I mentioned earlier though, I started using SQLite for testing because it is quicker.