Remove Database settings of django using bash script - django

I am trying to create a bash script to create a boilerplate django project that suits for my company. I need to delete the DATABASES in settings and append new one.
The DATABASES is a python dictionary with structure
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
}
For this I tried something like this
sed -i -e "/DATABASES = {/,/}/d" settings.py
But it resulted in a trailing }.
The output is
}
I understood that the pattern it matches is for the first curly braces but not the second. What should be the approach for this.

Why not just append a new DATABASE Configuration at the end of settings? It will automatically override previous DATABASE configuration. I use like this:
DBCONIG="DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': '${dbName}',
'USER': '${dbUName}',
'PASSWORD': '${dbPass}',
'HOST': '127.0.0.1',
'PORT': '3306',
}
}"
echo $DBCONIG >> settings.py
Here I collection dbName, dbUName etc from shell input.
For cleaner implementation approach, I keep a local_settings.py which has user machine specific setups. Inside in local_settings.py, I put the user's DB configurations. and I import this file in settings.py like this:
try:
from .local_settings import *
except:
pass
Also, I usually put local_settings.py file in .gitignore, so that these settings are not pushed to repository.

Related

Heroku Posgres' DATABASE_URL cannot be loaded in Django's setting.py and application error occurs

When building an API server using Django+REST framework and uploading it to Heroku, hard-coding Heroku Postgres' DATABASE_URL in the DATABASES of setting.py as follows will work on Heroku without problems. (xxx is originally the information of DATABASE_URL, but we dare to hide it this time)
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'xxx',
'HOST': 'xxx',
'PORT': '5432',
'USER': 'xxx',
'PASSWORD': 'xxx,
}
}
However, you can add a .env file to your Django project, stating the following (where xxx is the information originally entered for the DATABASE_URL, but we've dared to hide it this time)
DATABASE_URL=postgres://xxxx
In setting.py
import dj_database_url
from dotenv import (
find_dotenv,
load_dotenv,
)
load_dotenv(find_dotenv())
DATABASES = {
'default': dj_database_url.config(conn_max_age=600),
}
If I put something like this, it works in the local environment, but when I upload it to heroku, I get an application error.
Do you know the cause?

Is the following enough to protect my secret key and database password for heroku deployment?

I have added the environment variables through to get secret key database name and password. Now anything else to make it more secure.Let me know.
Used "Advance system settings" and under that added environment variables in "User variables" .
Also this is going to be my first deployment on 'heroku' that's why following a tutorial.
SECRET_KEY = os.environ['DB_SECRET']
databases
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'portfoliodb',
'USER': os.environ.get('DB_USER'),
'PASSWORD': os.environ.get('DB_PASS'),
'HOST': 'localhost',
'PORT': '5432',
}
}
Heroku has a package that handles that for you. More info here. Essentially, it overrides default with secure credentials automatically generated in the Heroku platform.
Add the following import statement to the top of settings.py:
import django_heroku
Then add the following to the bottom of settings.py:
Activate Django-Heroku.
django_heroku.settings(locals())

Create a separate clean database for tests in django

Hi I would like to create a separate empty database for tests. I read on django docs (https://docs.djangoproject.com/en/2.1/topics/testing/overview/)
that :
The default test database names are created by prepending test_ to the value of each NAME in DATABASES. When using SQLite, the tests will use an in-memory database by default (i.e., the database will be created in memory, bypassing the filesystem entirely!). The TEST dictionary in DATABASES offers a number of settings to configure your test database. For example, if you want to use a different database name, specify NAME in the TEST dictionary for any given database in DATABASES.
So I tried:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
},
'test_db': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'test_db.sqlite3'),
}
}
but tests still use the default database when running them with
./manage.py test
How can I create and specify a new, empty database for tests purposes?
I think you misread the documentation. Django automatically uses a separate database.
Say your config file looks like:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
}
Then Django will not use the 'default' database, if the name of the database is 'FOO', it will create a database with the name test_FOO. Such that testing and running the Django project should - without of course "patching" this behaviour - not interfere (at least not the databases).
If you however want to specify a different NAME (or other attributes), you can add a 'TEST' key [Django-doc] in the databases, like:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
'TEST': {
'NAME': os.path.join(BASE_DIR, 'other_db.sqlite3'),
}
}
}

Changing from Sqllite to Postgres on Heroku- why it is not working?

I am a bit confused , hope someone can help me out
Originally I uploaded my django app to heroku account with sqllite db
This is what I had in my settings file for DB
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
} }
import dj_database_url
db_from_env = dj_database_url.config()
DATABASES['default'].update(db_from_env)
It didnt work great since SQLlite was getting flashed to its original state every 24 hours (but worked perfectly since I needed demo system)
However now I have to make it productive so I want to change the db to connect to postgres . So I used credentials from postgres DB I created with heroku and my db in settings looks like this
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'd4drq1yytest',
'USER': 'xvvqgkjtest',
'PASSWORD': 'test5y55y5y5y5y5y5y5y5y5y54y45',
'HOST': '777-77-77-67-7.compute-1.amazonaws.com',
'PORT': '5432',
}
}
import dj_database_url
db_from_env = dj_database_url.config()
DATABASES['default'].update(db_from_env)
But even I changed the settings file and redeployed the new file I still see that my heroku app is connected to my sqllite DB.
What did I do wrong? (I deploy using a master brunch from my github.)
If you have not already, you might need to add heroku postgres as an app in your heroku app's dashboard.
It seems like you have used the same code that I used to connect my heroku's postgres to heroku. You should only have to state
import dj_database_url
db_from_env = dj_database_url.config()
DATABASES['default'].update(db_from_env)
then make sure you have pip installed whitenoise and added that to requirements.txt. After commmiting, and pushing to heroku, you may want to run heroku run bash on cmd or terminal. Then you may want to run python manage.py makemigrations yourappname afterwards, you may want to run python manage.py migrate, and if you get no errors, you might want to check on your heroku dashboard for your app, clicking on postgres, to see if any of the postgres' tables are in use. If they are, then you are using the heroku postgres.
you may not need the lines of code below for heroku
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'd4drq1yytest',
'USER': 'xvvqgkjtest',
'PASSWORD': 'test5y55y5y5y5y5y5y5y5y5y54y45',
'HOST': '777-77-77-67-7.compute-1.amazonaws.com',
'PORT': '5432',
}
}
this should be where you are calling the database you will be testing with your localhost.

Different db for testing in Django?

DATABASES = {
# 'default': {
# 'ENGINE': 'postgresql_psycopg2',
# ...
# }
# for unit tests
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'mydatabase'
}
}
I have two databases: one I'd like to use for unit tests, and one for everything else. Is it possible to configure this in Django 1.2.4?
(The reason I ask is because with postgresql I'm getting the following error:
foo#bar:~/path/$ python manage.py test
Creating test database 'default'...
Got an error creating the test database: permission denied to create database
Type 'yes' if you would like to try deleting the test database 'test_baz', or 'no' to cancel: yes
Destroying old test database...
Got an error recreating the test database: database "test_baz" does not exist
Why could I be getting this error? I guess I don't really care if I can always use SQLite for unit tests, as that works fine.)
In your settings.py (or local_settings.py):
import sys
if 'test' in sys.argv:
DATABASES['default'] = {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'mydatabase'
}
The way I handle this is through having multiple settings files, since I use that to maintain a set of common settings with modifications for each instance. It's a little more complicated to set up than some of the other solutions, but I needed to do it anyway because I was managing slightly different settings for local development, remote development, staging and production.
https://code.djangoproject.com/wiki/SplitSettings has a number of options for managing settings, and I've chosen a practice similar to the one described at https://code.djangoproject.com/wiki/SplitSettings#SimplePackageOrganizationforEnvironments
So, in my Django project directory, I have a settings folder that looks like this:
$ tree settings
settings
├── defaults.py
├── dev.py
├── dev.pyc
├── __init__.py
├── lettuce.py
├── travis.py
├── unittest.py
The common settings are in settings/defaults.py and I import these in my instance settings files. So settings/unittest.py looks like this:
from defaults import *
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'my_database',
}
}
Then, when I want to run tests, I just execute:
$ ./manage.py test --settings=settings.unittest
to use sqlite for testing. I'll use a different settings module if I want to use a different test runner or database configuration.
You can specify test database in settings.py. See
https://docs.djangoproject.com/en/3.0/topics/testing/overview/#the-test-database
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'USER': 'mydatabaseuser',
'NAME': 'mydatabase',
'TEST': {
'NAME': 'mytestdatabase',
},
},
}
I solved this issue simply creating other settings constant DATABASES_AVAILABLE.
DATABASES_AVAILABLE = {
'main': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'nep',
'USER': 'user',
'PASSWORD': 'passwd',
'HOST': 'localhost',
},
'remote': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'nes_dev',
'USER': 'usr',
'PASSWORD': 'passwd',
'HOST': '200.144.254.136',
},
'sqlite': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
},
}
# This solves the problem with tests
# Define a system variable called DJANGO_DATABASE_TEST and set it to the
# the database you want
database = os.environ.get('DJANGO_DATABASE_TEST', 'main')
DATABASES = {
'default': DATABASES_AVAILABLE[database]
}
This accelerated dramatically test execution.
import sys
if 'test' in sys.argv:
DATABASES['default'] = {
'ENGINE': 'django.db.backends.sqlite3',
'TEST_CHARSET': 'UTF8', # if your normal db is utf8
'NAME': ':memory:', # in memory
'TEST_NAME': ':memory:', # in memory
}
DEBUG = False # might accelerate a bit
TEMPLATE_DEBUG = False
from django.core.management import call_command
call_command('syncdb', migrate=True) # tables don't get created automatically for me
Though this is already solved...
If your database for tests is just a normal DB:
I think you are not doing unit test since you rely in the database. Anyway, django contains a test type for that (not unitary): django.test.TestCase
You need to derive from django.test.TestCase instead of unittest.TestCase that will create a fresh rehershal database for you that will be destroyed when the test end.
There are interesting explanations/tips about testing with db in the following link
Testing Django Applications
If you have access to manually create the database, you could use django-nose as your TEST_RUNNER. Once installed, if you pass the following environment variable, it will not delete and re-create the database.
REUSE_DB=1 ./manage.py test
You can also add the following to settings.py so you don't have to write REUSE_DB=1 every time you want to run tests:
os.environ['REUSE_DB'] = "1"
Note: this will also leave all your tables in the databases which means test setup will be a little quicker, but you will have to manually update the tables (or delete and re-create the database yourself) when you change your models.
You can mirror your db by editing settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'USER': 'mydatabaseuser',
'NAME': 'mydatabase',
'TEST': {
'MIRROR': 'default',
},
},
}
Why could I be getting this error?
Because of insufficient permissions. You can alter the user permissions by ALTER USER username CREATEDB; after running psql with superuser priviliges.
Example,
$ sudo su - postgres
$ psql
psql (9.3.18)
Type "help" for help.
postgres=# ALTER USER username CREATEDB;
ALTER ROLE
I had your same issue. I resolved it just adding in settings.py the TEST value in the database, available in my Django version 4.0.2, in this way:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'HOST': host,
'NAME': name,
'TEST': {'NAME': 'test_db'},
'USER': user,
'PASSWORD': your_password,
}
}.
This temporary creates db with different name and the conflict is resolved.