Heroku Database Settings Injection - How do I setup my dev django database? - django

I'm trying to get my local dev django app to work after following these instructions on adding env database settings.
https://devcenter.heroku.com/articles/django-injection
I followed the instructions but get the following error when my app tries to access the local database
Request Method: GET
Request URL: http://localhost:8000
Django Version: 1.4
Exception Type: ImproperlyConfigured
Exception Value:
You need to specify NAME in your Django settings file.
My database settings originally,
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'db', # Or path to database file if using sqlite3.
'USER': 'foo', # Not used with sqlite3.
'PASSWORD': 'bar', # Not used with sqlite3.
'HOST': 'localhost',
'PORT': '5432',
}
}
the heroku article says to add the following to the settings file
import dj_database_url
DATABASES = {'default': dj_database_url.config(default='postgres://localhost')}
how do I get dj_database_url.config to use my my dev settings when the DATABASE_URL is not available in dev?

You can just add your dev settings to the default values like this...
import dj_database_url
DATABASES = {'default': dj_database_url.config(default='postgres://foo:bar#localhost:5432/db')}

Use this in your settings.py:
DATABASES = {'default': dj_database_url.config(default=os.environ['DATABASE_URL'])}
and in your .env file have this:
DATABASE_URL=postgres://localhost/yourdbname
when you launch with "foreman start" it will look at the .env file and create all those environment variables, just like running on Heroku itself. Type "heroku config" to confirm that you have a DATABASE_URL set, which you should if you added the postgres database addon.

Just set an environment variable on your operating system and check weither or not it's set. For instance, with a UNIX system:
# In ~/.bash_profile
export LOCAL_DEV=true
# In settings.py
import dj_database_url
DATABASES = {'default': dj_database_url.config(default='postgres://localhost')}
if bool(os.environ.get('LOCAL_DEV', False)):
# Override DATABASES['default'] with your local database configuration
Also, if you need to set an environment variable on your heroku space:
heroku config:add MY_VAR='my_value'

I just tried this and here is my code:
import dj_database_url
local_db = 'postgres://django_login:123456#localhost/django_db'
DATABASES = {'default': dj_database_url.config(default=local_db)}
My database name is "django_db", user name is "django_login", password is "123456".
My code can run both in local machine and heroku.

import dj_database_url
DATABASES = {'default':
dj_database_url.config(default='postgres://yourusername:yourpassword#yourhosturl:5432/yourdbname')}
** Replace bold string with your database settings
if you are using local database then replace yourhosturl by localhost

Related

Can't import value from .env file / Django

I got a file .env with 4 values to hide sensitive data:
DATABASE_PASSWD=Password1
SECRET_KEY=Password2
VAR3=Password3
VAR4=Password4
All of above values are properly imported in Django code except the DATABASE_PASSWORD. When the DATABASES configuration is as follows:
# settings.py
from decouple import config
# ...
DB_PASSWORD=config('DATABASE_PASSWD')
SECRET_KEY=config('SECRET_KEY')
VAR3=config('VAR3')
VAR4=config('VAR4')
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': "database_name",
'USER': "database_test_admin",
'PASSWORD': DB_PASSWORD,
'HOST': "localhost",
'PORT': "5432",
}
}
The django outputs:
raise UndefinedValueError('{} not found. Declare it as envvar or define a default value.'.format(option))
decouple.UndefinedValueError: DATABASE_PASSWD not found. Declare it as envvar or define a default value.
If I hardcode password that is just the same in .env the problem is gone - the password is correct since it's my private project. Other variables works well with the same config('VAR#') function in views for example. I have no clue what could be wrong here.
SOLUTION:
The official docs pushed me to try this:
from decouple import config, AutoConfig # <-- AutoConfig added
# ...
config = AutoConfig(search_path='.env') # <-- .env file located next to manage.py
SECRET_KEY = config('SECRET_KEY')
DB_PASSWORD = config('DB_PASSWORD')
VAR3 = config('VAR3')
VAR4 = config('VAR4')
When I provided empty string by the default value the code started to work, however no real string was passed by config() - just the default, which didi not solve anything but made django server running.

django on AWS with multiple settings file

Well, I had one setting file before and it was working perfectly both in local and AWS of course with both the database configuration in one setting file. When ever I use to push my code on AWS I would comment my local db configuration and then push. So for sure, that's annoying. I can't comment and uncomment constantly both the database configuration in one file.
Therefore, I decided to have 2 setting files. one for the local and the other for AWS.
Once I pulled the code into AWS server and run migrations
python manage.py migrate --settings=settings.staging
It worked and migrated. By the way, this is the setting file which resides my RDS configuration. Now the moment I hit the endpoints via postmant the output is as
OperationalError at /account/v1/login
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
means it is still reading the default settings file. How come I make the server run this particular staging setting file. Do I have to declare it on nginx, supervisor or gunicorn?
I'm using these 3 services as well.
Below is my settings file for staging.
from .base import *
# --------------- AWS RDS ---------------
DATABASES = {
'default': {
# 'ENGINE': 'django.db.backends.sqlite3',
# 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'db name here',
'HOST': 'RDS host here',
'USER': 'postgres',
'PASSWORD': 'pass here',
'PORT': '5432'
}
}
This is my complete staging.py file. which only have this while the rest of the setting is being imported from base.py which is a default setting file. Also it has the local settings.
Any recommendations?
This is also what I've tried.
import socket
hostname = socket.gethostname()
if hostname == "staging":
from settings.staging import *
Thank you
The way I solved this issue is using hostnames. We have 1 settings file with all the default settings. We then import specific files depending on the hostname. Of course you could also use things like IAM-instance roles or something.
We would have this in the default settings file:
import socket
DATABASE = {'default': {'ENGINE': 'django.db.backends.sqlite3', ...}}
hostname = socket.gethostname()
if hostname == "staging-blabla"
from staging import *
staging.py would contain the following:
DATABASE = {'default': {'ENGINE': 'django.db.backends.postgresql_psycopg2', ...}}

Is there a way to mention the database settings while starting the Django app?

From the docs, it looks like the database needs to be present in the settings.py. Is there a way to pass the database details while running python manage.py runserver given the migrations have been completed?
Edit -
Use case - The database is not known beforehand so I can't have it hardcoded in settings.py. Also, there will be one database for the entire app.
Using a settings file other than settings.py in Django - This questions answers how to use a settings file different than settings.py. However, I am looking for ways in which I don't use settings file to specify the database credentials.
split your settings.py into two files
base_settings.py (add all the common settings here)
development.py
(add your database settings here)
like :
from .base_settings import *
DATABASES = {
'default': {
'ENGINE': 'XX.db.backends.postgresql',
'NAME': 'XX',
'USER': 'postgres',
'ATOMIC_REQUESTS':True,
'PASSWORD': '*****',
'HOST': '0.0.0.0',
'PORT': '5432',
}
}
and run your project using
python manage.py runserver 0.0.0.0:8002 --settings=django_project_name.development

Django on Heroku can't connect to Postgres with different schema

I devoted today to migrating my app from my local environment to Heroku. It's been frustrating and fun at the same time, but now I'm very stuck.
I have a schema called DCPViews which I want the DB connections to default to when running queries. I've read all the relevant tutorials / posts / tips and here's where I am:
DB hierarchy
postgres (default system database)
-> DCP (app database)
-> DCP (base tables schema)
-> DCPViews (views layer schema)
settings.py
import django_heroku
import dj_database_url
...
DATABASES = {}
# DATABASE_URL = 'postgres://<user>:<pass>#<host>:<port>/<db_name>?currentSchema=<schema>'
DATABASE_URL = 'postgres://' + \
config('DB_USER') + ':' + \
config('DB_PASSWORD') + '#' + \
config('DB_HOST') + ':' + \
config('DB_PORT') + '/' + \
config('DB_NAME') + \
'?currentSchema=' + config('DB_SCHEMA_NAME')
DATABASES['default'] = dj_database_url.config(default=DATABASE_URL, ssl_require=True)
...
# Configure Django App for Heroku
django_heroku.settings(locals())
The problem
I've tried everything to get Heroku to use the correct DATABASE_URL (with currentSchema = DCPViews), but no luck. I don't have DB permissions to create new roles nor set a search path for Heroku's default DB user. It also won't let me export DATABASE_URL manually and doesn't seem to accept the value I pass in the settings.py file. The heroku config -s command always returns the same DATABASE_URL value.
Everything works fine in my local environment, but this is a major snag. How can I get Heroku to use the correct search_path or default to the DCPViews schema when running queries?
Update
I'm send send the DB credentials and currentSchema I want to use in my settings.py file, but Heroku seems to ignore them and overwrites them when it creates the DATABASES['default'] key. Here's the Django debug when I load a page in my browser:
DATABASES
{'default': {'ATOMIC_REQUESTS': False,
'AUTOCOMMIT': True,
'CONN_MAX_AGE': 600,
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'HOST': '<host_name>',
'NAME': '<db_name>',
'OPTIONS': {'sslmode': 'require'},
'PASSWORD': '********************',
'PORT': 5432,
'TEST': {'CHARSET': None,
'COLLATION': None,
'MIRROR': None,
'NAME': None},
'TIME_ZONE': None,
'USER': '<user_name>'}}
DATABASE_URL
'postgres://<user>:<pass>#<host>:<port>/<db_name>?currentSchema=<schema>'
I cannot comment yet, but since your settings file is using an environment file, you can just go to your heroku app in heroku.com and go to your database that is connected to your app. Then go to settings and click reveal config vars. Change your DATABASE_URL to whatever your need it to be (mind you it wil probably be just an extension off of your actual heroku postgres database url)
Well, I was on the right path. I added the search_path to the ROLE and that did the trick:
ALTER ROLE <role_name> SET search_path = <go, pirates>

OperationalError could not connect to server

I put a Django app on Heroku recently. The home page looks fine, but when I try to go to a page that involves making a query (e.g. p = Photo.objects.get(title=title)), I get this error:
could not connect to server: Connection refused
Is the server running on host "localhost" and accepting
TCP/IP connections on port 5432?
In accordance with this answer, I did $ heroku pg:promote HEROKU_POSTGRESQL_GREEN_URL
Then in my settings.py:
DATABASES = {'default': dj_database_url.config(default=os.environ['DATABASE_URL'])}
Still got the same error, so I tried looking at the results of this (as this answer suggests):
$ heroku run python manage.py shell
>>> from django.conf import settings
>>> print settings.DATABASES['default']
{'TIME_ZONE': 'UTC', 'TEST_MIRROR': None, 'NAME': 'snorthway', 'OPTIONS': {},
'HOST': 'localhost', 'TEST_NAME': None, 'PASSWORD': '******', 'ENGINE':
'django.db.backends.postgresql_psycopg2', 'PORT': '', 'USER': 'snorthway',
'TEST_COLLATION': None, 'TEST_CHARSET': None}
At which point I realized I don't know what I should even be looking for in that. I still don't understand what the error means, so I am unsure how to go about debugging it.
You have not configured your django database correctly in settings.py. It thinks your database is on localhost. Sounds like you have a heroku postgres database so your host should be something like:
df3-64-304-50-250.compute-1.amazonaws.com
Heroku exposes a special database URL through an environment variable called:
DATABASE_URL
There is a very cool python package here called dj_database_url: https://github.com/kennethreitz/dj-database-url it converts that environment variable to what django expects.
you can install it with:
$pip install dj-database-url
I use the following in my settings.py
import dj_database_url
DATABASES = {
'default': dj_database_url.config()
}