django on AWS with multiple settings file - django

Well, I had one setting file before and it was working perfectly both in local and AWS of course with both the database configuration in one setting file. When ever I use to push my code on AWS I would comment my local db configuration and then push. So for sure, that's annoying. I can't comment and uncomment constantly both the database configuration in one file.
Therefore, I decided to have 2 setting files. one for the local and the other for AWS.
Once I pulled the code into AWS server and run migrations
python manage.py migrate --settings=settings.staging
It worked and migrated. By the way, this is the setting file which resides my RDS configuration. Now the moment I hit the endpoints via postmant the output is as
OperationalError at /account/v1/login
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
means it is still reading the default settings file. How come I make the server run this particular staging setting file. Do I have to declare it on nginx, supervisor or gunicorn?
I'm using these 3 services as well.
Below is my settings file for staging.
from .base import *
# --------------- AWS RDS ---------------
DATABASES = {
'default': {
# 'ENGINE': 'django.db.backends.sqlite3',
# 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'db name here',
'HOST': 'RDS host here',
'USER': 'postgres',
'PASSWORD': 'pass here',
'PORT': '5432'
}
}
This is my complete staging.py file. which only have this while the rest of the setting is being imported from base.py which is a default setting file. Also it has the local settings.
Any recommendations?
This is also what I've tried.
import socket
hostname = socket.gethostname()
if hostname == "staging":
from settings.staging import *
Thank you

The way I solved this issue is using hostnames. We have 1 settings file with all the default settings. We then import specific files depending on the hostname. Of course you could also use things like IAM-instance roles or something.
We would have this in the default settings file:
import socket
DATABASE = {'default': {'ENGINE': 'django.db.backends.sqlite3', ...}}
hostname = socket.gethostname()
if hostname == "staging-blabla"
from staging import *
staging.py would contain the following:
DATABASE = {'default': {'ENGINE': 'django.db.backends.postgresql_psycopg2', ...}}

Related

Django is trying to authenticate Wrong user for AWS RDS and failing

I am hosting a postgresql database on AWS using the RDS service. I am trying to connect the django project to the aws database using the settings.py file in the django project. I keep getting the following error:
connection to server at "database-1.xxxxxx.us-east-1.rds.amazonaws.com" (xx.xxx.xx.xxx), port 5432 failed: FATAL: password authentication failed for user "local_user"
This error is unexpected because it should not be trying to authenticate the local_user as this is the user for my local postgres server, it should be trying to authenticate the user for the hosted database, which is completely different.
This is what my settings file looks like:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': os.environ.get('DATABASE_NAME'),
'USER': os.environ.get('DATABASE_USER'),
'PASSWORD': os.environ.get('DATABASE_PASSWORD'),
'HOST': "database-1.xxxxxx.us-east-1.rds.amazonaws.com",
'PORT': 5432
}
}
I can't figure out what the issue seems to be. Any help will be appreciated!
I named the environment variable wrong so USER was none!

Why am I getting error "Client with IP address x.x.x.x isnt allowed to connect to this MySql server" when trying to connect to MySql from Django?

I am trying to create tables for my models in this full stack project (Django/React). After I run the python manage.py migrate command I'm expecting to see:
Migrations for X:
X\migrations\0001_initial_py
- Create model Departments
- Create model Employees
Instead I get the following error: (9000, "Client with IP address 'X' is not allowed to connect to this MySQL server."
I have tried the command pip install pymysql
then edited the settings.py file adding
import pymysql
pymysql.install_as_MySQLdb()
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'mytestdb',
'USER': 'testadmin#mytestmysql',
'PASSWORD': 'XXXXXXXXXX#',
'HOST': 'mytestmysql.mysql.database.XXXXXXXXXXXXX.com',
'PORT': '3306'
MySQL permissions are such that you have to specify 'user'#'host' when defining privileges. My guess is that you didn't set the MySQL database user up correctly.
Read here to understand what I'm referring to.

Connect Django to Google Cloud SQL

I'm trying to connect Django to the Google cloud SQL, working with python 2.7 and django 1.5 under windows. I went through the instructions on this page: https://developers.google.com/appengine/docs/python/cloud-sql/django
My settings.py file has basic database settings of the form:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'INSTANCE': 'my_project:instance1',
'NAME': 'my_database',
}
}
With of course a proper could SQL instance and a database created through the SQL prompt of the google apis console
When I try to run manage.py syncdb for the first time in order to obtain an OAuth2 token, I get this:
OperationalError: (2003, "Can't connect to MySQL server on 'localhost'
(10061)")
Before you ask, I did make sure that both the django and google packages are in my PYTHONPATH, as well as "C:\Program Files (x86)\Google\google_appengine\lib\django-1.5"
Any help would be really welcome!
That database configuration only makes sense when connecting from AppEngine. If you want to access your CloudSQL database from your local machine using django, you should use the google.appengine.ext.django.backends.rdbms engine.
You can see the different configuration options here:
https://developers.google.com/appengine/docs/python/cloud-sql/django#development-settings
EDIT: The google.appengine.ext.django.backends.rdbms engine has been deprecated. If you want to connect to Google Cloud SQL from your local machine you should use IP connectivity. You can use the Cloud SQL instance IP (IPv4 or IPv6) and connect using the standard django.db.backends.mysql engine.
Example connection to Google Cloud SQL in Django:
AppEngine Standard, Python 2.7:
try:
import MySQLdb # noqa: F401
except ImportError:
import pymysql
pymysql.install_as_MySQLdb()
if os.getenv('SERVER_SOFTWARE', '').startswith('Google App Engine'):
# Running on production App Engine, so connect to Google Cloud SQL using
# the unix socket at /cloudsql/<your-cloudsql-connection string>
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'HOST': '/cloudsql/<your-cloudsql-connection-string>',
'NAME': '<your-database-name>',
'USER': '<your-database-user>',
'PASSWORD': '<your-database-password>',
}
}
else:
# Running locally so connect to either a local MySQL instance or connect to
# Cloud SQL via the proxy. To start the proxy via command line:
#
# $ cloud_sql_proxy -instances=[INSTANCE_CONNECTION_NAME]=tcp:3306
#
# See https://cloud.google.com/sql/docs/mysql-connect-proxy
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'HOST': '127.0.0.1', # DB's IP address
'PORT': '3306',
'NAME': '<your-database-name>',
'USER': '<your-database-user>',
'PASSWORD': '<your-database-password>',
}
}
Source: GCP Python Django Samples AppEngine Standard Python 2.7
AppEngine Standard, Python 3.7:
# Install PyMySQL as mysqlclient/MySQLdb to use Django's mysqlclient adapter
# See https://docs.djangoproject.com/en/2.1/ref/databases/#mysql-db-api-drivers
# for more information
import pymysql # noqa: 402
pymysql.install_as_MySQLdb()
if os.getenv('GAE_APPLICATION', None):
# Running on production App Engine, so connect to Google Cloud SQL using
# the unix socket at /cloudsql/<your-cloudsql-connection string>
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'HOST': '/cloudsql/[YOUR-CONNECTION-NAME]',
'USER': '[YOUR-USERNAME]',
'PASSWORD': '[YOUR-PASSWORD]',
'NAME': '[YOUR-DATABASE]',
}
}
else:
# Running locally so connect to either a local MySQL instance or connect to
# Cloud SQL via the proxy. To start the proxy via command line:
#
# $ cloud_sql_proxy -instances=[INSTANCE_CONNECTION_NAME]=tcp:3306
#
# See https://cloud.google.com/sql/docs/mysql-connect-proxy
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'HOST': '127.0.0.1', # DB's IP address
'PORT': '3306',
'NAME': '[YOUR-DATABASE]',
'USER': '[YOUR-USERNAME]',
'PASSWORD': '[YOUR-PASSWORD]',
}
}
Source GCP Python Django Samples AppEngine Standard Python 3.7
AppEngine Flexible:
DATABASES = {
'default': {
# If you are using Cloud SQL for MySQL rather than PostgreSQL, set
# 'ENGINE': 'django.db.backends.mysql' instead of the following.
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'polls',
'USER': '<your-database-user>',
'PASSWORD': '<your-database-password>',
# For MySQL, set 'PORT': '3306' instead of the following. Any Cloud
# SQL Proxy instances running locally must also be set to tcp:3306.
'PORT': '5432',
}
}
# In the flexible environment, you connect to CloudSQL using a unix socket.
# Locally, you can use the CloudSQL proxy to proxy a localhost connection
# to the instance
DATABASES['default']['HOST'] = '/cloudsql/<your-cloudsql-connection-string>'
if os.getenv('GAE_INSTANCE'):
pass
else:
DATABASES['default']['HOST'] = '127.0.0.1' # DB's IP address
Source GCP Python Django Samples AppEngine Flexible

OperationalError could not connect to server

I put a Django app on Heroku recently. The home page looks fine, but when I try to go to a page that involves making a query (e.g. p = Photo.objects.get(title=title)), I get this error:
could not connect to server: Connection refused
Is the server running on host "localhost" and accepting
TCP/IP connections on port 5432?
In accordance with this answer, I did $ heroku pg:promote HEROKU_POSTGRESQL_GREEN_URL
Then in my settings.py:
DATABASES = {'default': dj_database_url.config(default=os.environ['DATABASE_URL'])}
Still got the same error, so I tried looking at the results of this (as this answer suggests):
$ heroku run python manage.py shell
>>> from django.conf import settings
>>> print settings.DATABASES['default']
{'TIME_ZONE': 'UTC', 'TEST_MIRROR': None, 'NAME': 'snorthway', 'OPTIONS': {},
'HOST': 'localhost', 'TEST_NAME': None, 'PASSWORD': '******', 'ENGINE':
'django.db.backends.postgresql_psycopg2', 'PORT': '', 'USER': 'snorthway',
'TEST_COLLATION': None, 'TEST_CHARSET': None}
At which point I realized I don't know what I should even be looking for in that. I still don't understand what the error means, so I am unsure how to go about debugging it.
You have not configured your django database correctly in settings.py. It thinks your database is on localhost. Sounds like you have a heroku postgres database so your host should be something like:
df3-64-304-50-250.compute-1.amazonaws.com
Heroku exposes a special database URL through an environment variable called:
DATABASE_URL
There is a very cool python package here called dj_database_url: https://github.com/kennethreitz/dj-database-url it converts that environment variable to what django expects.
you can install it with:
$pip install dj-database-url
I use the following in my settings.py
import dj_database_url
DATABASES = {
'default': dj_database_url.config()
}

Heroku Database Settings Injection - How do I setup my dev django database?

I'm trying to get my local dev django app to work after following these instructions on adding env database settings.
https://devcenter.heroku.com/articles/django-injection
I followed the instructions but get the following error when my app tries to access the local database
Request Method: GET
Request URL: http://localhost:8000
Django Version: 1.4
Exception Type: ImproperlyConfigured
Exception Value:
You need to specify NAME in your Django settings file.
My database settings originally,
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'db', # Or path to database file if using sqlite3.
'USER': 'foo', # Not used with sqlite3.
'PASSWORD': 'bar', # Not used with sqlite3.
'HOST': 'localhost',
'PORT': '5432',
}
}
the heroku article says to add the following to the settings file
import dj_database_url
DATABASES = {'default': dj_database_url.config(default='postgres://localhost')}
how do I get dj_database_url.config to use my my dev settings when the DATABASE_URL is not available in dev?
You can just add your dev settings to the default values like this...
import dj_database_url
DATABASES = {'default': dj_database_url.config(default='postgres://foo:bar#localhost:5432/db')}
Use this in your settings.py:
DATABASES = {'default': dj_database_url.config(default=os.environ['DATABASE_URL'])}
and in your .env file have this:
DATABASE_URL=postgres://localhost/yourdbname
when you launch with "foreman start" it will look at the .env file and create all those environment variables, just like running on Heroku itself. Type "heroku config" to confirm that you have a DATABASE_URL set, which you should if you added the postgres database addon.
Just set an environment variable on your operating system and check weither or not it's set. For instance, with a UNIX system:
# In ~/.bash_profile
export LOCAL_DEV=true
# In settings.py
import dj_database_url
DATABASES = {'default': dj_database_url.config(default='postgres://localhost')}
if bool(os.environ.get('LOCAL_DEV', False)):
# Override DATABASES['default'] with your local database configuration
Also, if you need to set an environment variable on your heroku space:
heroku config:add MY_VAR='my_value'
I just tried this and here is my code:
import dj_database_url
local_db = 'postgres://django_login:123456#localhost/django_db'
DATABASES = {'default': dj_database_url.config(default=local_db)}
My database name is "django_db", user name is "django_login", password is "123456".
My code can run both in local machine and heroku.
import dj_database_url
DATABASES = {'default':
dj_database_url.config(default='postgres://yourusername:yourpassword#yourhosturl:5432/yourdbname')}
** Replace bold string with your database settings
if you are using local database then replace yourhosturl by localhost