Same Django projects running on different databases - django

I am trying to run two identical Django projects on different databases, one for production using a certain port number (say, 80) and the other for testing using another port number (say, 8000). I also used Nginx and Gunicorn as the reverse proxy server and the application server, with Nginx listening to ports 80 and 8000 and forwarding to gunicorn of ports 8001 and 8002, respectively.
The problem is: how do I know the port number of the request in Django's settings.py so that the project can choose different databases?

The standard practice for doing this in django is to create a local_settings.py file
Put this at the top of the local_settings.py file:
try:
from settings import *
except ImportError:
pass
Now in local_settings.py you must override the following variable:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'dbname',
'USER': 'dbuser',
'PASSWORD': 'dbpassword',
'HOST': '127.0.0.1',
'PORT': '5432',
}
}
With different values for both projects
Then when running Django you need to set the following environment variable:
export DJANGO_SETTINGS_MODULE="appname.local_settings"
One way to bunch this all together is to create a run.sh file that first sets this variable, then runs gunicorn
To sum it up, settings.py is common between both projects, local_settings.py overrides those variables that are different between different projects

Related

Test database for Django + Heroku. Error creating the test database: permission denied to create database

I'm trying to run the tests for my Django project. I wrote this project some time ago, I had different settings then and tests were passing. Now I changed settings and deployed it on Heroku with Heroku Postgres database. Everything works fine already except I can't run tests. I've tried many different settings and nothing worked. Most of the time I'm getting this error: permission denied to create database
My last setting is following the instruction from this article on medium
Basically I have added 2nd Heroku Postgres database, add settings like below (but with valid variables of my heroku databases):
if 'test' in sys.argv:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'd7osdssadag0ugv5',
'USER': 'lhwwasadqlgjra',
'PASSWORD': '1524f48a2ce41177c4ssdadasd3a11680b735302d14979d312ff36',
'HOST': 'ec2-54-75-2326-118.eu-west-1.compute.amazonaws.com',
'PORT': 5432,
'TEST': {
'NAME': 'd7osdssadag0ugv5',
}
}
}
else:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'd7hasadas9hqts5',
'USER': 'nybkduadsdgqot',
'PASSWORD': 'bb535b9cdsfsdfdsfdsfac54851f267444dd8cc230b2a786ab9f446',
'HOST': 'ec2-54-247-132-38.eu-west-1.compute.amazonaws.com',
'PORT': 5432,
'TEST': {
'NAME': 'd7hasadas9hqts5',
}
}
}
Then run python manage.py test --keepdb in my venv. Then I get an error:
RuntimeWarning: Normally Django will use a connection to the 'postgres' database to avoid running initialization queries against the production database when it's not needed (for example, when running tests). Django was unable to create a connection to the 'postgres' database and will use the first PostgreSQL database instead.
RuntimeWarning
Got an error creating the test database: permission denied to create database
I have also tried what is advised in this article
Do you have any ideas what I could do about this error? I don't know Django well. I play with it from time to time.
I'm using: Python 3.6.9, Django 3.0.3, Heroku Postgresql Hobby Dev
EDIT:
I'm not sure if this is now an issue with my settings DATABASES.
Now when I commented out all my settings concerning DATABASES and I run python manage.py runserver my development server starts as normal and I have access to a database I set before (even after restarting a computer). This looks like actual settings don't have effect (??) Any thoughts?
Django version 3.0.3, using settings 'forumproject.settings'
Starting development server at http://127.0.0.1:8000/
Ok, I found out what it was. My database settings were not taken into account, even I had DEBUG=True because I had this line on the end of the settings:
# Activate Django-Heroku.
django_heroku.settings(locals())
After commenting this out the error permission denied to create database
goes away and I can run tests with
python manage.py test --keepdb
I'm surprised how always posting a question on stack overflow help me to find answer immediately after. I was running in circles

Django Unit tests fail over domain socket

I have Django configured to use the database with peer authentication over the local Unix Domain socket, instead of user/password authentication.
Here's the settings.DATABASES:
{'default': {'ENGINE': 'django.db.backends.postgresql',
'NAME': 'mcps',
'PORT': 5433,
'TEST': {'ENGINE': 'django.db.backends.postgresql',
'NAME': 'mytestdb',
'PORT': 5433,
'USER': 'mcp'},
'USER': 'mcp'}
}
The port is correctly configured, the application itself has no problem working correctly.
Yet, when I try to run pytest, with the environment variable DJANGO_SETTINGS_MODULE set to the above settings, a database is created - with the correct owner 'mcp' - but before tables are created I get an error:
django.db.utils.OperationalError: FATAL: Peer authentication failed for user "mcp"
What are unit tests doing differently anmd how can I fix this please?
#Nader Alexan There's no host to set, communication goes over the local Unix domain socket. I tried adding
'HOST': '/run/postgresql'
as I had to do in pgAdmin, but that didn't solve the issue.
It turns out that pytest also tries to access the database named 'postgres', even though the maintenance database is set to template1. I haven't been able to determine why, but adding access to that database explictely in pg_hba.conf enabled me to run the tests.
Sorry if this explanation is a bit shaky, I'm new to forms of PG authentication other than username/password myself.

Django settings when using pgbouncer

I have a Django website with Postgresql backend, for which I'm utilizing pgbouncer for db connection pooling (transaction mode).
The application and the DB reside on separate servers (1 server each). I have installed pgbouncer on the application server. My question is: what should the config be in settings.py? Note that I'm using Unix sockets for connecting to pgbouncer.
My current settings.py contains:
DATABASE_URL = 'postgres://user1:pass1#xx.xxx.xxx.xxx:5432/db1'
DATABASES = {
'default': dj_database_url.config(default=DATABASE_URL)
}
Relevant sections of pgbouncer.ini are:
[databases]
db1 = host=xx.xxx.xxx.xxx port=5432 dbname=db1
listen_addr = *
listen_port = 6432
auth_type = md5
unix_socket_dir = /var/run/postgresql
pool_mode = transaction
max_client_conn = 200
default_pool_size = 300
userlist.txt contains:
"user1" "pass1"
Note: One answer is here, but doesn't work for me since the DB isn't available locally in my case. I need to set the DATABASE_URL environment variable, instead of using default = '...'.
One suggestions seems to be to treat pgbouncer as a database in settings.py. In that case, would something like the following work?
if PRODUCTION == '1':
#PRODUCTION is set to '1' if in production environment
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'pgbouncer',
'USER': 'user1',
'PASSWORD': 'pass1',
'HOST': '/var/run/postgresql',
'PORT': '6432',
}
From the docs:
pgbouncer is a PostgreSQL connection pooler. Any target application
can be connected to pgbouncer as if it were a PostgreSQL server, and
pgbouncer will create a connection to the actual server, or it will
reuse one of its existing connections.
Also,
Have your application (or the psql client) connect to pgbouncer
instead of directly to PostgreSQL server.
The configurations:
pgbouncer.ini: An example pgbouncer.ini with comments about defaults
[databases]
db1 = host=xx.xxx.xxx.xxx port=5432 dbname=db1
[pgbouncer]
listen_addr = *
listen_port = 6432
auth_type = md5
auth_file = userlist.txt
unix_socket_dir = /var/run/postgresql
pool_mode = transaction
max_client_conn = 100
default_pool_size = 20
userlist.txt:
"user1" "pass1"
to put in settings.py:
if PRODUCTION == '1':
#PRODUCTION is set to '1' if in production environment
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'db1',
'USER': 'user1',
'PASSWORD': 'pass1',
'HOST': '/var/run/postgresql',
# 'PORT': '6432',
}
Extra:
In case not using unix socket - you can set HOST : '127.0.0.1' or 'localhost' if pgbouncer is running locally, or whatever the IP of server pgbouncer will be running on.
From the docs:
If you’re using PostgreSQL, by default (empty HOST), the connection to
the database is done through UNIX domain sockets (‘local’ lines in
pg_hba.conf). If your UNIX domain socket is not in the standard
location, use the same value of unix_socket_directory from
postgresql.conf. If you want to connect through TCP sockets, set HOST
to ‘localhost’ or ‘127.0.0.1’ (‘host’ lines in pg_hba.conf). On
Windows, you should always define HOST, as UNIX domain sockets are not
available.
In case of postgreSQL For ENGINE you can use postgresql or postgresql_psycopg2 - there's difference between the both given your Django version - postgresql_psycopg2 vs posgresql.
All of your DB settings in settings.py should be identical to the settings in your pgbouncer config, except the host in settings.py will point to pgbouncer. You probably need to change 'NAME': 'pgbouncer' to 'NAME': 'db1'. Since you're using a unix socket the port shouldn't matter.

Postgresql not connecting in django project

Working on django project with postgresql database but its giving me error :-
psql: FATAL: no pg_hba.conf entry for host "::1", user "dev_onpoint", database "dev_mypsdb", SSL off
My database configuration in django is :-
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'dev_mypsdb',
'USER': 'dev_onpoint',
'PASSWORD': '*****',
'HOST': 'localhost',
'PORT': '5432'
}
}
I also checked in postgresql configuration file for port and found that i am using right port number.
Then i checked ph_hba.conf file but because I am not familiar with postgresql database and I am using it first time, that's why I could not understand this file :-
enter image description here
Please let me know if i skipped anything in configuration.
Thanks
Your pg_hba.conf file has no access control entry for IPv6 connections, that's why connect from ::1 fails. You can add the following line to your pg_hba.conf:
host all all samehost md5
to allow connections from all local IPs using password authentication (you must restart postgresql after editing this file). For more information refer to the postgres documentation.

Upload Django app and set database correctly

Hi I want to upload my django app to openshift rhc with git. After push and refresh the mainpage is displayed, however everything which needs user instance is not working.
My error is:
'Ident authentication failed for user "admin"'
...py2.6.egg/django/db/backends/postgresql_psycopg2/base.py in _cursor, line 177
I think database is not connected properly:
in my settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'MY_APP_NAME',
'USER': 'admin',
'PASSWORD': 'MY_PASSWORD',
'HOST': '',
'PORT': '',
}
I didn't do manage.py syncdb, I do not know how to do it. Maybe that is the problem, because superuser is not created ?
What about path to db? On my computer settings.py looks a litle bit different:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': '//home//pachucx//Project//db//sqlite3.db',
'USER': '',
'PASSWORD': '',
'HOST': '',
'PORT': '',
}
Everything worked fine on the computer.
The difference is that in NAME I am using path to the DB file, where on serwer side I use only name of it. Should I add extension .db, or maybe there is need to add a path I dont know.
Or maybe the problem will be solved just by manage.py syncdb? If it is so, tell me how to do it properly e.g with git
Many thanks.
This is an error that is occurring because of your attempt to access the database that is either not created, or you have invalid username and password credentials in your settings file.
Make sure you configure all the settings correctly. - Check your host, db username and password as well as the port - it could be different from your local box.
In order to sync the database you need to navigate in SSH to your project root and locate the manage.py file. Once you have that run the following command:
python manage.py syncdb
This will either:
Create all the tables,
Give you a nasty error saying your settings for your database are not correct, this is a great error to get as you know you need to look into the settings again and correct the problem.
To test whether or not your server can be initiated run the following command in SSH again
python manage.py runserver
If successful, this will give you a local test environment and should spit out a url to test. Should be localhost:8000 or something similar.
If fails, this will let you know if you have models improperly configured, mainly your settings, or url file it will give an error on so you can double check everything is up.
As for using the database that is on your machine on a machine outside of your network I advise to not do that. Go and create the database on the box you have, and make notes of the host, username, password, database name etc, and go back to start of this answer.
All the best,