postgres & docker compose: change service status to ready after additional initialization - django

Question

In my case, a postgres database works as the main django backend database. Additional postgres initialization is required. The problem is that status of postgres service becomes ready before additional database initialization. As a result, dependent django app starts running prior to database initialization.
Is there any way to configure postgres service in a way that becomes ready after additional initialization?
docker-compose.yml:
version: "3.3"
services:
postgres:
image: library/postgres:11
volumes:
- some_folder:/docker-entrypoint-initdb.d
django_app:
image: custom_django_image:latest
volumes:
- $PWD:/app
ports:
- 8000:8000
depends_on:
- postgres

Answer 1

Your some_folder, which is mapped to the Postgres container's /docker-entrypoint-initdb.d location is where you (and seems like you are doing) should place your initialization scripts. As long as there is no data in a
possible volume attached to the Postgres container's /var/lib/postgresql/data directory (persisting data), upon container creation, Postgres will first run those scripts before setting Postgres to a ready state. The scripts must be either .sh or .sql files. (documentation). I'll show a typical workflow I use:
I have this script which creates multiple databases:
# create-multiple-databases.sh
set -e
set -u
function create_user_and_database() {
local database=$1
echo " Creating user and database '$database'"
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" <<-EOSQL
CREATE USER $database;
CREATE DATABASE $database;
GRANT ALL PRIVILEGES ON DATABASE $database TO $database;
EOSQL
}
if [ -n "$POSTGRES_MULTIPLE_DATABASES" ]; then
echo "Multiple database creation requested: $POSTGRES_MULTIPLE_DATABASES"
for db in $(echo $POSTGRES_MULTIPLE_DATABASES | tr ',' ' '); do
create_user_and_database $db
done
echo "Multiple databases created"
fi
In the docker-compose.yml file I set:
environment:
- POSTGRES_MULTIPLE_DATABASES=dev,test
Now when running docker-compose up, I see the output from the script, and then finally:
postgres_1 | CREATE DATABASE
postgres_1 | GRANT
postgres_1 | Multiple databases created
...
postgres_1 | PostgreSQL init process complete; ready for start up.
postgres_1 |
postgres_1 | 2020-05-23 16:18:40.055 UTC [1] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
postgres_1 | 2020-05-23 16:18:40.056 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
postgres_1 | 2020-05-23 16:18:40.056 UTC [1] LOG: listening on IPv6 address "::", port 5432
postgres_1 | 2020-05-23 16:18:40.063 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
Now inside my Django application, in a typical settings.py file (or similar related settings file), I create a while loop that only completes once that database is ready. Once completed, the Django server continues its initialization process and then starts running the server.
while True:
from django.db import connections
from django.db.utils import OperationalError
conn = connections["default"] # or some other key in `DATBASES`
try:
c = conn.cursor()
LOGGER.info("Postgres Ready")
break
except OperationalError:
LOGGER.warning("Postgres Not Ready...")
time.sleep(0.5)
Providing I have understood your question, I hope this gives you the information needed in your question.

Answer 2

I would highly suggest looking at health checks of docker-compose.yml.
You can change the healthcheck command to specific postgresql check, Once health-check is ready, then only djano will start sending the request to the postgresql container.
Please consider using the below file.
version: "3.3"
services:
postgres:
image: library/postgres:11
volumes:
- some_folder:/docker-entrypoint-initdb.d
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
django_app:
image: custom_django_image:latest
volumes:
- $PWD:/app
ports:
- 8000:8000
depends_on:
- postgres
Ref:- https://docs.docker.com/compose/compose-file/#healthcheck

Answer 3

You can take a look at how Django Cookiecutter Template does it: https://github.com/pydanny/cookiecutter-django
What they do here is use a startup script for production using Docker: here
This entrypoint script checks, whether postgres is up and running and only proceeds then, otherwise it loops over the connection. You can write a similar script based on that snippet to check, whether your postgres instance is up and running correctly. Whether it is ready or not you can decide in your script docker-entrypoint-initdb.d. It should do all the operations it requires. Once it's done, you can startup postgres and the script for Django should then proceed because it can reach the postgres instance.
It works well for me, I do some other checks in the entrypoint script before I proceed to starting the server itself.