Set-up: postgres, django and redis services configured via docker-compose.
I've read through similar questions on SO but can't see why my compose config results in django not being able to see the db service. The db service is internally configured to port 5432 in the compose network, but mapped to 5433 on the host machine (5432 on the host is used by another docker db container). I'm not sure what I'm missing here, hoping someone can see something in the config or logs that I can't...
error logs from web service:
stage_web_0.10 | django.db.utils.OperationalError: could not connect to server: Connection refused
stage_web_0.10 | Is the server running on host "db" (172.25.0.2) and accepting
stage_web_0.10 | TCP/IP connections on port 5432?
Django settings from inside the container (correctly looking for db at 5432):
settings.DATABASES
{'default': {'ENGINE': 'django.db.backends.postgresql_psycopg2',
'HOST': 'db',
'PORT': '5432',
'NAME': 'stage',
'USER': 'admin',
'TEST': {'NAME': 'test_stage'}}}
docker-compose services
db:
container_name: stage_${DJANGO_ENV}_db_${TAG}
env_file:
- .env
build:
context:
./db/
args: # Make these available at build time to create the new db.
- POSTGRES_HOST
- POSTGRES_DB
- POSTGRES_USER
- POSTGRES_PASSWORD
- POSTGRES_PORT
volumes:
- /data/postgresql/stage_${DJANGO_ENV}_${TAG}/:/var/lib/postgresql/data/
environment:
- TAG
- DJANGO_ENV
- POSTGRES_HOST
- POSTGRES_DB
- POSTGRES_USER
- POSTGRES_PASSWORD
- POSTGRES_PORT
ports:
- "5433:5432"
expose:
- "5432" # Publishes 5432 to other containers but NOT to host machine
web:
container_name: stage_${DJANGO_ENV}_web_${TAG}
build:
context:
./web
dockerfile: Dockerfile.stage
args:
- PIP_INDEX_URL
- PIP_TRUSTED_HOST
env_file:
- .env
# Command gets appended to the entrypoint in the dockerfile and will be run
# when all containers are up (need postgres container started for migrations).
command: /code/start-django.sh
volumes:
- ./web/web_application/:/code/web
- ${STORAGE_PATH_HOST}:${STORAGE_PATH_CONTAINER}
# certificates
- /etc/apache2/ssl:/etc/apache2/ssl
ports:
- 8080:8000
depends_on:
- db
- redis
Logs for db container:
Attaching to stage_db_0.10
stage_db_0.10 | 2022-04-25 16:44:16.734 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
stage_db_0.10 | 2022-04-25 16:44:16.734 UTC [1] LOG: listening on IPv6 address "::", port 5432
stage_db_0.10 | 2022-04-25 16:44:16.782 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
stage_db_0.10 | 2022-04-25 16:44:16.867 UTC [19] LOG: database system was shut down at 2022-04-25 16:42:54 UTC
stage_db_0.10 | 2022-04-25 16:44:16.941 UTC [1] LOG: database system is ready to accept connections
docker ps shows correct mapping for db container
0.0.0.0:5433->5432/tcp, :::5433->5432/tcp stage_db_0.10
Any help much appreciated!
Related
My system is Ubuntu 22.04.1 LTS.
I was going through the Django for professionals book and follow all line by line but somehow when i dockerize my django project with postgres it just doesn't run localhost properly, with sqlite all works fine.
Dockerfile
# Pull base image
FROM python:3.10.6-slim-bullseye
# Set environment variables
ENV PIP_DISABLE_PIP_VERSION_CHECK 1
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# Set work directory
WORKDIR /code
# Install dependencies
COPY ./requirements.txt .
RUN pip install -r requirements.txt
# Copy project
COPY . .
docker-compose.yml
version: "3.9"
services:
web:
build: .
command: python /code/manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- 8000:8000
depends_on:
- db
db:
image: postgres:13
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- "POSTGRES_HOST_AUTH_METHOD=trust"
volumes:
postgres_data:
django.settings.py
DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql",
"NAME": "postgres",
"USER": "postgres",
"PASSWORD": "postgres",
"HOST": "db", # set in docker-compose.yml
"PORT": 5432, # default postgres port
}
}
When I run docker-compose up that shows and seems like all must be fine, but somehow it just doesn't work:
Attaching to ch3-postgresql_web_1, ch3-postgresql_db_1
web_1 | Watching for file changes with StatReloader
web_1 | Performing system checks...
web_1 |
web_1 | System check identified no issues (0 silenced).
db_1 |
db_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1 |
db_1 | 2022-12-05 11:58:21.572 UTC [1] LOG: starting PostgreSQL 13.9 (Debian 13.9-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
db_1 | 2022-12-05 11:58:21.574 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2022-12-05 11:58:21.575 UTC [1] LOG: listening on IPv6 address "::", port 5432
db_1 | 2022-12-05 11:58:21.577 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2022-12-05 11:58:21.580 UTC [26] LOG: database system was shut down at 2022-12-05 11:58:17 UTC
db_1 | 2022-12-05 11:58:21.584 UTC [1] LOG: database system is ready to accept connections
And when I go to the url 127.0.0.1:8000:
This site can’t be reached
The webpage at http://127.0.0.1:8000/ might be temporarily down or it may have moved permanently to a new web address.
ERR_SOCKET_NOT_CONNECTED
If i wait some time and then check docker-compose logs, this messages appears
psycopg2.OperationalError: connection to server at "db" (172.21.0.2), port 5432 failed: Connection timed out
web_1 | Is the server running on that host and accepting TCP/IP connections?
I spend a lot of hours of searching for info about this problem but nothing did help. I try even to reinstall different versions of Postgres and docker.
Simply try this:
change this:
volumes:
- postgres_data:/var/lib/postgresql/data/
To this:
volumes:
- postgres_data:/var/lib/postgresql/data #removed forward slash from here.
Try above thing and see if it solves that problem.
Note: after making above changes don't forget to re-build docker-compsoe
You will need to add more environment variables to make the PostgreSQL work.
Please refer to https://hub.docker.com/_/postgres "Environment Variables"
The only variable required is POSTGRES_PASSWORD, the rest are optional.
db:
image: postgres:13
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=postgres
- POSTGRES_HOST_AUTH_METHOD=trust
And, you are encouraged to make your docker compose file more secure:
db:
image: postgres:13
volumes:
- postgres_data:/var/lib/postgresql/data/
env_file:
- .env.dev
.env.dev
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=postgres
POSTGRES_HOST_AUTH_METHOD=trust
I have a problem connecting my Postgres database to django project on docker container.
I have already tried every solution found on the internet and nothing seems to work.
I've already added host all all all md5 to my pg_hba.conf file, listen_addresses = '*' to postgresql.conf.
Here is log from lsof -i :5432:
postgres 19774 postgres 5u IPv4 2046377 0t0 TCP *:postgresql (LISTEN)
postgres 19774 postgres 6u IPv6 2046378 0t0 TCP *:postgresql (LISTEN)
I've tried to open 5432 port by using ufw allow 5432/tcp, it seems to be open.
My docker-compose.yml file:
version: '3.8'
services:
db:
image: postgres:12.0-alpine
environment:
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB}
build: .
container_name: db
volumes:
- postgres_data:/var/lib/postgresql/data/
networks:
- djangonetwork
ports:
- '5432'
web:
build:
context: .
dockerfile: Dockerfile
command: python manage.py runserver 127.0.0.1:8000
volumes:
- ./TwitterClone/:/usr/src/TwitterClone/
ports:
- 8000:8000
env_file:
- ./.env
depends_on:
- db
links:
- db:db
networks:
- djangonetwork
networks:
djangonetwork:
driver: bridge
volumes:
postgres_data:
settings.py:
try:
DATABASE_PASSWORD = os.environ.get('POSTGRES_PASSWORD')
DATABASE_USER = os.environ.get('POSTGRES_USER')
DATABASE_NAME = os.environ.get('POSTGRES_DB')
except:
print('Could not get environment variables')
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': DATABASE_NAME,
'USER': DATABASE_USER,
'PASSWORD': DATABASE_PASSWORD,
'HOST': '127.0.0.1',
'PORT': 5432,
}
}
No matter what I try to do, I always end up getting that error:
django.db.utils.OperationalError: could not connect to server: Connection refused
Is the server running on host "127.0.0.1" and accepting
TCP/IP connections on port 5432?
Thank you in advance for any help, anything could help. Just let me know if you need more information from me, I will gladly provide it.
EDIT:
I have edited my docker-compose file by deleting all networks related lines. Now I don't get the previous error, but it seems to have a problem with my PostgreSQL server running on port 5432:
ERROR: for db Cannot start service db: driver failed programming external connectivity on endpoint db (a1ed9a70eddb931e5d2a26522cb95a2873e019029420f8b994146e6bebe5ce09): Error starting userland proxy: listen tcp 0.0.0.0:5432: bind: address already in use
I am trying to have my database and my django rest api connect using docker-compose. However, my django container is giving me this error:
django.db.utils.OperationalError: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Cannot assign requested address
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
Postgres starts up normally, and I am able to access it locally using the terminal. I'm just not sure why my django container can't connect.
$ psql -h 0.0.0.0 -p 5432 -U bli1 -d redribbon
Password for user bli1:
psql (12.1)
Type "help" for help.
Within my settings.py for django this is my database values
DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql_psycopg2",
"NAME": "redribbon",
"USER": "bli1",
"PASSWORD": "password",
"HOST": "0.0.0.0",
"PORT": "5432",
}
}
docker-compose.yml
version: "3"
services:
postgres:
image: postgres:latest
restart: always
ports:
- "5432:5432"
volumes:
- ./pgdata:/var/lib/postgresql/data/
environment:
POSTGRES_DB: redribbon
POSTGRES_USER: bli1
POSTGRES_PASSWORD: password
api:
build:
dockerfile: Dockerfile.dev
context: ./redribbon-api
command: python manage.py runserver 0.0.0.0:8000
volumes:
- ./redribbon-api/api:/usr/src/api
depends_on:
- postgres
ports:
- "8000:8000"
updates:
Within the django container, I am able to ping postgres and got this as a response
# ping postgres
PING postgres (172.19.0.2) 56(84) bytes of data.
64 bytes from redribbon_postgres_1.redribbon_default (172.19.0.2): icmp_seq=1 ttl=64 time=0.175 ms
64 bytes from redribbon_postgres_1.redribbon_default (172.19.0.2): icmp_seq=2 ttl=64 time=0.096 ms
64 bytes from redribbon_postgres_1.redribbon_default (172.19.0.2): icmp_seq=3 ttl=64 time=0.097 ms
I even tried changing "HOST": "0.0.0.0", to "HOST": "postgres", in my settings.py but I am getting the same error as above
local.yml
version: '3'
volumes:
local_postgres_data: {}
local_postgres_data_backups: {}
services:
django:
build:
context: .
dockerfile: ./compose/local/django/Dockerfile
image: apkomat_local_django
container_name: django
depends_on:
- postgres
volumes:
- .:/app:z
env_file:
- ./.envs/.local/.django
- ./.envs/.local/.postgres
ports:
- "8000:8000"
command: /start
postgres:
build:
context: .
dockerfile: ./compose/production/postgres/Dockerfile
image: apkomat_production_postgres
container_name: postgres
volumes:
- local_postgres_data:/var/lib/postgresql/data:Z
- local_postgres_data_backups:/backups:z
env_file:
- ./.envs/.local/.postgres
This is my local docker-compose configuration.
depends_on:
- postgres
This setting assures your django service won't start before postgres is ready to accept connections. To get your application running with these configuration use
docker-compose -f local.yml up
I have a dockerized Django application using Postgresql. Everything runs find until I add volumes to the docker-compose file so I can have persistent data storage for the DB.
docker-compose.yml :
version: "3"
services:
app:
build:
context: .
ports:
- "8000:8000"
volumes:
- ./app:/app
command: >
sh -c "python manage.py wait_for_db &&
python manage.py migrate &&
python manage.py runserver 0.0.0.0:8000"
environment:
- DB_HOST=db
- DB_NAME=app
- DB_USER=postgres
- DB_PASS=dbpassword
depends_on:
- db
db:
image: postgres:10-alpine
volumes:
- ./postgres-data:/var/lib/postgresql/data
environment:
- POSTGRES_DB=app
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=dbpassword
ports:
- "5432:5432"
Error Message here:
django.db.utils.OperationalError: could not connect to server: Connection refused
app_1 | Is the server running on host "db" (172.26.0.2) and accepting
app_1 | TCP/IP connections on port 5432?
app_1 |
recipe-api_app_1 exited with code 1
db_1 | 2019-08-31 22:52:35.969 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2019-08-31 22:52:35.969 UTC [1] LOG: listening on IPv6 address "::", port 5432
db_1 | 2019-08-31 22:52:35.972 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2019-08-31 22:52:36.042 UTC [18] LOG: database system was shut down at 2019-08-31 22:41:04 UTC
db_1 | 2019-08-31 22:52:36.055 UTC [1] LOG: database system is ready to accept connections
Once you add the volumes for the docker-compose I keep getting this error and without volumes everything runs fine.
EDIT : wait_for_db command :
def handle(self, *args, **options):
"""Handle the command"""
self.stdout.write('Waiting for database...')
db_conn = None
while not db_conn:
try:
db_conn = connections['default']
except OperationalError:
self.stdout.write('Database unavailable, waiting 1 second...')
time.sleep(1)
self.stdout.write(self.style.SUCCESS('Database available!'))
I am running a PyCharm 2018.2 version.
I have my django code running on a docker instance and the Postgres running on another docker instance.
My docker-compose.yml file is here
version: "2.3"
services:
cv_db:
container_name: cv_db
image: postgres:10.3-alpine
restart: always
ports:
- "9081:5432"
environment:
POSTGRES_USER: root
POSTGRES_PASSWORD: ****
POSTGRES_DB: cv
volumes:
- cv_db:/var/lib/postgresql/data
expose:
- "9081:5432"
cv_redis:
container_name: cv_redis
image: redis:4.0.5-alpine
restart: always
ports:
- "6379:6379"
cv:
container_name: cv
image: cv
restart: always
depends_on:
- cv_redis
- cv_db
ports:
- "9080:9080"
- "8080:8080"
expose:
- "9080"
build:
context: .
args:
http_proxy:
https_proxy:
no_proxy:
TF_ANNOTATION: "no"
USER: "django"
DJANGO_CONFIGURATION: "production"
WITH_TESTS: "no"
environment:
DJANGO_MODWSGI_EXTRA_ARGS: ""
DJANGO_LOG_SERVER_URL: ""
volumes:
- cvat_data:/home/django/data
- cvat_keys:/home/django/keys
- cvat_logs:/home/django/logs
- ./share:/home/django/share
volumes:
cv_db:
cv_data:
cv_keys:
cv_logs:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
# 'HOST': 'cv_db',
'HOST': 'localhost',
'PORT': '9801',
'NAME': 'cv_dev',
'USER': 'root',
'PASSWORD': os.getenv('POSTGRES_PASSWORD', '****'),
}
I am using the django debugger:
Edit Configuration Settings are below:
Host: 0.0.0.0 Port: 9080
Run Browser: http://0.0.0.0:9080/
WHen I click on debug, I get the error saying it cant connect to Database:
django.db.utils.OperationalError: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 9081?
could not connect to server: Cannot assign requested address
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 9081?
what might be going wrong?