I m creating a docker-compose config for an django app, the Dockerfile builds successfully but when I compose them up, django return an issue -- cannot connect to postgres.
I run docker-compose run web bash, found redis and posgres both cannot be connected.
My docker-compose.yml file
web:
build: .
ports:
- "8000:8000"
environment:
- 'DATABASE_HOST=db'
- 'DATABASE_NAME=mydb'
- 'DATABASE_USER=root'
- 'DATABASE_PASSWORD=root'
links:
- db
db:
image: postgres:9.1
when running sudo docker-compose up i got the following error.
web_1 | File "/usr/local/lib/python2.7/site packages/django/db/backends/postgresql/base.py", line 175, in get_new_connection
web_1 | connection = Database.connect(**conn_params)
web_1 | File "/usr/local/lib/python2.7/site-packages/psycopg2/__init__.py", line 164, in connect
web_1 | conn = _connect(dsn, connection_factory=connection_factory, async=async)
web_1 | django.db.utils.OperationalError: could not connect to server: Connection refused
web_1 | Is the server running on host "localhost" (::1) and accepting
web_1 | TCP/IP connections on port 5432?
web_1 | could not connect to server: Connection refused
web_1 | Is the server running on host "localhost" (127.0.0.1) and accepting
web_1 | TCP/IP connections on port 5432?
I also built a clustering with docker-compose, it probably will help you and answer your problem (here is the repo). You can see the docker-compose.yml file, and the django settings file (I marked the lines you need).
You can also clone this repo and get django, angular2, postgresql, and nginx containers, all link together already.
You are linking your web container with the postgres container but you are not defining database name, password and user.
web:
build: .
ports:
- "8000:8000"
links:
- db
db:
restart: always
image: postgres:9.1
ports:
- "5432:5432"
volumes:
- pgvolume:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=root
- POSTGRES_DB= aiotadb
- POSTGRES_USER=root
data:
restart: always
image: postgres:9.1
volumes:
- /var/lib/postgresql
command: tail -f /dev/null
Also, if you already define your database options in your settings file, you don't need to declare it as env variables in web container.
version: '3'
services:
basicproject:
build: .
container_name: basicproject-container
depends_on:
- postgres
ports:
- "8000:8000"
postgres:
image: postgres:9.4
ports:
- "5432"
environment:
- POSTGRES_USER=test
- POSTGRES_PASSWORD=testing
- POSTGRES_DB=test_db
Add dependency in your 'web' service like below:
depends_on:
- db
Related
Yes i know this question has been asked, and the usual error is making the DB_HOST = docker service name, in my case db.
Here's my docker-compose.yml:
version: "3"
services:
db:
image: postgres:latest
restart: always
ports:
- "54320:5432"
expose:
- "54320"
volumes:
- ./database-data:/var/lib/postgresql/data/ # persist data even if container shuts down
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
# POSTGRES_HOST_AUTH_METHOD: trust
django:
build: ./api
restart: always
command: ["python", "manage.py", "runserver", "0.0.0.0:8000"]
volumes:
- ./api:/app/api
ports:
- "8000:8000"
depends_on:
- db
raddl_admin:
build: ./frontend/raddl_admin
stdin_open: true # docker run -i
tty: true # docker run -t
command: ["npm", "start"]
ports:
- "3000:3000"
volumes:
- ./frontend/raddl_admin:/app/frontend/raddl_admin
- /app/frontend/raddl_admin/node_modules
volumes:
database-data: # named volumes can be managed easier using docker-compose
Logging the django service gives:
django.db.utils.OperationalError: could not connect to server: Connection refused
Is the server running on host "db" (172.18.0.2) and accepting
TCP/IP connections on port 54320?
So it's clearly trying to connect to db on that port and my environment vars are set for the django instance.
I can even ping my db service from my django service:
docker exec -it 1f764333ace2 bash | ping db
PING db (23.221.222.250): 56 data bytes
64 bytes from 23.221.222.250: icmp_seq=0 ttl=50 time=42.055 ms
64 bytes from 23.221.222.250: icmp_seq=1 ttl=50 time=42.382 ms
6
Why can't I connect to the db service from Django?
Ah, so my issue was with the port number. For some reason, keeping the db exposed port to "5432" resolved this. I thought that it would have conflicted with my host's localhost instance of postgres. I guess not.
My system is Ubuntu 22.04.1 LTS.
I was going through the Django for professionals book and follow all line by line but somehow when i dockerize my django project with postgres it just doesn't run localhost properly, with sqlite all works fine.
Dockerfile
# Pull base image
FROM python:3.10.6-slim-bullseye
# Set environment variables
ENV PIP_DISABLE_PIP_VERSION_CHECK 1
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# Set work directory
WORKDIR /code
# Install dependencies
COPY ./requirements.txt .
RUN pip install -r requirements.txt
# Copy project
COPY . .
docker-compose.yml
version: "3.9"
services:
web:
build: .
command: python /code/manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- 8000:8000
depends_on:
- db
db:
image: postgres:13
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- "POSTGRES_HOST_AUTH_METHOD=trust"
volumes:
postgres_data:
django.settings.py
DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql",
"NAME": "postgres",
"USER": "postgres",
"PASSWORD": "postgres",
"HOST": "db", # set in docker-compose.yml
"PORT": 5432, # default postgres port
}
}
When I run docker-compose up that shows and seems like all must be fine, but somehow it just doesn't work:
Attaching to ch3-postgresql_web_1, ch3-postgresql_db_1
web_1 | Watching for file changes with StatReloader
web_1 | Performing system checks...
web_1 |
web_1 | System check identified no issues (0 silenced).
db_1 |
db_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1 |
db_1 | 2022-12-05 11:58:21.572 UTC [1] LOG: starting PostgreSQL 13.9 (Debian 13.9-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
db_1 | 2022-12-05 11:58:21.574 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2022-12-05 11:58:21.575 UTC [1] LOG: listening on IPv6 address "::", port 5432
db_1 | 2022-12-05 11:58:21.577 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2022-12-05 11:58:21.580 UTC [26] LOG: database system was shut down at 2022-12-05 11:58:17 UTC
db_1 | 2022-12-05 11:58:21.584 UTC [1] LOG: database system is ready to accept connections
And when I go to the url 127.0.0.1:8000:
This site can’t be reached
The webpage at http://127.0.0.1:8000/ might be temporarily down or it may have moved permanently to a new web address.
ERR_SOCKET_NOT_CONNECTED
If i wait some time and then check docker-compose logs, this messages appears
psycopg2.OperationalError: connection to server at "db" (172.21.0.2), port 5432 failed: Connection timed out
web_1 | Is the server running on that host and accepting TCP/IP connections?
I spend a lot of hours of searching for info about this problem but nothing did help. I try even to reinstall different versions of Postgres and docker.
Simply try this:
change this:
volumes:
- postgres_data:/var/lib/postgresql/data/
To this:
volumes:
- postgres_data:/var/lib/postgresql/data #removed forward slash from here.
Try above thing and see if it solves that problem.
Note: after making above changes don't forget to re-build docker-compsoe
You will need to add more environment variables to make the PostgreSQL work.
Please refer to https://hub.docker.com/_/postgres "Environment Variables"
The only variable required is POSTGRES_PASSWORD, the rest are optional.
db:
image: postgres:13
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=postgres
- POSTGRES_HOST_AUTH_METHOD=trust
And, you are encouraged to make your docker compose file more secure:
db:
image: postgres:13
volumes:
- postgres_data:/var/lib/postgresql/data/
env_file:
- .env.dev
.env.dev
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=postgres
POSTGRES_HOST_AUTH_METHOD=trust
I'm currently trying to make a connection between two of my docker containers (The requesting container running Gunicorn/Django and the api container running kroki).
I've had a look at other answers but seem to be coming up blank with a solution, so was hoping for a little poke in the right direction.
Docker-compose:
version: '3.8'
services:
app:
build:
context: ./my_app
dockerfile: Dockerfile.prod
command: gunicorn my_app.wsgi:application --bind 0.0.0.0:8000 --access-logfile -
volumes:
- static_volume:/home/app/web/staticfiles
expose:
- 8000
environment:
- DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1 kroki
env_file:
- ./.env.prod
depends_on:
- db
db:
image: postgres:13.0-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
env_file:
- ./.env.prod.db
nginx:
build: ./nginx
volumes:
- static_volume:/home/app/web/staticfiles
ports:
- 1337:80
depends_on:
- app
kroki:
image: yuzutech/kroki
ports:
- 7331:8000
volumes:
postgres_data:
static_volume:
settings.py
ALLOWED_HOSTS = os.environ.get("DJANGO_ALLOWED_HOSTS").split(" ")
Requesting code in django
url = 'http://kroki:7331/bytefield/svg/' + base64_var
try:
response = requests.get(url)
return response.text
except ConnectionError as e:
print("Connection to bytefield module, unavailable")
return None
I'm able to access both containers via my browser successfully, however initiating the code for an internal call between the two throws out
requests.exceptions.ConnectionError: HTTPConnectionPool(host='kroki', port=7331): Max retries exceeded with url: /bytefield/svg/<API_URL_HERE> (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f286f5ecaf0>: Failed to establish a new connection: [Errno 111] Connection refused'))
I've had a go accessing the url via localhost:7331 & 127.0.0.1:7331 however neither seem to help at all
When you access to other containers within the same network you don't have to access to them using the exposed port in the host but using instead the actual port where the application within the container is listening.
I made a really simple example so you can understand what's your problem:
version: '3.8'
services:
app:
image: busybox
entrypoint: tail -f /dev/null
kroki:
image: yuzutech/kroki
ports:
- 7331:8000
From Host
❯ curl -s -o /dev/null -w "%{http_code}" localhost:7331
200
From App
/ # wget kroki:7331
Connecting to kroki:7331 (172.18.0.3:7331)
wget: can't connect to remote host (172.18.0.3): Connection refused
/ # wget kroki:8000
Connecting to kroki:8000 (172.18.0.3:8000)
saving to 'index.html'
index.html 100% |************************************************************************| 51087 0:00:00 ETA
'index.html' saved
I'm trying to connect my application to my mysql database which I've got up and running in a docker-compose file. I'm using flask and trying to connect using DBUtils
I keep getting the error message described in my title:
(pymysql.err.OperationalError: (2003, "Can't connect to MySQL server on 'db' ([Errno 8] nodename nor servname provided, or not known)"))
I've tried using the IPAddresses of my docker instances as well as several other solutions in similar problems discussed here on StackOverflow:
Docker-Compose can't connect to MySQL
Connecting to MySQL from Flask Application using docker-compose.
however, the offered solutions don't seem to be working for me.
my docker-compose file looks as follows:
version: '3.3'
services:
db:
image: mysql:8.0
container_name: mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: 'myPassword'
MYSQL_DATABASE: 'databaseName'
volumes:
- .:/dockerFiles
ports:
- "3306:3306"
expose:
- "3306"
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: phpmyadmin
restart: always
ports:
- "8080:80"
volumes:
- /sessions
and my connect string looks as follows:
def connect_db():
# Connects to the database and takes care of the connection
return PersistentDB(
creator=pymysql, host='db',
user='root', password='myPassword', database='databaseName', port=3306,
autocommit=True, charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor)
The error [Errno 8] nodename nor servname provided, or not known states that it can't find the database server. Thus, host='db' will only work if your Python code is inside a docker in the same network of the database service db. Try adding a service for the Python code in your docker-compose:
version: '3.3'
services:
db:
image: mysql:8.0
container_name: mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: 'myPassword'
MYSQL_DATABASE: 'databaseName'
volumes:
- .:/dockerFiles
ports:
- "3306:3306"
expose:
- "3306"
web:
build: . # where is your dockerfile
command: sh -c 'python app.py' # this should be the command to start your application
ports:
- "8082:8082"
volumes:
- .:/code # it depends on the WORKDIR of your dockerfile
links:
- db
I am trying to have my database and my django rest api connect using docker-compose. However, my django container is giving me this error:
django.db.utils.OperationalError: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Cannot assign requested address
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
Postgres starts up normally, and I am able to access it locally using the terminal. I'm just not sure why my django container can't connect.
$ psql -h 0.0.0.0 -p 5432 -U bli1 -d redribbon
Password for user bli1:
psql (12.1)
Type "help" for help.
Within my settings.py for django this is my database values
DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql_psycopg2",
"NAME": "redribbon",
"USER": "bli1",
"PASSWORD": "password",
"HOST": "0.0.0.0",
"PORT": "5432",
}
}
docker-compose.yml
version: "3"
services:
postgres:
image: postgres:latest
restart: always
ports:
- "5432:5432"
volumes:
- ./pgdata:/var/lib/postgresql/data/
environment:
POSTGRES_DB: redribbon
POSTGRES_USER: bli1
POSTGRES_PASSWORD: password
api:
build:
dockerfile: Dockerfile.dev
context: ./redribbon-api
command: python manage.py runserver 0.0.0.0:8000
volumes:
- ./redribbon-api/api:/usr/src/api
depends_on:
- postgres
ports:
- "8000:8000"
updates:
Within the django container, I am able to ping postgres and got this as a response
# ping postgres
PING postgres (172.19.0.2) 56(84) bytes of data.
64 bytes from redribbon_postgres_1.redribbon_default (172.19.0.2): icmp_seq=1 ttl=64 time=0.175 ms
64 bytes from redribbon_postgres_1.redribbon_default (172.19.0.2): icmp_seq=2 ttl=64 time=0.096 ms
64 bytes from redribbon_postgres_1.redribbon_default (172.19.0.2): icmp_seq=3 ttl=64 time=0.097 ms
I even tried changing "HOST": "0.0.0.0", to "HOST": "postgres", in my settings.py but I am getting the same error as above
local.yml
version: '3'
volumes:
local_postgres_data: {}
local_postgres_data_backups: {}
services:
django:
build:
context: .
dockerfile: ./compose/local/django/Dockerfile
image: apkomat_local_django
container_name: django
depends_on:
- postgres
volumes:
- .:/app:z
env_file:
- ./.envs/.local/.django
- ./.envs/.local/.postgres
ports:
- "8000:8000"
command: /start
postgres:
build:
context: .
dockerfile: ./compose/production/postgres/Dockerfile
image: apkomat_production_postgres
container_name: postgres
volumes:
- local_postgres_data:/var/lib/postgresql/data:Z
- local_postgres_data_backups:/backups:z
env_file:
- ./.envs/.local/.postgres
This is my local docker-compose configuration.
depends_on:
- postgres
This setting assures your django service won't start before postgres is ready to accept connections. To get your application running with these configuration use
docker-compose -f local.yml up