I've been attempting to make a docker-compose file work, and so far it's been closer and closer, but not quite there. Maybe asking the questions will be useful to someone else in the future. Anyway, the previous thread was here:
Docker-Compose Postgresql Init Entrypoint
From which I learned,
between runs a docker-compose file should have docker compose volumes --down and
if the POSTGRES_DATABASE isn't specified, then the POSTGRES_NAME will replicate to the POSTGRES_DATABASE.
another suggestion from Discord has been to rewrite some of the volume signature to include the file path to init.sql as in docker-entrypoint-initdb.d/init.sql
So if I docker-compose down --volume and then sudo docker-compose build && docker-compose up with
version: "3.9"
services:
db:
restart: always
image: postgres
volumes:
# - ./data/db:/var/lib/postgresql/data
- ./init.sql:/docker-entrypoint-initdb.d/init.db
environment:
- POSTGRES_NAME=dev-postgres
- POSTGRES_USER=pixel
- POSTGRES_DATABASE=lightchan
- POSTGRES_PASSWORD=stardust
web:
build: .
restart: always
command: sh -c "./waitfor.sh db:5432 -- python3 manage.py runserver"
volumes:
- .:/code
ports:
- "8001:8001"
environment:
- POSTGRES_NAME=dev-postgres
- POSTGRES_USER=pixel
- POSTGRES_DATABASE=lightchan
- POSTGRES_PASSWORD=stardust
depends_on:
- db
Then the following is output:
Attaching to lightchan-db-1, lightchan-web-1
lightchan-db-1 | The files belonging to this database system will be owned by user "postgres".
lightchan-db-1 | This user must also own the server process.
lightchan-db-1 |
lightchan-db-1 | The database cluster will be initialized with locale "en_US.utf8".
lightchan-db-1 | The default database encoding has accordingly been set to "UTF8".
lightchan-db-1 | The default text search configuration will be set to "english".
lightchan-db-1 |
lightchan-db-1 | Data page checksums are disabled.
lightchan-db-1 |
lightchan-db-1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
lightchan-db-1 | creating subdirectories ... ok
lightchan-db-1 | selecting dynamic shared memory implementation ... posix
lightchan-db-1 | selecting default max_connections ... 100
lightchan-db-1 | selecting default shared_buffers ... 128MB
lightchan-db-1 | selecting default time zone ... Etc/UTC
lightchan-db-1 | creating configuration files ... ok
lightchan-db-1 | running bootstrap script ... ok
lightchan-db-1 | performing post-bootstrap initialization ... ok
lightchan-db-1 | syncing data to disk ... ok
lightchan-db-1 |
lightchan-db-1 | initdb: warning: enabling "trust" authentication for local connections
lightchan-db-1 | You can change this by editing pg_hba.conf or using the option -A, or
lightchan-db-1 | --auth-local and --auth-host, the next time you run initdb.
lightchan-db-1 |
lightchan-db-1 | Success. You can now start the database server using:
lightchan-db-1 |
lightchan-db-1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
lightchan-db-1 |
lightchan-db-1 | waiting for server to start....2022-03-02 02:08:59.604 UTC [49] LOG: starting PostgreSQL 14.2 (Debian 14.2-1.pgdg110+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
lightchan-db-1 | 2022-03-02 02:08:59.605 UTC [49] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
lightchan-db-1 | 2022-03-02 02:08:59.611 UTC [50] LOG: database system was shut down at 2022-03-02 02:08:59 UTC
lightchan-db-1 | 2022-03-02 02:08:59.617 UTC [49] LOG: database system is ready to accept connections
lightchan-db-1 | done
lightchan-db-1 | server started
lightchan-db-1 | CREATE DATABASE
lightchan-db-1 |
lightchan-db-1 |
lightchan-db-1 | /usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/init.sql
lightchan-db-1 | psql:/docker-entrypoint-initdb.d/init.sql: error: could not read from input file: Is a directory
lightchan-db-1 exited with code 1
lightchan-db-1 | 2022-03-02 02:09:00.594 UTC [28] LOG: database system was not properly shut down; automatic recovery in progress
lightchan-db-1 | 2022-03-02 02:09:00.596 UTC [28] LOG: redo starts at 0/16FB5C8
lightchan-db-1 | 2022-03-02 02:09:00.596 UTC [28] LOG: invalid record length at 0/16FB6C0: wanted 24, got 0
lightchan-db-1 | 2022-03-02 02:09:00.596 UTC [28] LOG: redo done at 0/16FB678 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
lightchan-db-1 | 2022-03-02 02:09:00.604 UTC [1] LOG: database system is ready to accept connections
lightchan-web-1 | Watching for file changes with StatReloader
lightchan-db-1 | 2022-03-02 02:09:01.480 UTC [36] FATAL: database "lightchan" does not exist
So the database can't be seen on the web side but PSQL says it created it. If I uncomment out the data volume on PSQL then I get:
Attaching to lightchan-db-1, lightchan-web-1
lightchan-db-1 |
lightchan-db-1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
lightchan-db-1 |
lightchan-db-1 | 2022-03-02 02:11:29.346 UTC [1] LOG: starting PostgreSQL 14.2 (Debian 14.2-1.pgdg110+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
lightchan-db-1 | 2022-03-02 02:11:29.346 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
lightchan-db-1 | 2022-03-02 02:11:29.346 UTC [1] LOG: listening on IPv6 address "::", port 5432
lightchan-db-1 | 2022-03-02 02:11:29.359 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
lightchan-db-1 | 2022-03-02 02:11:29.420 UTC [28] LOG: database system was interrupted; last known up at 2022-03-02 02:11:06 UTC
lightchan-web-1 | Watching for file changes with StatReloader
lightchan-db-1 | 2022-03-02 02:11:30.835 UTC [30] FATAL: the database system is starting up
lightchan-web-1 | Exception in thread django-main-thread:
lightchan-web-1 | Traceback (most recent call last):
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 230, in ensure_connection
lightchan-web-1 | self.connect()
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 25, in inner
lightchan-web-1 | return func(*args, **kwargs)
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 211, in connect
lightchan-web-1 | self.connection = self.get_new_connection(conn_params)
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 25, in inner
lightchan-web-1 | return func(*args, **kwargs)
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 199, in get_new_connection
lightchan-web-1 | connection = Database.connect(**conn_params)
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 122, in connect
lightchan-web-1 | conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
lightchan-web-1 | psycopg2.OperationalError: connection to server at "db" (192.168.96.2), port 5432 failed: FATAL: the database system is starting up
lightchan-web-1 |
lightchan-web-1 |
lightchan-web-1 | The above exception was the direct cause of the following exception:
lightchan-web-1 |
lightchan-web-1 | Traceback (most recent call last):
lightchan-web-1 | File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
lightchan-web-1 | self.run()
lightchan-web-1 | File "/usr/local/lib/python3.8/threading.py", line 870, in run
lightchan-web-1 | self._target(*self._args, **self._kwargs)
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/utils/autoreload.py", line 64, in wrapper
lightchan-web-1 | fn(*args, **kwargs)
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/core/management/commands/runserver.py", line 127, in inner_run
lightchan-web-1 | self.check_migrations()
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/core/management/base.py", line 505, in check_migrations
lightchan-web-1 | executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/db/migrations/executor.py", line 18, in __init__
lightchan-web-1 | self.loader = MigrationLoader(self.connection)
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/db/migrations/loader.py", line 53, in __init__
lightchan-web-1 | self.build_graph()
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/db/migrations/loader.py", line 223, in build_graph
lightchan-web-1 | self.applied_migrations = recorder.applied_migrations()
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/db/migrations/recorder.py", line 77, in applied_migrations
lightchan-web-1 | if self.has_table():
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/db/migrations/recorder.py", line 55, in has_table
lightchan-web-1 | with self.connection.cursor() as cursor:
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 25, in inner
lightchan-web-1 | return func(*args, **kwargs)
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 270, in cursor
lightchan-web-1 | return self._cursor()
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 246, in _cursor
lightchan-web-1 | self.ensure_connection()
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 25, in inner
lightchan-web-1 | return func(*args, **kwargs)
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 230, in ensure_connection
lightchan-web-1 | self.connect()
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
lightchan-web-1 | raise dj_exc_value.with_traceback(traceback) from exc_value
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 230, in ensure_connection
lightchan-web-1 | self.connect()
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 25, in inner
lightchan-web-1 | return func(*args, **kwargs)
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 211, in connect
lightchan-web-1 | self.connection = self.get_new_connection(conn_params)
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 25, in inner
lightchan-web-1 | return func(*args, **kwargs)
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 199, in get_new_connection
lightchan-web-1 | connection = Database.connect(**conn_params)
lightchan-web-1 | File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 122, in connect
lightchan-web-1 | conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
lightchan-web-1 | django.db.utils.OperationalError: connection to server at "db" (192.168.96.2), port 5432 failed: FATAL: the database system is starting up
lightchan-web-1 |
lightchan-db-1 | 2022-03-02 02:11:32.337 UTC [28] LOG: database system was not properly shut down; automatic recovery in progress
lightchan-db-1 | 2022-03-02 02:11:32.340 UTC [28] LOG: invalid record length at 0/16FF138: wanted 24, got 0
lightchan-db-1 | 2022-03-02 02:11:32.340 UTC [28] LOG: redo is not required
lightchan-db-1 | 2022-03-02 02:11:32.368 UTC [1] LOG: database system is ready to accept connections
Again I used docker-compose volumes --down to clean the errors from the previous thread, but apparently this may cause either the waitfor.sh script to run improperly when the data volumes are included or there's some other error for some reason.
Any ideas anyone? I'm out of debugging options that I know of.
Try opening ports in your db services:
db:
restart: always
image: postgres
ports:
- 5432:5432
Related
This question already has answers here:
Docker Compose wait for container X before starting Y
(20 answers)
Closed last month.
Here's the current docker-compose.yml
version: '3'
services:
db:
image: postgres
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=my_app
ports:
- '5432:5432'
web:
build: .
image: my-app
ports:
- "8000:8000"
depends_on:
- db
command: sh -c "python manage.py migrate &&
python manage.py runserver 0.0.0.0:8000"
volumes:
- .:/code
environment:
- DB_USER=postgres
- DB_PASSWORD=postgres
- DB_HOST=db
- DB_NAME=my_app
When I run the app for the first time, this happens:
% docker compose build && docker compose up
[+] Building 2.5s (10/10) FINISHED
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 189B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 150B 0.0s
=> [internal] load metadata for docker.io/library/python:3.11.1-bullseye 2.0s
=> [1/5] FROM docker.io/library/python:3.11.1-bullseye#sha256:cc4910af48 0.0s
=> [internal] load build context 0.2s
=> => transferring context: 1.21MB 0.2s
=> CACHED [2/5] COPY requirements.txt requirements.txt 0.0s
=> CACHED [3/5] RUN pip install -r requirements.txt 0.0s
=> CACHED [4/5] COPY . /app 0.0s
=> CACHED [5/5] WORKDIR /app 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:d7b4a64b01b9de03dec4a0732eaf975b7bc68f1daefb4 0.0s
=> => naming to docker.io/library/my-app 0.0s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
[+] Running 3/3
⠿ Network my_app_default Created 0.2s
⠿ Container my_app-db-1 Created 0.1s
⠿ Container my_app-web-1 Created 0.2s
Attaching to my_app-db-1, my_app-web-1
my_app-db-1 | The files belonging to this database system will be owned by user "postgres".
my_app-db-1 | This user must also own the server process.
my_app-db-1 |
my_app-db-1 | The database cluster will be initialized with locale "en_US.utf8".
my_app-db-1 | The default database encoding has accordingly been set to "UTF8".
my_app-db-1 | The default text search configuration will be set to "english".
my_app-db-1 |
my_app-db-1 | Data page checksums are disabled.
my_app-db-1 |
my_app-db-1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
my_app-db-1 | creating subdirectories ... ok
my_app-db-1 | selecting dynamic shared memory implementation ... posix
my_app-db-1 | selecting default max_connections ... 100
my_app-db-1 | selecting default shared_buffers ... 128MB
my_app-db-1 | selecting default time zone ... Etc/UTC
my_app-db-1 | creating configuration files ... ok
my_app-db-1 | running bootstrap script ... ok
my_app-db-1 | performing post-bootstrap initialization ... ok
my_app-db-1 | syncing data to disk ... ok
my_app-db-1 |
my_app-db-1 |
my_app-db-1 | Success. You can now start the database server using:
my_app-db-1 |
my_app-db-1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
my_app-db-1 |
my_app-db-1 | initdb: warning: enabling "trust" authentication for local connections
my_app-db-1 | initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.
my_app-web-1 | Traceback (most recent call last):
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 282, in ensure_connection
my_app-web-1 | self.connect()
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/utils/asyncio.py", line 26, in inner
my_app-web-1 | return func(*args, **kwargs)
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 263, in connect
my_app-web-1 | self.connection = self.get_new_connection(conn_params)
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/utils/asyncio.py", line 26, in inner
my_app-web-1 | return func(*args, **kwargs)
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/db/backends/postgresql/base.py", line 215, in get_new_connection
my_app-web-1 | connection = Database.connect(**conn_params)
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/psycopg2/__init__.py", line 122, in connect
my_app-web-1 | conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
my_app-web-1 | psycopg2.OperationalError: could not connect to server: Connection refused
my_app-web-1 | Is the server running on host "db" (172.24.0.2) and accepting
my_app-web-1 | TCP/IP connections on port 5432?
my_app-web-1 |
my_app-web-1 |
my_app-web-1 | The above exception was the direct cause of the following exception:
my_app-web-1 |
my_app-web-1 | Traceback (most recent call last):
my_app-web-1 | File "/app/manage.py", line 22, in <module>
my_app-web-1 | main()
my_app-web-1 | File "/app/manage.py", line 18, in main
my_app-web-1 | execute_from_command_line(sys.argv)
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
my_app-web-1 | utility.execute()
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 440, in execute
my_app-web-1 | self.fetch_command(subcommand).run_from_argv(self.argv)
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 402, in run_from_argv
my_app-web-1 | self.execute(*args, **cmd_options)
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 448, in execute
my_app-web-1 | output = self.handle(*args, **options)
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 96, in wrapped
my_app-web-1 | res = handle_func(*args, **kwargs)
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/core/management/commands/migrate.py", line 114, in handle
my_app-web-1 | executor = MigrationExecutor(connection, self.migration_progress_callback)
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/db/migrations/executor.py", line 18, in __init__
my_app-web-1 | self.loader = MigrationLoader(self.connection)
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/db/migrations/loader.py", line 58, in __init__
my_app-web-1 | self.build_graph()
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/db/migrations/loader.py", line 235, in build_graph
my_app-web-1 | self.applied_migrations = recorder.applied_migrations()
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/db/migrations/recorder.py", line 81, in applied_migrations
my_app-web-1 | if self.has_table():
my_app-web-1 | ^^^^^^^^^^^^^^^^
my_app-db-1 | waiting for server to start....2023-01-08 12:43:30.440 UTC [47] LOG: starting PostgreSQL 15.1 (Debian 15.1-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/db/migrations/recorder.py", line 57, in has_table
my_app-web-1 | with self.connection.cursor() as cursor:
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/utils/asyncio.py", line 26, in inner
my_app-web-1 | return func(*args, **kwargs)
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 323, in cursor
my_app-web-1 | return self._cursor()
my_app-web-1 | ^^^^^^^^^^^^^^
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 299, in _cursor
my_app-web-1 | self.ensure_connection()
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/utils/asyncio.py", line 26, in inner
my_app-web-1 | return func(*args, **kwargs)
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 281, in ensure_connection
my_app-web-1 | with self.wrap_database_errors:
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
my_app-web-1 | raise dj_exc_value.with_traceback(traceback) from exc_value
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 282, in ensure_connection
my_app-web-1 | self.connect()
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/utils/asyncio.py", line 26, in inner
my_app-web-1 | return func(*args, **kwargs)
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/base.py", line 263, in connect
my_app-web-1 | self.connection = self.get_new_connection(conn_params)
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/utils/asyncio.py", line 26, in inner
my_app-web-1 | return func(*args, **kwargs)
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/django/db/backends/postgresql/base.py", line 215, in get_new_connection
my_app-web-1 | connection = Database.connect(**conn_params)
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
my_app-web-1 | File "/usr/local/lib/python3.11/site-packages/psycopg2/__init__.py", line 122, in connect
my_app-web-1 | conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
my_app-web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
my_app-db-1 | 2023-01-08 12:43:30.457 UTC [47] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
my_app-web-1 | django.db.utils.OperationalError: could not connect to server: Connection refused
my_app-web-1 | Is the server running on host "db" (172.24.0.2) and accepting
my_app-web-1 | TCP/IP connections on port 5432?
my_app-web-1 |
my_app-db-1 | 2023-01-08 12:43:30.476 UTC [50] LOG: database system was shut down at 2023-01-08 12:43:29 UTC
my_app-db-1 | 2023-01-08 12:43:30.490 UTC [47] LOG: database system is ready to accept connections
my_app-db-1 | done
my_app-db-1 | server started
my_app-db-1 | CREATE DATABASE
my_app-db-1 |
my_app-db-1 |
my_app-db-1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
my_app-db-1 |
my_app-web-1 exited with code 1
my_app-db-1 | 2023-01-08 12:43:31.286 UTC [47] LOG: received fast shutdown request
my_app-db-1 | waiting for server to shut down....2023-01-08 12:43:31.288 UTC [47] LOG: aborting any active transactions
my_app-db-1 | 2023-01-08 12:43:31.319 UTC [47] LOG: background worker "logical replication launcher" (PID 53) exited with exit code 1
my_app-db-1 | 2023-01-08 12:43:31.320 UTC [48] LOG: shutting down
my_app-db-1 | 2023-01-08 12:43:31.326 UTC [48] LOG: checkpoint starting: shutdown immediate
my_app-db-1 | 2023-01-08 12:43:31.469 UTC [48] LOG: checkpoint complete: wrote 918 buffers (5.6%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.054 s, sync=0.076 s, total=0.149 s; sync files=250, longest=0.023 s, average=0.001 s; distance=4217 kB, estimate=4217 kB
my_app-db-1 | 2023-01-08 12:43:31.488 UTC [47] LOG: database system is shut down
my_app-db-1 | done
my_app-db-1 | server stopped
my_app-db-1 |
my_app-db-1 | PostgreSQL init process complete; ready for start up.
my_app-db-1 |
my_app-db-1 | 2023-01-08 12:43:31.692 UTC [1] LOG: starting PostgreSQL 15.1 (Debian 15.1-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
my_app-db-1 | 2023-01-08 12:43:31.695 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
my_app-db-1 | 2023-01-08 12:43:31.695 UTC [1] LOG: listening on IPv6 address "::", port 5432
my_app-db-1 | 2023-01-08 12:43:31.701 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
my_app-db-1 | 2023-01-08 12:43:31.708 UTC [63] LOG: database system was shut down at 2023-01-08 12:43:31 UTC
my_app-db-1 | 2023-01-08 12:43:31.720 UTC [1] LOG: database system is ready to accept connections
It fails to connect to db because the service hasn't started yet depite being specified as a dependency. Here's the 2nd time:
% docker compose build && docker compose up
[+] Building 4.8s (11/11) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
=> [internal] load metadata for docker.io/library/python:3.11.1-bullseye 4.2s
=> [auth] library/python:pull token for registry-1.docker.io 0.0s
=> [1/5] FROM docker.io/library/python:3.11.1-bullseye#sha256:cc4910af48 0.0s
=> [internal] load build context 0.1s
=> => transferring context: 8.42kB 0.1s
=> CACHED [2/5] COPY requirements.txt requirements.txt 0.0s
=> CACHED [3/5] RUN pip install -r requirements.txt 0.0s
=> CACHED [4/5] COPY . /app 0.0s
=> CACHED [5/5] WORKDIR /app 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:d7b4a64b01b9de03dec4a0732eaf975b7bc68f1daefb4 0.0s
=> => naming to docker.io/library/my-app 0.0s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
[+] Running 2/0
⠿ Container my_app-db-1 Created 0.0s
⠿ Container my_app-web-1 Created 0.0s
Attaching to my_app-db-1, my_app-web-1
my_app-db-1 |
my_app-db-1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
my_app-db-1 |
my_app-db-1 | 2023-01-08 12:48:54.148 UTC [1] LOG: starting PostgreSQL 15.1 (Debian 15.1-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
my_app-db-1 | 2023-01-08 12:48:54.152 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
my_app-db-1 | 2023-01-08 12:48:54.153 UTC [1] LOG: listening on IPv6 address "::", port 5432
my_app-db-1 | 2023-01-08 12:48:54.157 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
my_app-db-1 | 2023-01-08 12:48:54.171 UTC [28] LOG: database system was shut down at 2023-01-08 12:48:30 UTC
my_app-db-1 | 2023-01-08 12:48:54.187 UTC [1] LOG: database system is ready to accept connections
my_app-web-1 | Operations to perform:
my_app-web-1 | Apply all migrations: admin, auth, contenttypes, core, sessions
my_app-web-1 | Running migrations:
my_app-web-1 | Applying contenttypes.0001_initial... OK
my_app-web-1 | Applying auth.0001_initial... OK
my_app-web-1 | Applying admin.0001_initial... OK
my_app-web-1 | Applying admin.0002_logentry_remove_auto_add... OK
my_app-web-1 | Applying admin.0003_logentry_add_action_flag_choices... OK
my_app-web-1 | Applying contenttypes.0002_remove_content_type_name... OK
my_app-web-1 | Applying auth.0002_alter_permission_name_max_length... OK
my_app-web-1 | Applying auth.0003_alter_user_email_max_length... OK
my_app-web-1 | Applying auth.0004_alter_user_username_opts... OK
my_app-web-1 | Applying auth.0005_alter_user_last_login_null... OK
my_app-web-1 | Applying auth.0006_require_contenttypes_0002... OK
my_app-web-1 | Applying auth.0007_alter_validators_add_error_messages... OK
my_app-web-1 | Applying auth.0008_alter_user_username_max_length... OK
my_app-web-1 | Applying auth.0009_alter_user_last_name_max_length... OK
my_app-web-1 | Applying auth.0010_alter_group_name_max_length... OK
my_app-web-1 | Applying auth.0011_update_proxy_permissions... OK
my_app-web-1 | Applying auth.0012_alter_user_first_name_max_length... OK
my_app-web-1 | Applying sessions.0001_initial... OK
my_app-web-1 | Watching for file changes with StatReloader
my_app-web-1 | Performing system checks...
my_app-web-1 |
my_app-web-1 | System check identified no issues (0 silenced).
my_app-web-1 | January 08, 2023 - 12:49:01
my_app-web-1 | Django version 4.1.5, using settings 'my_app.settings'
my_app-web-1 | Starting development server at http://0.0.0.0:8000/
my_app-web-1 | Quit the server with CONTROL-C.
It runs as it should've had in the first time.
According to the docker compose documentation, depends_on it does not wait until the service is ready, only that it has started. In my experience starting a postgres container usually takes a few seconds and since you need to wait until postgres is ready to recieve connections, it is recommended to use some form of control startup.
You could use the example Docker provides or do this yourself in Django, here is an example from a similar stackoverflow thread.
I have a django-rest-framwork app which uses PosgreSql as db. So I am using docker containers for them one image for django-rest-framwork and one for PosgreSql, and then docker compose file to handle them.
db service refrese to PostgreSQL
backend refers to Django Rest app
I have a docker-compose file
version: '3.9'
services:
db:
image: postgres:latest
restart: always
ports:
- "5432:5432"
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=85842475DB
- POSTGRES_DB=sports_center_db
- POSTGRES_PORT=5432
backend:
build: ./api
depends_on:
- db
ports:
- "8000:8000"
environment:
- DB_NAME=sports_center_db
- DB_USER_NAME=postgres
- DB_PASSWORD=85842475DB
- DB_HOST=db
- DB_PORT=5432
It builds correctly but when I ran
docker compose up
I got following logs
[+] Running 2/2
⠿ Container sports_center-db-1 Created 0.1s
⠿ Container sports_center-backend-1 Created 0.1s
Attaching to sports_center-backend-1, sports_center-db-1
sports_center-db-1 | The files belonging to this database system will be owned by user "postgres".
sports_center-db-1 | This user must also own the server process.
sports_center-db-1 |
sports_center-db-1 | The database cluster will be initialized with locale "en_US.utf8".
sports_center-db-1 | The default database encoding has accordingly been set to "UTF8".
sports_center-db-1 | The default text search configuration will be set to "english".
sports_center-db-1 |
sports_center-db-1 | Data page checksums are disabled.
sports_center-db-1 |
sports_center-db-1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
sports_center-db-1 | creating subdirectories ... ok
sports_center-db-1 | selecting dynamic shared memory implementation ... posix
sports_center-db-1 | selecting default max_connections ... 100
sports_center-db-1 | selecting default shared_buffers ... 128MB
sports_center-db-1 | selecting default time zone ... Etc/UTC
sports_center-db-1 | creating configuration files ... ok
sports_center-db-1 | running bootstrap script ... ok
sports_center-db-1 | performing post-bootstrap initialization ... ok
sports_center-backend-1 | Exception in thread django-main-thread:
sports_center-backend-1 | Traceback (most recent call last):
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/base.py", line 282, in ensure_connection
sports_center-backend-1 | self.connect()
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/utils/asyncio.py", line 26, in inner
sports_center-backend-1 | return func(*args, **kwargs)
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/base.py", line 263, in connect
sports_center-backend-1 | self.connection = self.get_new_connection(conn_params)
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/utils/asyncio.py", line 26, in inner
sports_center-backend-1 | return func(*args, **kwargs)
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/postgresql/base.py", line 215, in get_new_connection
sports_center-backend-1 | connection = Database.connect(**conn_params)
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/psycopg2/__init__.py", line 122, in connect
sports_center-backend-1 | conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
sports_center-backend-1 | psycopg2.OperationalError: connection to server at "db" (172.18.0.2), port 5432 failed: Connection refused
sports_center-backend-1 | Is the server running on that host and accepting TCP/IP connections?
sports_center-backend-1 |
sports_center-backend-1 |
sports_center-backend-1 | The above exception was the direct cause of the following exception:
sports_center-backend-1 |
sports_center-backend-1 | Traceback (most recent call last):
sports_center-backend-1 | File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
sports_center-backend-1 | self.run()
sports_center-backend-1 | File "/usr/lib/python3.10/threading.py", line 953, in run
sports_center-backend-1 | self._target(*self._args, **self._kwargs)
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/utils/autoreload.py", line 64, in wrapper
sports_center-backend-1 | fn(*args, **kwargs)
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/core/management/commands/runserver.py", line 137, in inner_run
sports_center-backend-1 | self.check_migrations()
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 564, in check_migrations
sports_center-backend-1 | executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/executor.py", line 18, in __init__
sports_center-backend-1 | self.loader = MigrationLoader(self.connection)
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/loader.py", line 58, in __init__
sports_center-backend-1 | self.build_graph()
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/loader.py", line 235, in build_graph
sports_center-backend-1 | self.applied_migrations = recorder.applied_migrations()
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/recorder.py", line 81, in applied_migrations
sports_center-backend-1 | if self.has_table():
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/recorder.py", line 57, in has_table
sports_center-backend-1 | with self.connection.cursor() as cursor:
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/utils/asyncio.py", line 26, in inner
sports_center-backend-1 | return func(*args, **kwargs)
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/base.py", line 323, in cursor
sports_center-backend-1 | return self._cursor()
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/base.py", line 299, in _cursor
sports_center-backend-1 | self.ensure_connection()
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/utils/asyncio.py", line 26, in inner
sports_center-backend-1 | return func(*args, **kwargs)
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/base.py", line 281, in ensure_connection
sports_center-backend-1 | with self.wrap_database_errors:
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/utils.py", line 91, in __exit__
sports_center-backend-1 | raise dj_exc_value.with_traceback(traceback) from exc_value
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/base.py", line 282, in ensure_connection
sports_center-backend-1 | self.connect()
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/utils/asyncio.py", line 26, in inner
sports_center-backend-1 | return func(*args, **kwargs)
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/base.py", line 263, in connect
sports_center-backend-1 | self.connection = self.get_new_connection(conn_params)
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/utils/asyncio.py", line 26, in inner
sports_center-backend-1 | return func(*args, **kwargs)
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/postgresql/base.py", line 215, in get_new_connection
sports_center-backend-1 | connection = Database.connect(**conn_params)
sports_center-backend-1 | File "/usr/local/lib/python3.10/dist-packages/psycopg2/__init__.py", line 122, in connect
sports_center-backend-1 | conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
sports_center-backend-1 | django.db.utils.OperationalError: connection to server at "db" (172.18.0.2), port 5432 failed: Connection refused
sports_center-backend-1 | Is the server running on that host and accepting TCP/IP connections?
sports_center-backend-1 |
sports_center-db-1 | syncing data to disk ... ok
sports_center-db-1 |
sports_center-db-1 |
sports_center-db-1 | Success. You can now start the database server using:
sports_center-db-1 |
sports_center-db-1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
sports_center-db-1 |
sports_center-db-1 | initdb: warning: enabling "trust" authentication for local connections
sports_center-db-1 | initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.
sports_center-db-1 | waiting for server to start....2022-10-16 19:57:48.569 UTC [48] LOG: starting PostgreSQL 15.0 (Debian 15.0-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
sports_center-db-1 | 2022-10-16 19:57:48.575 UTC [48] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
sports_center-db-1 | 2022-10-16 19:57:48.596 UTC [51] LOG: database system was shut down at 2022-10-16 19:57:41 UTC
sports_center-db-1 | 2022-10-16 19:57:48.607 UTC [48] LOG: database system is ready to accept connections
sports_center-db-1 | done
sports_center-db-1 | server started
sports_center-db-1 | CREATE DATABASE
sports_center-db-1 |
sports_center-db-1 |
sports_center-db-1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
sports_center-db-1 |
sports_center-db-1 | waiting for server to shut down...2022-10-16 19:57:48.814 UTC [48] LOG: received fast shutdown request
sports_center-db-1 | .2022-10-16 19:57:48.820 UTC [48] LOG: aborting any active transactions
sports_center-db-1 | 2022-10-16 19:57:48.823 UTC [48] LOG: background worker "logical replication launcher" (PID 54) exited with exit code 1
sports_center-db-1 | 2022-10-16 19:57:48.824 UTC [49] LOG: shutting down
sports_center-db-1 | 2022-10-16 19:57:48.830 UTC [49] LOG: checkpoint starting: shutdown immediate
sports_center-db-1 | 2022-10-16 19:57:49.416 UTC [49] LOG: checkpoint complete: wrote 918 buffers (5.6%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.047 s, sync=0.516 s, total=0.592 s; sync files=250, longest=0.009 s, average=0.003 s; distance=4217 kB, estimate=4217 kB
sports_center-db-1 | 2022-10-16 19:57:49.442 UTC [48] LOG: database system is shut down
sports_center-db-1 | done
sports_center-db-1 | server stopped
sports_center-db-1 |
sports_center-db-1 | PostgreSQL init process complete; ready for start up.
sports_center-db-1 |
sports_center-db-1 | 2022-10-16 19:57:49.580 UTC [1] LOG: starting PostgreSQL 15.0 (Debian 15.0-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
sports_center-db-1 | 2022-10-16 19:57:49.581 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
sports_center-db-1 | 2022-10-16 19:57:49.581 UTC [1] LOG: listening on IPv6 address "::", port 5432
sports_center-db-1 | 2022-10-16 19:57:49.593 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
sports_center-db-1 | 2022-10-16 19:57:49.609 UTC [63] LOG: database system was shut down at 2022-10-16 19:57:49 UTC
sports_center-db-1 | 2022-10-16 19:57:49.621 UTC [1] LOG: database system is ready to accept connections
sports_center-db-1 | 2022-10-16 20:02:49.708 UTC [61] LOG: checkpoint starting: time
sports_center-db-1 | 2022-10-16 20:02:53.897 UTC [61] LOG: checkpoint complete: wrote 44 buffers (0.3%); 0 WAL file(s) added, 0 removed, 0 recycled; write=4.131 s, sync=0.029 s, total=4.190 s; sync files=12, longest=0.006 s, average=0.003 s; distance=252 kB, estimate=252 kB
As I understand containers can not connect to each other. Any error or if I missing something here in the compose file, please tell me why this is happening and how to solve it.
Your application started before the database finished initializing. Depends_on waits for the other container to start, but there's still a delay before it's available. There are several possible solutions:
Configure the app to retry connecting to the database.
Configure the app to wait for the database using a command like wait-for-it.
Adjust the depends_on condition to also wait on the healthcheck:
version: "2.4"
services:
web:
build: .
depends_on:
db:
condition: service_healthy
redis:
condition: service_started
redis:
image: redis
db:
image: postgres
healthcheck:
test: "exit 0"
EDIT: as suggested by BMitch, maybe you need to create a network.
Initial answer:
Maybe you need to link them ?
Try adding the following to the backend:
links:
- db
There's a lot of dated information out there on similar issues, which I all went through. I'm running docker-compose file version 3.8. My issue is that:
psycopg2 or Django can not resolve my database's container name.
However, I can after a docker-compose up:
gerard#solace ~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
21bcbcc35083 project_web "/app/entrypoint.sh …" 6 seconds ago Up 6 seconds 0.0.0.0:8006->8000/tcp project_web_1
a92e3e98477f postgres:12.0-alpine "docker-entrypoint.s…" 7 seconds ago Up 6 seconds 0.0.0.0:5432->5432/tcp project_db_1
gerard#solace ~$ docker exec -it project_web_1 ping -c 2 db
PING db (172.25.0.2): 56 data bytes
64 bytes from 172.25.0.2: seq=0 ttl=64 time=0.078 ms
64 bytes from 172.25.0.2: seq=1 ttl=64 time=0.302 ms
--- db ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.078/0.190/0.302 ms```
Also, my entrypoint.sh connects before continuing, and does continu so host and port seems accessible:
Entrypoint snippet:
while ! nc -z $SQL_HOST $SQL_PORT; do
sleep 1
done
What am I missing?
To be complete. I've 'verbosed' everything in the webcontainer to make sure everything comes in from the .env files and what not, and everything looks good
db_1 | 2020-11-15 14:35:18.557 UTC [1] LOG: starting PostgreSQL 12.0 on x86_64-pc-linux-musl, compiled by gcc (Alpine 8.3.0) 8.3.0, 64-bit
db_1 | 2020-11-15 14:35:18.557 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2020-11-15 14:35:18.557 UTC [1] LOG: listening on IPv6 address "::", port 5432
web_1 | Waiting for postgres...
web_1 | PostgreSQL started
db_1 | 2020-11-15 14:35:18.559 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2020-11-15 14:35:18.575 UTC [18] LOG: database system was shut down at 2020-11-15 14:35:16 UTC
db_1 | 2020-11-15 14:35:18.580 UTC [1] LOG: database system is ready to accept connections
web_1 | YO: -----------------------------------------
web_1 | SQL_ENGINE ('django.db.backends.postgresql',)
web_1 | SQL_DATABASE ('diff_api_d',)
web_1 | SQL_HOST ('db',)
web_1 | SQL_PORT 5432
web_1 | SQL_USER ('diff_api_u',)
web_1 | SQL_PASSWORD ('diff_api_p',)
web_1 | Traceback (most recent call last):
web_1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 220, in ensure_connection
web_1 | self.connect()
web_1 | File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
web_1 | return func(*args, **kwargs)
web_1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/base/base.py", line 197, in connect
web_1 | self.connection = self.get_new_connection(conn_params)
web_1 | File "/usr/local/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
web_1 | return func(*args, **kwargs)
web_1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/postgresql/base.py", line 185, in get_new_connection
web_1 | connection = Database.connect(**conn_params)
web_1 | File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 127, in connect
web_1 | conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
web_1 | psycopg2.OperationalError: could not translate host name "('db'" to address: Name does not resolve
web_1 | could not translate host name ")" to address: Name does not resolve
As it turns out (if not always), it was a typo. I've moved the database settings away from the DATABASES dict to setting them at the top as individual declarations.
DATABASES = {
'default': {
"ENGINE": os.environ.get("SQL_ENGINE", "django.db.backends.sqlite3"),
"NAME": os.environ.get("SQL_DATABASE", os.path.join(BASE_DIR, "db.sqlite3")),
"USER": os.environ.get("SQL_USER", "user"),
"PASSWORD": os.environ.get("SQL_PASSWORD", "password"),
"HOST": os.environ.get("SQL_HOST", "localhost"),
"PORT": os.environ.get("SQL_PORT", "5432")
}
}
The issue was I forgot to remove the trailing comma's. Making them a tuple instead of a string.
# check the trailing comma
SQL_HOST = os.environ.get("SQL_HOST"),
Thanks to Adrian for finding the proverbial needle.
I have exactly followed this tutorial (http://docs.docker.com/compose/django/) but when doing docker-compose up I get this error:
$ docker-compose up
Recreating composeexample_db_1...
Recreating composeexample_web_1...
Attaching to composeexample_db_1, composeexample_web_1
db_1 | LOG: database system was shut down at 2015-07-27 16:17:21 UTC
db_1 | LOG: MultiXact member wraparound protections are now enabled
db_1 | LOG: database system is ready to accept connections
db_1 | LOG: autovacuum launcher started
web_1 | Performing system checks...
web_1 |
web_1 | System check identified no issues (0 silenced).
web_1 | Unhandled exception in thread started by <function wrapper at 0x7f43f0126ed8>
web_1 | Traceback (most recent call last):
web_1 | File "/usr/local/lib/python2.7/site-packages/django/utils/autoreload.py", line 225, in wrapper
web_1 | fn(*args, **kwargs)
web_1 | File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/runserver.py", line 112, in inner_run
web_1 | self.check_migrations()
web_1 | File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/runserver.py", line 164, in check_migrations
web_1 | executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
web_1 | File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 19, in __init__
web_1 | self.loader = MigrationLoader(self.connection)
web_1 | File "/usr/local/lib/python2.7/site-packages/django/db/migrations/loader.py", line 47, in __init__
web_1 | self.build_graph()
web_1 | File "/usr/local/lib/python2.7/site-packages/django/db/migrations/loader.py", line 182, in build_graph
web_1 | self.applied_migrations = recorder.applied_migrations()
web_1 | File "/usr/local/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 59, in applied_migrations
web_1 | self.ensure_schema()
web_1 | File "/usr/local/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 49, in ensure_schema
web_1 | if self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor()):
web_1 | File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 162, in cursor
web_1 | cursor = self.make_debug_cursor(self._cursor())
web_1 | File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 135, in _cursor
web_1 | self.ensure_connection()
web_1 | File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 130, in ensure_connection
web_1 | self.connect()
web_1 | File "/usr/local/lib/python2.7/site-packages/django/db/utils.py", line 97, in __exit__
web_1 | six.reraise(dj_exc_type, dj_exc_value, traceback)
web_1 | File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 130, in ensure_connection
web_1 | self.connect()
web_1 | File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 119, in connect
web_1 | self.connection = self.get_new_connection(conn_params)
web_1 | File "/usr/local/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 176, in get_new_connection
web_1 | connection = Database.connect(**conn_params)
web_1 | File "/usr/local/lib/python2.7/site-packages/psycopg2/__init__.py", line 164, in connect
web_1 | conn = _connect(dsn, connection_factory=connection_factory, async=async)
web_1 | django.db.utils.OperationalError: could not connect to server: No route to host
web_1 | Is the server running on host "db" (172.17.0.19) and accepting
web_1 | TCP/IP connections on port 5432?
Now, this only happens on my Fedora 21 system(kernel=4.0.4-202.fc21.x86_64) but when I spin up an Ubuntu instance in Virtualbox, it works just fine.
Any pointers?
I've run into the same issue and FWIW if you run the same command again then it works. It's almost as if "docker-compose up" brings up two containers at the same time and the django app container tries to run while postgres container is still being set up? It's mad confusing. :*(
UPDATE:
It seems that my suspicion was right, have a read at https://github.com/docker/compose/issues/374 .
Somewhat crude but a simple workaround to this race condition is to let django app container to sleep for a few seconds before running the command, so that the services that these containers depend on, e.g. PostgreSQL, are ready to accept connections. For example:
command: bash -c "sleep 3 && python manage.py runserver 0.0.0.0:8000" under django service in your yml file for docker-compose.
Another alternative is to run the django service with gunicorn or uwsgi which doesn't seem to instantiate the django application until request is received.
Hope this helps others currently fighting with docker.
I'm creating a docker-compose config for an django app, the Dockerfile builds successfully but when I compose them up, django return an issue -- cannot connect to posgres.
I run docker-compose run web bash, found redis and posgres both cannot be connected.
My docker-compose.yml file:
db:
image: postgres:9.1
environment:
- POSTGRES_PASSWORD=mysecretpassword
redis:
image: redis:2.8
web:
links:
- db
- redis
build: .
volumes:
- .:/workspace
ports:
- "8000:8000“
command: python /workspace/BreadTripServer/webapps/manage.py runserver 0.0.0.0:8000 --settings=configs.local_default
Error info when I do docker-compose up:
sudo docker-compose up
Recreating breadtrip_db_1...
Recreating breadtrip_redis_1...
Recreating breadtrip_web_1...
Attaching to breadtrip_redis_1, breadtrip_web_1
redis_1 | [1] 06 May 06:07:30.469 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
...
redis_1 | [1] 06 May 06:07:30.490 # Server started, Redis version 2.8.19
redis_1 | [1] 06 May 06:07:30.490 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1 | [1] 06 May 06:07:30.490 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1 | [1] 06 May 06:07:30.491 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1 | [1] 06 May 06:07:30.491 * DB loaded from disk: 0.000 seconds
redis_1 | [1] 06 May 06:07:30.491 * The server is now ready to accept connections on port 6379
web_1 | Traceback (most recent call last):
web_1 | File "/workspace/BreadTripServer/webapps/manage.py", line 14, in <module>
web_1 | execute_manager(settings)
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 438, in execute_manager
web_1 | utility.execute()
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 379, in execute
web_1 | self.fetch_command(subcommand).run_from_argv(self.argv)
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 191, in run_from_argv
web_1 | self.execute(*args, **options.__dict__)
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 209, in execute
web_1 | translation.activate('en-us')
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/utils/translation/__init__.py", line 100, in activate
web_1 | return _trans.activate(language)
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py", line 202, in activate
web_1 | _active.value = translation(language)
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py", line 185, in translation
web_1 | default_translation = _fetch(settings.LANGUAGE_CODE)
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/utils/translation/trans_real.py", line 162, in _fetch
web_1 | app = import_module(appname)
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/utils/importlib.py", line 35, in import_module
web_1 | __import__(name)
web_1 | File "/workspace/BreadTripServer/webapps/lib/haystack/__init__.py", line 83, in <module>
web_1 | backend = load_backend(settings.HAYSTACK_SEARCH_ENGINE)
web_1 | File "/workspace/BreadTripServer/webapps/lib/haystack/__init__.py", line 57, in load_backend
web_1 | return importlib.import_module('haystack.backends.%s_backend' % backend_name)
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/utils/importlib.py", line 35, in import_module
web_1 | __import__(name)
web_1 | File "/workspace/BreadTripServer/webapps/lib/haystack/backends/__init__.py", line 6, in <module>
web_1 | from django.db.models import Q
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/db/__init__.py", line 78, in <module>
web_1 | connection = connections[DEFAULT_DB_ALIAS]
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 94, in __getitem__
web_1 | conn = backend.DatabaseWrapper(db, alias)
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/contrib/gis/db/backends/postgis/base.py", line 11, in __init__
web_1 | self.ops = PostGISOperations(self)
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/contrib/gis/db/backends/postgis/operations.py", line 91, in __init__
web_1 | vtup = self.postgis_version_tuple()
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/contrib/gis/db/backends/postgis/operations.py", line 445, in postgis_version_tuple
web_1 | version = self.postgis_lib_version()
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/contrib/gis/db/backends/postgis/operations.py", line 425, in postgis_lib_version
web_1 | return self._get_postgis_func('postgis_lib_version')
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/contrib/gis/db/backends/postgis/operations.py", line 406, in _get_postgis_func
web_1 | cursor = self.connection._cursor()
web_1 | File "/usr/local/lib/python2.7/dist-packages/django/db/backends/postgresql_psycopg2/base.py", line 140, in _cursor
web_1 | self.connection = Database.connect(**conn_params)
web_1 | File "/usr/local/lib/python2.7/dist-packages/psycopg2/__init__.py", line 179, in connect
web_1 | connection_factory=connection_factory, async=async)
web_1 | psycopg2.OperationalError: could not connect to server: Connection refused
web_1 | Is the server running on host "localhost" (::1) and accepting
web_1 | TCP/IP connections on port 5432?
web_1 | could not connect to server: Connection refused
web_1 | Is the server running on host "localhost" (127.0.0.1) and accepting
web_1 | TCP/IP connections on port 5432?
web_1 |
breadtrip_web_1 exited with code 1
Update:
Cause I set a link to other image, docker linked pg on host 172.17.0.67, I need to set the pg host to that address which not mentioned on docker's official docs. (This address would change every time, but can be gotten by env)
Now, I got another issue:
web_1 | Unknown command: 'runserver'
web_1 | Type 'manage.py help' for usage.
breadtrip_web_1 exited with code 1
If I change the command python /workspace/BreadTripServer/webapps/manage.py runserver 0.0.0.0:8000 --settings=configs.local_default to python /workspace/BreadTripServer/webapps/manage.py runserver, it works OK except I cannot reach the web page.
It looks like your application is looking for the Database on localhost. It should be looking for the db at the host db (the host name will have been added to /etc/hosts by the link argument).
somewhere in your BUILD tree there is probably a file called settings.py. you can find this by typing the command :
find . -name settings.py
once found, you need to edit that file. It will look something like this:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
'NAME': 'mydb', # Or path to database file if using sqlite3.
# The following settings are not used with sqlite3:
'USER': 'myuser',
'PASSWORD': 'password',
'HOST': 'localhost', # Empty for localhost through domain sockets or '127.0.0.1' for localhost through TCP.
'PORT': '', # Set to empty string for default.
}
}
see the 'localhost' ? Change that to 'db' (As #Adrian suggested) and you will have this solved. (well, you will move on to the next problem :-) ) You should give Adrian credit for the answer.
I had this issue and was grasping at straws as I came across this SO answer: django.db.utils.OperationalError Could not connect to server.
The solution, which worked for me, was to kill all postgres processes using
sudo pkill postgres
I had this issue and It was only because I was using a VPN. If you are using something like sshuttle just turn it off.