I am testing my app with docker-compose which consist of DynamoDB as an inner container.docker-compose file as below:
version: '2'
services:
appName:
mem_limit: 1024m
build:
dockerfile: dockerfile.test
context: .
ports:
- "8090:8090"
env_file:
- env/test.env
depends_on:
- redis
- postgres
- dynamodb
- memcached
entrypoint: "./bin/entrypoint.sh"
redis:
image: "redis:alpine"
postgres:
image: "postgres:9.6-alpine"
dynamodb:
image: "tutum/dynamodb:latest"
ports:
- "8000:8000"
hostname: dynamodb
memcached:
image: "memcached:alpine"
On building up the code i am getting following error:
java.lang.IncompatibleClassChangeError: Class com.amazonaws.http.conn.ssl.SdkTLSSocketFactory does not implement the requested interface org.apache.http.conn.scheme.SchemeSocketFactory at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:165) org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304) org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611) org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446) org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863) org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57) com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:837) com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:607)
com.amazonaws.http.AmazonHttpClient.doExecute(AmazonHttpClient.java:376) com.amazonaws.http.AmazonHttpClient.executeWithTimer(AmazonHttpClient.java:338) com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:287) com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:2000) com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:1970) com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.getItem(AmazonDynamoDBClient.java:1329) com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper.load(DynamoDBMapper.java:433) com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper.load(DynamoDBMapper.java:496) com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBMapper.load(DynamoDBMapper.java:400)
The fix required to get it working is changing the hostname of dynamodb from dynamodb to http://dynamodb, In docker file
Related
I'm running two applications using docker-compose. Each application has a bunch of containers. The intention is for App A (django app) to host the OIDC provider, while App B (some other app) will authenticate users by calling the App A API.
I'm using the django-oidc-provider library (https://django-oidc-provider.readthedocs.io/en/latest/index.html)
I've already configured the OIDC integration on both sides. However, every time App B redirects to App A, I hit the following error:
Redirect URI Error
The request fails due to a missing, invalid, or mismatching redirection URI (redirect_uri).
Even though the redirect_uri matches exactly on both sides.
Here's my docker-compose.yml:
version: '3'
networks:
default:
external:
name: datahub-gms_default
services:
django:
build:
context: .
dockerfile: ./compose/local/django/Dockerfile
image: dqt
container_name: dqt
hostname: dqt
platform: linux/x86_64
depends_on:
- postgres
volumes:
- .:/app:z
environment:
- DJANGO_READ_DOT_ENV_FILE=true
env_file:
- ./.envs/.local/.django
- ./.envs/.local/.postgres
ports:
- "8000:8000"
command: /start
postgres:
build:
context: .
dockerfile: ./compose/local/postgres/Dockerfile
image: postgres
container_name: postgres
hostname: postgres
volumes:
- dqt_local_postgres_data:/var/lib/postgresql/data:Z
- dqt_local_postgres_data_backups:/backups:z
env_file:
- ./.envs/.local/.postgres
broker:
container_name: broker
depends_on:
- zookeeper
environment:
- KAFKA_BROKER_ID=1
- KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
- KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
- KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092
- KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1
- KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS=0
- KAFKA_HEAP_OPTS=-Xms256m -Xmx256m
hostname: broker
image: confluentinc/cp-kafka:5.4.0
ports:
- 29092:29092
- 9092:9092
datahub-actions:
depends_on:
- datahub-gms
environment:
- GMS_HOST=datahub-gms
- GMS_PORT=8080
- KAFKA_BOOTSTRAP_SERVER=broker:29092
- SCHEMA_REGISTRY_URL=http://schema-registry:8081
- METADATA_AUDIT_EVENT_NAME=MetadataAuditEvent_v4
- METADATA_CHANGE_LOG_VERSIONED_TOPIC_NAME=MetadataChangeLog_Versioned_v1
- DATAHUB_SYSTEM_CLIENT_ID=__datahub_system
- DATAHUB_SYSTEM_CLIENT_SECRET=JohnSnowKnowsNothing
- KAFKA_PROPERTIES_SECURITY_PROTOCOL=PLAINTEXT
hostname: actions
image: public.ecr.aws/datahub/acryl-datahub-actions:${ACTIONS_VERSION:-head}
datahub-frontend-react:
container_name: datahub-frontend-react
depends_on:
- datahub-gms
environment:
- DATAHUB_GMS_HOST=datahub-gms
- DATAHUB_GMS_PORT=8080
- DATAHUB_SECRET=YouKnowNothing
- DATAHUB_APP_VERSION=1.0
- DATAHUB_PLAY_MEM_BUFFER_SIZE=10MB
- JAVA_OPTS=-Xms512m -Xmx512m -Dhttp.port=9002 -Dconfig.file=datahub-frontend/conf/application.conf
-Djava.security.auth.login.config=datahub-frontend/conf/jaas.conf -Dlogback.configurationFile=datahub-frontend/conf/logback.xml
-Dlogback.debug=false -Dpidfile.path=/dev/null
- KAFKA_BOOTSTRAP_SERVER=broker:29092
- DATAHUB_TRACKING_TOPIC=DataHubUsageEvent_v1
- ELASTIC_CLIENT_HOST=elasticsearch
- ELASTIC_CLIENT_PORT=9200
- AUTH_OIDC_ENABLED=true
- AUTH_OIDC_CLIENT_ID=778948
- AUTH_OIDC_CLIENT_SECRET=some-client-secret
- AUTH_OIDC_DISCOVERY_URI=http://dqt:8000/openid/.well-known/openid-configuration/
- AUTH_OIDC_BASE_URL=http://datahub:9002/
hostname: datahub
image: linkedin/datahub-frontend-react:${DATAHUB_VERSION:-head}
ports:
- 9002:9002
datahub-gms:
container_name: datahub-gms
depends_on:
- mysql
environment:
- DATASET_ENABLE_SCSI=false
- EBEAN_DATASOURCE_USERNAME=datahub
- EBEAN_DATASOURCE_PASSWORD=datahub
- EBEAN_DATASOURCE_HOST=mysql:3306
- EBEAN_DATASOURCE_URL=jdbc:mysql://mysql:3306/datahub?verifyServerCertificate=false&useSSL=true&useUnicode=yes&characterEncoding=UTF-8
- EBEAN_DATASOURCE_DRIVER=com.mysql.jdbc.Driver
- KAFKA_BOOTSTRAP_SERVER=broker:29092
- KAFKA_SCHEMAREGISTRY_URL=http://schema-registry:8081
- ELASTICSEARCH_HOST=elasticsearch
- ELASTICSEARCH_PORT=9200
- GRAPH_SERVICE_IMPL=elasticsearch
- JAVA_OPTS=-Xms1g -Xmx1g
- ENTITY_REGISTRY_CONFIG_PATH=/datahub/datahub-gms/resources/entity-registry.yml
- MAE_CONSUMER_ENABLED=true
- MCE_CONSUMER_ENABLED=true
hostname: datahub-gms
image: linkedin/datahub-gms:${DATAHUB_VERSION:-head}
ports:
- 8080:8080
volumes:
- ${HOME}/.datahub/plugins:/etc/datahub/plugins
elasticsearch:
container_name: elasticsearch
environment:
- discovery.type=single-node
- xpack.security.enabled=false
- ES_JAVA_OPTS=-Xms256m -Xmx256m -Dlog4j2.formatMsgNoLookups=true
healthcheck:
retries: 4
start_period: 2m
test:
- CMD-SHELL
- curl -sS --fail 'http://localhost:9200/_cluster/health?wait_for_status=yellow&timeout=0s' || exit 1
hostname: elasticsearch
image: elasticsearch:7.9.3
mem_limit: 1g
ports:
- 9200:9200
volumes:
- esdata:/usr/share/elasticsearch/data
elasticsearch-setup:
container_name: elasticsearch-setup
depends_on:
- elasticsearch
environment:
- ELASTICSEARCH_HOST=elasticsearch
- ELASTICSEARCH_PORT=9200
- ELASTICSEARCH_PROTOCOL=http
hostname: elasticsearch-setup
image: linkedin/datahub-elasticsearch-setup:${DATAHUB_VERSION:-head}
kafka-setup:
container_name: kafka-setup
depends_on:
- broker
- schema-registry
environment:
- KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
- KAFKA_BOOTSTRAP_SERVER=broker:29092
hostname: kafka-setup
image: linkedin/datahub-kafka-setup:${DATAHUB_VERSION:-head}
mysql:
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_bin
container_name: mysql
environment:
- MYSQL_DATABASE=datahub
- MYSQL_USER=datahub
- MYSQL_PASSWORD=datahub
- MYSQL_ROOT_PASSWORD=datahub
hostname: mysql
image: mysql:5.7
ports:
- 3306:3306
volumes:
- ../mysql/init.sql:/docker-entrypoint-initdb.d/init.sql
- mysqldata:/var/lib/mysql
mysql-setup:
container_name: mysql-setup
depends_on:
- mysql
environment:
- MYSQL_HOST=mysql
- MYSQL_PORT=3306
- MYSQL_USERNAME=datahub
- MYSQL_PASSWORD=datahub
- DATAHUB_DB_NAME=datahub
hostname: mysql-setup
image: acryldata/datahub-mysql-setup:head
schema-registry:
container_name: schema-registry
depends_on:
- zookeeper
- broker
environment:
- SCHEMA_REGISTRY_HOST_NAME=schemaregistry
- SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL=zookeeper:2181
hostname: schema-registry
image: confluentinc/cp-schema-registry:5.4.0
ports:
- 8081:8081
zookeeper:
container_name: zookeeper
environment:
- ZOOKEEPER_CLIENT_PORT=2181
- ZOOKEEPER_TICK_TIME=2000
hostname: zookeeper
image: confluentinc/cp-zookeeper:5.4.0
ports:
- 2181:2181
volumes:
- zkdata:/var/opt/zookeeper
volumes:
dqt_local_postgres_data: {}
dqt_local_postgres_data_backups: {}
esdata: null
mysqldata: null
zkdata: null
In the above, container datahub-frontend-react is supposed to integrate into container dqt for the OIDC authentication.
The docker log doesn't show any exceptions, and the http code is 200:
dqt | [28/Feb/2022 10:43:43] "GET /openid/.well-known/openid-configuration/ HTTP/1.1" 200 682
dqt | [28/Feb/2022 10:43:44] "GET /openid/authorize?response_type=code&redirect_uri=http%3A%2F%2Fdatahub%3A9002%2F%2Fcallback%2Foidc&state=9Fj1Bog-ZN8fhN2kufWng2fRGaqCYnkMz6n3yKxPowo&client_id=778948&scope=openid+profile+email HTTP/1.1" 200 126
Here's the redirect_uri configuration in django admin:
I'm suspecting it could be related to the fact that they are different containers with different hostnames (I don't know what to do about that).
What could be the root cause of this issue?
Your log shows that the app is redirecting with this login URL, with two %2F characters, so the URL used by the app is different to that configured:
http://datahub:9002//callback/oidc
INTERNAL AND EXTERNAL URLs
Not sure if it will work once you resolve that though, since the callback URL looks like a Docker Compose internal URL, that the browser will be unable to reach. Aim to return a URL such as this instead:
http://localhost:9002/callback/oidc
One option that can be useful to make URLs more understandable during development, and to plan the real deployment, is to add custom host names to your computer's hosts file. You can then login via URLs such as http://www.myapp.com, which I find clearer.
See these resources for something to compare against, which describe a setup with both internal and external URLs.
Custom Hosts
Docker Compose Example
I'm having issues connecting with my Elasticsearch container since day 1.
First I was using elasticsearch as the hostname, then I've tried the container name web_elasticsearch_1, and finally I'd set a Static IP address to the container and passed it in my configuration file.
PYPI packages:
django==3.2.9
elasticsearch==7.15.1
elasticsearch-dsl==7.4.0
docker-compose.yml
version: "3.3"
services:
web:
build:
context: .
dockerfile: local/Dockerfile
image: project32439/python
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
env_file:
- local/python.env
depends_on:
- elasticsearch
elasticsearch:
image: elasticsearch:7.10.1
environment:
- xpack.security.enabled=false
- discovery.type=single-node
networks:
default:
ipv4_address: 172.18.0.10
settings.py
# Elasticsearch
ELASTICSEARCH_HOST = "172.18.0.10"
ELASTICSEARCH_PORT = 9200
service.py
from django.conf import settings
from elasticsearch import Elasticsearch, RequestsHttpConnection
es = Elasticsearch(
hosts=[{"host": settings.ELASTICSEARCH_HOST, "port": settings.ELASTICSEARCH_PORT}],
use_ssl=False,
verify_certs=False,
connection_class=RequestsHttpConnection,
)
traceback
HTTPConnectionPool(host='172.18.0.10', port=9200): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f1973ebd6d0>, 'Connection to 172.18.0.10 timed out. (connect timeout=5)'))
By default Docker Compose uses a bridge network to provision inter-container communication. You can read more about this network at the Debian Wiki.
What matters for you, is that by default Docker Compose creates a hostname that equals the service name in the docker-compose.yml file. So update your file:
version: "3.3"
services:
web:
build:
context: .
dockerfile: local/Dockerfile
image: project32439/python
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
env_file:
- local/python.env
depends_on:
- elasticsearch
elasticsearch:
image: elasticsearch:7.10.1
environment:
- xpack.security.enabled=false
- discovery.type=single-node
And now you can connect with elasticsearch:9200 instead of 172.18.0.10 from your web container. For more info see this article.
these are my first times with docker, I'm trying to put the directives for MariaDB in my composer file. Once the file is written, if I try to run the docker compose up command, it gives me the following error: yaml: line 8: did not find expected key. Does anyone have the same problem as me? How can I solve? Thanks so much.
Below is my docker-compose.yaml file
version: '3'
services:
backend:
build: ./
restart: always
volumes:
- ./application:/var/www/html
ports: [80:80]
mariadb:
image: 'bitnami/mariadb:latest'
ports:
- '3306:3306'
volumes:
- './mariadb_data:/bitnami/mariadb'
environment:
- ALLOW_EMPTY_PASSWORD=yes
- MARIADB_DATABASE=db_test
- MARIADB_USER=test_user
- MARIADB_PASSWORD=password
- MARIADB_ROOT_HOST='%'
volumes:
application:
driver: local
mariadb_data:
driver: local
It happens when we do our own yml file for docker, you need to indent two spaces for sub entries under image details:
version: '1'
services:
mariadb-ikg:
image: bitnami/mariadb:10.3
ports:
- 3306:3306
volumes:
- D:/docker/bitnami-mariadb/databases:/bitnami/mariadb
environment:
- MARIADB_ROOT_PASSWORD=123456
phpfpm-ikg:
image: wyveo/nginx-php-fpm:php80
ports:
- 80:80
volumes:
- D:/docker/wyveo-nginx-php-fpm/wordpress:/usr/share/nginx/html
depends_on:
- mariadb-ikg
My django app is failing to connect to the psql container with the standard connection refused error. I used django-cookiecutter which supplies the psql username and password automatically via environment variables and then this I gather is passed back into django with via a .env file that hosts a DATABASE_URL string.
Error
django.db.utils.OperationalError: could not connect to server: Connection refused
Is the server running on host "127.0.0.1" and accepting
TCP/IP connections on port 5432?
When I set a breakpoint in django settings I can see that the DATABASE_URL seems to be converted appropriately into the standard db dict:
{'NAME': 'hustlestat', 'USER': 'HjhPLEwuVjUIIKEHebPqNG<redacted>', 'PASSWORD': 'I43443fR42wRkUaaQ8mkd<redacted>', 'HOST': 'postgres', 'PORT': 5432, 'ENGINE': 'django.db.backends.postgresql'}
When I exec into the psql container with psql hustlestat -U HjhPLEwuVjUIIKEHebPqN<redcated> I can connect to the db using that username. I'm not 100% on the password as it isn't asking me for one when I try to connect.
Here is the docker compose which is generated automatically by cookie cutter:
version: '3'
volumes:
local_postgres_data: {}
local_postgres_data_backups: {}
services:
django: &django
build:
context: .
dockerfile: ./compose/local/django/Dockerfile
image: hustlestat_local_django
container_name: django
depends_on:
- postgres
- mailhog
volumes:
- .:/app:z
env_file:
- ./.envs/.local/.django
- ./.envs/.local/.postgres
ports:
- "8000:8000"
command: /start
postgres:
build:
context: .
dockerfile: ./compose/production/postgres/Dockerfile
image: hustlestat_production_postgres
container_name: postgres
volumes:
- local_postgres_data:/var/lib/postgresql/data:Z
- local_postgres_data_backups:/backups:z
env_file:
- ./.envs/.local/.postgres
docs:
image: hustlestat_local_docs
container_name: docs
build:
context: .
dockerfile: ./compose/local/docs/Dockerfile
env_file:
- ./.envs/.local/.django
volumes:
- ./docs:/docs:z
- ./config:/app/config:z
- ./hustlestat:/app/hustlestat:z
ports:
- "7000:7000"
command: /start-docs
mailhog:
image: mailhog/mailhog:v1.0.0
container_name: mailhog
ports:
- "8025:8025"
redis:
image: redis:5.0
container_name: redis
celeryworker:
<<: *django
image: hustlestat_local_celeryworker
container_name: celeryworker
depends_on:
- redis
- postgres
- mailhog
ports: []
command: /start-celeryworker
celerybeat:
<<: *django
image: hustlestat_local_celerybeat
container_name: celerybeat
depends_on:
- redis
- postgres
- mailhog
ports: []
command: /start-celerybeat
flower:
<<: *django
image: hustlestat_local_flower
container_name: flower
ports:
- "5555:5555"
command: /start-flower
node:
build:
context: .
dockerfile: ./compose/local/node/Dockerfile
image: hustlestat_local_node
container_name: node
depends_on:
- django
volumes:
- .:/app:z
# http://jdlm.info/articles/2016/03/06/lessons-building-node-app-docker.html
- /app/node_modules
command: npm run dev
ports:
- "3000:3000"
# Expose browsersync UI: https://www.browsersync.io/docs/options/#option-ui
- "3001:3001"
The only oddity I have noticed is that despite django being named in the docker compose, when I view the running containers it has a random name such as:
hustlestat_django_run_37888ff2c9ca
Not sure if that is relevant.
Thanks for any help!
Okay have figured this out. I set a DATABASE_URL environment variable because I was originally getting an error saying it was unset. After googling I came across a cookie cutter doc that said to set it but didn't read it well enough to realise that the instruction was intended for non-docker setups. Mine is docker.
The reason I was getting that error is because I was exec'ing into the container and running management commands like this:
docker exec -it django bash then python manage.py migrate
The way this project is setup and environment variables are setup, you can't do that, you have to use this method from outside the exec:
docker-compose -f local.yml run --rm django python manage.py migrate
I thought the two methods were interchangeable but they are not. Everything works now.
I realize this question comes up a lot. I've read many threads and blog posts, but here I am. Long story short, I have a docker container running wurstmeister/zookeeper & wurstmeister/kafka, and then some services running in their own containers. I'll just mention the NoddJS one for now. Everything works fine at home, using IP addresses (not localhost) so I'm baffled at what the difference is here. On AWS, it simply "doesn't work" even though it seems to at least connect to the broker in the beginning. I'm explicitly using internal IPs in the config as I don't want this exposed to anything externally.
Reading around, I've tried 2 setups. 1 works at home (KAFKA_ADVERTISED_HOST_NAME). 1 doesn't (KAFKA_ADVERTISED_LISTENERS). Neither work on my EC2 Linux box:
Kafka docker-compose.yml
version: '2'
services:
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"
networks:
- my-network
kafka:
image: wurstmeister/kafka
depends_on:
- zookeeper
ports:
- "9092:9092"
environment:
KAFKA_ADVERTISED_HOST_NAME: <internal-ip>
KAFKA_ADVERTISED_PORT: "9092"
KAFKA_CREATE_TOPICS: "test:1:1"
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
volumes:
- /var/run/docker.sock:/var/run/docker.sock
networks:
- my-network
networks:
my-network:
NodeJS docker-compose.yml
version: '2'
services:
nodejs:
build:
context: ./
dockerfile: Dockerfile
networks:
- kafka_my-network
restart: unless-stopped
ports:
- "1337:3000"
volumes:
- "/tmp:/tmp"
- "/var/log:/var/log"
networks:
kafka_my-network:
external: true
Then in NodeJS
const kafkaHost = '<internal-ip>:9092';
const client = new kafka.KafkaClient({kafkaHost});
const producer = new kafka.Producer(client)
const kakfaTopic = 'test';
producer.on('ready', function() {
console.log(`kafka producer is ready`); // I see this, so I'm assuming all is well
ready = true;
});
producer.on('error', function(err) {
console.error(err);
});
const payload = [
{
topic: kafkaTopic,
messages: JSON.stringify(myMessages);
}
]
producer.send(payload, function(err, data) {
if (err) {
console.error(`Send error ${JSON.stringify(err}`);
}
console.log(`Sent data ${JSON.stringify(data)}`);
});
When I start my NodeJS server, I see that I've connected to a Kafka broker. I can confirm as well that :9092 is open after checking w/ telnet and/or nc. Then, when it sends a request, the callback gets an empty error.
I realize KAKFA_ADVERTISED_HOST_NAME is deprecated, so in the name of completion, here is my attempt using ADVERTISED_LISTENERS which failed. With this configuration I seemed to get the same results at home as I did on EC2.
version: '2'
services:
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"
networks:
- my-network
kafka:
image: wurstmeister/kafka
depends_on:
- zookeeper
ports:
- "9092:9092"
environment:
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://<internal-ip>:9092
KAFKA_CREATE_TOPICS: "test:1:1"
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
volumes:
- /var/run/docker.sock:/var/run/docker.sock
networks:
- my-network
networks:
my-network:
EDIT: I will not provide this as a solution, but the bitnami image w/ the following config works. The main difference is it had a pretty straight forward README that I went through. I can't be certain if I tried the equivalent config w/ wurstmeister (I tried many - and again, at least one of which in docker containers on my own machine but not on a single EC2 instance).
Note that I did list 'kafka' with an internal IP (not loopback) in /etc/hosts for this. This should be tantamount to using the internal IP explicitly which I had done above.
version: '2'
services:
zookeeper:
image: 'bitnami/zookeeper:3'
ports:
- '2181:2181'
volumes:
- 'zookeeper_data:/bitnami'
environment:
- ALLOW_ANONYMOUS_LOGIN=yes
networks:
- my-network
kafka:
image: 'bitnami/kafka:2'
ports:
- '9092:9092'
- '29092:29092'
volumes:
- 'kafka_data:/bitnami'
- /var/run/docker.sock:/var/run/docker.sock
environment:
- KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
- ALLOW_PLAINTEXT_LISTENER=yes
- KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
- KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,PLAINTEXT_HOST://:29092
- KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
depends_on:
- zookeeper
networks:
- my-network
volumes:
zookeeper_data:
driver: local
kafka_data:
driver: local
networks:
my-network:
driver: bridge