I am uploading the .yml multicontainer file to EB from the AWS console and everything works fine the fist time, but next time I deploy the same .yml EB start a new mySql database, it seems that doesn't detect the volumes. Any ideas? Thanks!
Here my .yml file:
version: '3.3'
services:
database:
image: mysql:5.7.25
environment:
- MYSQL_ROOT_PASSWORD=testpass
- MYSQL_DATABASE=testdb
container_name: local_database
ports:
- '3306:3306'
volumes:
- './mysql_data:/var/lib/mysql'
nest:
image: rsteercen/nest_local:latest
container_name: nest
command: sh -c "sleep 10 && npm run migrations:run && node dist/src/main"
environment:
- DATABASE_TYPE=mysql
- DATABASE_HOST=local_database
- DATABASE_PORT=3306
- PORT=3000
- DATABASE_USER=root
- DATABASE_PASS=testpass
- DATABASE_NAME=testdb
ports:
- '80:3000'
depends_on:
- database
How is your volume defined?
If you're just using docker-compose, you should define also the volume there, fe. like:
volumes:
db-data:
driver_opts:
lifecycle_policy: AFTER_30_DAYS
and then attach it to the service:
services:
database:
volumes:
- db-data:/mysql
Related
I have a docker-compose file that spins up several services. I just got an error stating I ran out of disk space so I typed "docker system df" and saw I have 21 volumes. If I have 3 docker containers with each having a volume attached to them, why its showing me a total count of 21 volumes?
I am using AWS EC2. This is my compose file, is there anything wrong with the way I have the volumes set? The postgres data got persisted when I shutdown and restart, I am just confused aboutthe volume size and the message that I can not rebuild due to no space on a T2 Large instance.
version: "3"
services:
nftapi:
env_file:
- .env
build:
context: .
ports:
- '443:5000'
depends_on:
- postgres
volumes:
- .:/app
- /app/node_modules
networks:
- postgres
postgres:
container_name: postgres
image: postgres:latest
ports:
- "5432:5432"
volumes:
- /data/postgres:/data/postgres
env_file:
- docker.env
networks:
- postgres
pgadmin:
links:
- postgres:postgres
container_name: pgadmin
image: dpage/pgadmin4
ports:
- "8080:80"
volumes:
- /data/pgadmin:/root/.pgadmin
env_file:
- docker.env
networks:
- postgres
networks:
postgres:
driver: bridge
A Docker image's Dockerfile can contain a VOLUME directive. This is an instruction to Docker that tells it that some container directory contains data that needs to be persisted, and Docker should always ensure a volume of some sort is mounted on that directory.
More specifically, the postgres image declares
VOLUME /var/lib/postgresql/data
Your Compose setup doesn't mount anything on that specific directory. Because of this, Docker creates an anonymous volume and mounts it there for you. This isn't specific to the postgres image and other containers in your stack may have similar local data directories. Those anonymous volumes are what you're seeing in the docker system df output (docker volume ls will also show them).
In a later question you also note that Compose has trouble finding these anonymous volumes, and it's better to not rely on this functionality. Make sure you're mounting a host directory or named volume for these data directories via Compose volumes:.
if your main containers are up and running run
docker volume prune
and it should remove any volumes that are detached or unused by any container
i make it a habit to run periodically on my aws instance
docker system prune
I am trying to deploy an application in Docker running on 64bit Amazon Linux 2. I am using a pipeline, which publishes images to a private repository on Dockerhub. Elastic beanstalk uses docker-compose to run containers, but so far I've had no success in accessing the application. I am not using a dockerrun.aws.json file, as v.3 does not support any container configuration, and as far as I know, it's not needed for docker compose.
My docker-compose file contains several services, one of which is a RabbitMQ message broker.
version: '3.9'
services:
Some.API:
image: ...
container_name: some-api
networks:
- my-network
ports:
- "9002:80"
Another.API:
image: ...
container_name: another-api
networks:
- my-network
ports:
- "9003:80"
rabbitmQ:
image: rabbitmq:3-management-alpine
container_name: rabbit-mq
labels:
NAME: rabbitmq
volumes:
- ./rabbitconfig/rabbitmq-isolated.conf:/etc/rabbitmq/rabbitmq.config
networks:
- my-network
ports:
- "4369:4369"
- "5671:5671"
- "5672:5672"
- "25672:25672"
- "15671:15671"
- "15672:15672"
front-end:
image: ...
container_name: front-end
networks:
- my-network
ports:
- "9001:80"
networks:
my-network:
driver: bridge
Once the current version of the application is successfuly deployed to Beanstalk, I see that there is no successful communication in the bridge network.
In the eb-stdouterr.log I see that there are errors while establishing connection between the apis and the message broker:
RabbitMQ.Client.Exceptions.BrokerUnreachableException: None of the specified endpoints were reachable.
The APIs are .NET Core applications, which use the Beanstalk's environment variables to determine the name of the broker service. In the Configuration/Software/Environment properties section there is a following entry:
RABBIT_HOSTNAME | rabbitmq
which should ensure that the services use a proper host name.
Yet, I get exceptions. Any advice?
It turned out that I needed to reference the automatically generated .env file in docker-compose.yml like so:
front-end:
image: ...
container_name: front-end
networks:
- my-network
ports:
- "9001:80"
env_file: <--- these
- .env <--- 2 lines
for each service. Only after doing this the Environment properties from AWS Beanstalk were passed to the containers.
I am following the AWS tutorial on deploying an ECS application using docker compose.
When I run docker compose up, I only receive the message docker UpdateInProgress User Initiated, but nothing else happens:
[+] Running 0/0
- docker UpdateInProgress User Initiated 0.0s
Previously, this worked fine and all the ECS resources (cluster, task definitions, services, load balancer) had been created.
For some reason, now, this does not work anymore (although I have not changed my docker-compose.yml file).
docker-compose.yml:
version: '3'
services:
postgres:
image: ${AWS_DOCKER_REGISTRY}/postgres
networks:
- my-network
ports:
- "5432:5432"
volumes:
- postgres:/data/postgres
server:
image: ${AWS_DOCKER_REGISTRY}/server
networks:
- my-network
env_file:
- .env
ports:
- "${PORT}:${PORT}"
depends_on:
- postgres
entrypoint: "/server/run.sh"
pgadmin:
image: ${AWS_DOCKER_REGISTRY}/pgadmin
networks:
- my-network
depends_on:
- postgres
volumes:
- pgadmin:/root/.pgadmin
ports:
- "${PGADMIN_PORT:-5050}:${PGADMIN_PORT:-5050}"
networks:
my-network:
#driver: bridge
volumes:
postgres:
pgadmin:
I also switched to the correct Docker context before (docker context use my-aws-context).
And I have updated to the latest version of Docker Desktop for Windows and AWS CLI.
Did someone already have a similar problem?
From the message it appears that you are trying to compose up a stack that is existing already (on AWS) and so it's trying to update the CFN stack. Can you check if this is the case? You have a couple of options if that is what's happening: 1) delete the CFN stack (either in AWS or with docker compose down) or 2) launch the docker compose up with the flag --project-name string (where string is an arbitrary name of your choice). By default compose will use the directory name as the project name so if you compose up twice it will try to work on the same stack.
I'm trying to set up the nginx proxy on my Amazon AWS Docker instance together with a pimcore instance. This is my compose file:
version: '3.4'
services:
nginx-proxy:
image: codesuki/ecs-nginx-proxy
ports:
- "80:80"
pimcore-jcii:
image: ****/pimcore5:current
ports:
- "8000:80"
links:
- "db"
volumes:
- efs-storage:/data
db:
image: mariadb
restart: always
ports:
- "3306:3306"
volumes:
efs-storage:
driver: cloudstor:aws
driver_opts:
backing: shared
If I deploy this stack the nginx proxy container can not start. It appears the following error message:
task: non-zero exit (1)
I've got the error message by "docker inspect ".
What am I doing wrong? Or where can I gather more informations about the state of the container?
Context :
I am trying to setup a selenium grid to run my UI tests on CI.CI is Jenkins 2.0 and it runs on AWS ECS.When I create a selenium grid using the docker compose and invoke the tests on my MAC (OS Sierra) , it works perfectly.
When run on the AWS ECS , it shows me an : java.awt.AWTError: Can't connect to X11 window server using '99.0' as the value of the DISPLAY variable.
The test code itself is in a container and using a bridge network I have added the container to the same network as the grid.
The docker compose looks something like this :
version: '3'
services:
chromenode:
image: selenium/node-chrome:3.4.0
volumes:
- /dev/shm:/dev/shm
- /var/run/docker.sock:/var/run/docker.sock
container_name: chromenode
hostname: chromenode
depends_on:
- seleniumhub
ports:
- "5900:5900"
environment:
- "HUB_PORT_4444_TCP_ADDR=seleniumhub"
- "HUB_PORT_4444_TCP_PORT=4444"
networks:
- grid_network
seleniumhub:
image: selenium/hub:3.4.0
ports:
- "4444:4444"
container_name: seleniumhub
hostname: seleniumhub
networks:
- grid_network
volumes:
- /var/run/docker.sock:/var/run/docker.sock
testservice:
build:
context: .
dockerfile: DockerfileTest
networks:
- grid_network
networks:
grid_network:
driver: bridge
Please let me know if more info is required.
unset DISPLAY This helped me to solve the problem
This helps in most cases (e.g. starting application servers or other java based tools) and avoids to modify all that many command lines.
It can also be comfortable to add it to the .bash_profile for a dedicated app-server/tools user.
Can you please try this
- no_proxy=""