Error while deploying web app to AWS ElasticBeanStalk - amazon-web-services

I am getting the below error while deploying to aws elastic beanstalk from travis CI.
Service:AmazonECS, Code:ClientException, Message:Container list cannot be empty., Class:com.amazonaws.services.ecs.model.ClientException
.travis.yml:
sudo: required
language: generic
services:
- docker
before_install:
- docker build -t sathishpskdocker/react-test -f ./client/Dockerfile.dev ./client
script:
- docker run -e CI=true sathishpskdocker/react-test npm test
after_success:
- docker build -t sathishpskdocker/multi-client ./client
- docker build -t sathishpskdocker/multi-nginx ./nginx
- docker build -t sathishpskdocker/multi-server ./server
- docker build -t sathishpskdocker/multi-worker ./worker
# Log in to the docker CLI
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_ID" --password-stdin
# Take those images and push them to docker hub
- docker push sathishpskdocker/multi-client
- docker push sathishpskdocker/multi-nginx
- docker push sathishpskdocker/multi-server
- docker push sathishpskdocker/multi-worker
deploy:
provider: elasticbeanstalk
region: 'us-west-2'
app: 'multi-docker'
env: 'Multidocker-env'
bucker_name: elasticbeanstalk-us-west-2-194531873493
bucker_path: docker-multi
On:
branch: master
access_key_id: $AWS_ACCESS_KEY
secret_access_key: $AWS_SECRET_KEY
Dockerrun.aws.json:
{
"AWSEBDockerrunVersion": 2,
"containerDefintions": [
{
"name": "client",
"image": "sathishpskdocker/multi-client",
"hostname": "client",
"essential": false,
"memory": 128
},
{
"name": "server",
"image": "sathishpskdocker/multi-server",
"hostname": "api",
"essential": false,
"memory": 128
},
{
"name": "worker",
"image": "sathishpskdocker/multi-worker",
"hostname": "worker",
"essential": false,
"memory": 128
},
{
"name": "nginx",
"image": "sathishpskdocker/multi-nginx",
"hostname": "nginx",
"essential": true,
"portMappings": [
{
"hostPort": 80,
"containerPort": 80
}
],
"links": ["client", "server"],
"memory": 128
}
]
}
Deploying part alone failing with the error:
Service:AmazonECS, Code:ClientException, Message:Container list cannot be empty., Class:com.amazonaws.services.ecs.model.ClientException

Ah, Never mind, it's my mistake. There is typo in the dockerrun config file which wrongly reads containerDefintions instead of containerDefinitions.
Thanks everyone whoever taking look at my question. Cheers!

Related

why can't amazon codebuild execute: docker-compose --file=docker-compose-prod.yml build?

I am new to AWS.
I am trying to deploy a multi-container docker application with with CI/CD using CodePipeline and AWS ECR. I am using following this example: https://aws.plainenglish.io/deploy-multi-container-docker-to-elastic-beanstalk-with-ci-cd-using-codepipeline-and-aws-ecr-d1d5be0aaa20
CodeBuild is executing this command:
docker-compose --file=docker-compose-prod.yml build from the buildspec.yml
build:
commands:
# Test Build (Dockerfile.dev)
#- echo Build started on `date`
#- echo Building Docker test image...
#- docker build -t $IMAGE_REPO_NAME:$TEST_IMAGE_TAG -f Dockerfile.dev .
#- docker run -e CI=true $IMAGE_REPO_NAME:$TEST_IMAGE_TAG npm run test
- echo Build started on `date`
- echo Building the Docker image...
- docker-compose --file=docker-compose-prod.yml build
the error message is:
[Container] 2022/11/12 18:09:44 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker-compose --file=docker-compose-prod.yml build. Reason: exit status 1
here is the docker-compose-prod.yml
version: '3'
services:
api:
build:
context: ./api
container_name: api
image: ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/app-api
ports:
- "3000:3000"
ui:
build:
context: ./ui
container_name: ui
image: ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/app-ui
ports:
- "4200:4200"
nginx:
build:
context: ./nginx
container_name: nginx
depends_on:
- ui
- api
image: ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/app-nginx
ports:
- "80:80"
here is my private registies
why can't amazon execute this command?
also here is Dockerrun.aws.json:
{
"AWSEBDockerrunVersion": 2,
"containerDefinitions": [
{
"name": "ui",
"image": "my_accnt_id.dkr.ecr.us-east-1.amazonaws.com/app-ui",
"hostname": "ui",
"essential": true,
"memory": 1024,
"portMappings": [
{
"hostPort": 4200,
"containerPort": 4200
}
]
},
{
"name": "api",
"image": "my_accnt_id.dkr.ecr.us-east-1.amazonaws.com/app-api",
"hostname": "api",
"essential": true,
"memory": 512,
"portMappings": [
{
"hostPort": 3000,
"containerPort": 3000
}
]
},
{
"name": "nginx",
"image": "my_accnt_id.dkr.ecr.us-east-1.amazonaws.com/app-nginx",
"hostname": "nginx",
"essential": true,
"portMappings": [
{
"hostPort": 80,
"containerPort": 80
}
],
"links": ["ui", "api"],
"memory": 512
}
]
}
my my_accnt_id is where i actually have my account id.
CloudWatch error msg:
2022-11-12T17:37:27.435+00:00
Copy
[Container] 2022/11/12 17:37:25 Running command docker-compose --file=docker-compose-prod.yml push
[Container] 2022/11/12 17:37:25 Running command docker-compose --file=docker-compose-prod.yml push
2022-11-12T17:37:27.435+00:00
Copy
Pushing api (280757731345.dkr.ecr.us-east-1.amazonaws.com/app-api:latest)...
Pushing api (280757731345.dkr.ecr.us-east-1.amazonaws.com/app-api:latest)...
2022-11-12T17:37:27.435+00:00
Copy
The push refers to repository [280757731345.dkr.ecr.us-east-1.amazonaws.com/app-api]
The push refers to repository [280757731345.dkr.ecr.us-east-1.amazonaws.com/app-api]
2022-11-12T17:38:18.717+00:00
Copy
EOF
EOF
2022-11-12T17:38:18.717+00:00
Copy
2022-11-12T17:38:18.717+00:00
Copy
[Container] 2022/11/12 17:38:16 Command did not exit successfully docker-compose --file=docker-compose-prod.yml push exit status 1
[Container] 2022/11/12 17:38:16 Command did not exit successfully docker-compose --file=docker-compose-prod.yml push exit status 1
2022-11-12T17:38:18.717+00:00 [Container] 2022/11/12 17:38:16 Phase complete: POST_BUILD State: FAILED
2022-11-12T17:38:18.717+00:00
Copy
[Container] 2022/11/12 17:38:16 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker-compose --file=docker-compose-prod.yml push. Reason: exit status 1
[Container] 2022/11/12 17:38:16 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker-compose --file=docker-compose-prod.yml push. Reason: exit status 1
I don't know why its failing to push.

Getting error `repository does not exist or may require 'docker login': denied: requested access to the resource is denied` in Elastic Beanstalk

While deploying dotnet app as docker with Milticontainer option in Elasticbean stalk, Getting the error like
2021-05-20 01:26:55 ERROR ECS task stopped due to: Task failed to start. (traveltouchapi: CannotPullContainerError: Error response from daemon: pull access denied for traveltouchapi, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
postgres_image: )
2021-05-20 01:26:58 ERROR Failed to start ECS task after retrying 2 times.
2021-05-20 01:27:00 ERROR [Instance: i-0844a50e307bd8b23] Command failed on instance. Return code: 1 Output: .
Environment details for: TravelTouchApi-dev3
Application name: TravelTouchApi
Region: ap-south-1
Deployed Version: app-c1ba-210520_065320
Environment ID: e-i9t6f6vszk
Platform: arn:aws:elasticbeanstalk:ap-south-1::platform/Multi-container Docker running on 64bit Amazon Linux/2.26.0
Tier: WebServer-Standard-1.0
CNAME: TravelTouchApi-dev3.ap-south-1.elasticbeanstalk.com
Updated: 2021-05-20 01:23:27.384000+00:00
My Dockerfile is
FROM mcr.microsoft.com/dotnet/core/aspnet:3.1 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS build
# Install Node.js
RUN curl -fsSL https://deb.nodesource.com/setup_14.x | bash - \
&& apt-get install -y \
nodejs \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /src/TravelTouchApi
COPY ["TravelTouchApi.csproj", "./"]
RUN dotnet restore "TravelTouchApi.csproj"
COPY . .
WORKDIR "/src/TravelTouchApi"
RUN dotnet build "TravelTouchApi.csproj" -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "TravelTouchApi.csproj" -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "TravelTouchApi.dll"]
My docker-compose.yml is
version: '3.4'
networks:
traveltouchapi-dev:
driver: bridge
services:
traveltouchapi:
image: traveltouchapi:latest
depends_on:
- "postgres_image"
build:
context: .
dockerfile: Dockerfile
ports:
- "80:80"
environment:
DB_CONNECTION_STRING: "host=postgres_image;port=5432;database=blogdb;username=bloguser;password=bloguser"
networks:
- traveltouchapi-dev
postgres_image:
image: postgres:latest
ports:
- "5432"
restart: always
volumes:
- db_volume:/var/lib/postgresql/data
environment:
POSTGRES_USER: "bloguser"
POSTGRES_PASSWORD: "bloguser"
POSTGRES_DB: "blogdb"
networks:
- traveltouchapi-dev
volumes:
db_volume:
My Dockerrun.aws.json
{
"AWSEBDockerrunVersion": 2,
"containerDefinitions": [
{
"environment": [
{
"name": "POSTGRES_USER",
"value": "bloguser"
},
{
"name": "POSTGRES_PASSWORD",
"value": "bloguser"
},
{
"name": "POSTGRES_DB",
"value": "blogdb"
}
],
"essential": true,
"image": "postgres:latest",
"memory": 200,
"mountPoints": [
{
"containerPath": "/var/lib/postgresql/data",
"sourceVolume": "Db_Volume"
}
],
"name": "postgres_image",
"portMappings": [
{
"containerPort": 5432
}
]
},
{
"environment": [
{
"name": "DB_CONNECTION_STRING",
"value": "host=postgres_image;port=5432;database=blogdb;username=bloguser;password=bloguser"
}
],
"essential": true,
"image": "traveltouchapi:latest",
"name": "traveltouchapi",
"portMappings": [
{
"containerPort": 80,
"hostPort": 80
}
],
"memory": 200
}
],
"family": "",
"volumes": [
{
"host": {
"sourcePath": "db_volume"
},
"name": "Db_Volume"
}
]
}
I think you are missing the login step before deploy the applications.
Can you try use this command before deploying?
aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_DEFAULT_ACCID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
The image name must contains with full repo/tag name 'natheesh/traveltouchapi: latest' in Dockerrun.json

elasticbeanstalk multicontainer will not pick the latest Image from

I have a python server setup of multi containers with Dockerrun.aws.json file that picks up the images from ECR:
{
"AWSEBDockerrunVersion": 2,
"volumes": [
{
"host": {
"sourcePath": "API"
},
"name": "_Api"
}
],
"containerDefinitions": [
{
"essential": true,
"Update": true,
"memory": 128,
"name": "my_api",
"image": "xxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com/my-api:test1",
"mountPoints": [
{
"containerPath": "/code",
"sourceVolume": "_Api"
}
]
},
{
"essential": true,
"memory": 128,
"name": "nginx",
"image": "xxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com/dashboard-nginx:test1",
"portMappings": [
{
"containerPort": 80,
"hostPort": 80
}
],
"links": [
"my_api"
]
},
{
"essential": true,
"memory": 128,
"name": "redis",
"image": "redis:latest",
"portMappings": [
{
"containerPort": 6379,
"hostPort": 6379
}
]
}
]
I have done some modifications to the containers and wish to test it locally using the eb local run command
But no matter what i do it is using original old images
I have a parallel docker-compose.yml i was using before EB - which works as expected:
version: '3'
services:
my_api:
image: xxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com/my-api:test1
build: ./API
expose:
- "5555"
volumes:
- ./API:/code
depends_on:
- redis
redis:
image: redis:latest
networks:
- service
ports:
- "6379:6379"
expose:
- "6379"
nginx:
image: xxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com/my-nginx:test1
build:
context: ./API
dockerfile: Dockerfile-nginx
ports:
- 80:80
depends_on:
- my_api
I tried to build and push with docker-compose . and with docker , with a new tag and more
But still i get the same behavior
The .elasticbeanstalk/docker-compose.yml file seems to get updated , but even running docker-compose up --build with it still uses the older image
I tried running docker system prune -a to make eb pull the new tagged container - but still , somehow, i got the old images again
Even deploy to AWS acts the same
When i run the docker ps -a , i can see that the containers used are only different by their names but uses different image ids:
89b258852e84 xxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com/my-nginx:test1 "nginx -g 'daemon of…" 9 minutes ago Exited (0) 5 minutes ago dashboard-nginx
23196d6e8016 xxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com/my-api:test1 "uwsgi --ini app.ini" 9 minutes ago Exited (0) 5 minutes ago dashboard-api
95b4473bc38f redis:latest "docker-entrypoint.s…" 9 minutes ago Exited (0) 5 minutes ago live_dashboard_redis_1
32be5539e905 xxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com/my-nginx:test1 "nginx -g 'daemon of…" 10 minutes ago Exited (0) 6 minutes ago elasticbeanstalk_nginx_1
51d89fcdfd94 redis:latest "docker-entrypoint.s…" 10 minutes ago Exited (0) 6 minutes ago elasticbeanstalk_redis_1
e10715455525 xxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com/my-api:test1 "uwsgi --ini app.ini" 10 minutes ago Exited (0) 6 minutes ago elasticbeanstalk_myapi_1
What have i missed or did not fully understand ?
is there any way to make eb local rebuild and use the latest images locally?
Why is not EB pull the latest version of an Image when deploying ?
Any help or suggestion would be greatly appreciated
EDIT1
Some more info i gathered
when i inspect the docker-compose image and the eb one i can see they have a diffrent Mount section which can explain the code diffrences :
docker-compose
"Mounts": [
{
"Type": "bind",
"Source": "/host_mnt/c/Workspaces/PRJ/DevOps/Tools/proj/API",
"Destination": "/code",
"Mode": "rw",
"RW": true,
"Propagation": "rprivate"
}
]
eb:
"Mounts": [
{
"Type": "volume",
"Name": "API",
"Source": "/var/lib/docker/volumes/API/_data",
"Destination": "/code",
"Driver": "local",
"Mode": "rw",
"RW": true,
"Propagation": ""
}
]
strange as i am working on windows
After few days with AWS Support on the phone
Finally we got an answer
So if anyone face this in the future , you need to check your mount setup
Make the volume pick from /var/app/current/API
"volumes": [
{
"host": {
"sourcePath": "/var/app/current/API"
},
"name": "_Api"
}
],

AWS Elastic Beanstalk gives "could not translate host name "db" to address" error

I've been trying to deploy my docker consisted of Django, Postgresql and Nginx. It works fine when I do sudo docker-compose up However when deploy it on AWS EB, it gives me
could not translate host name "db" to address: Name or service not known
What I've done is I pushed my docker to docker hub using sudo docker build -t myname/dockername -f Dockerfile . and I simply do eb deploy
File Structure
myproject
myproject
settings.py
urls.py
...
Dockerfile
Dockerrun.aws.json
manage.py
requirements.txt
...
Dockerfile
FROM python:3
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
COPY requirements.txt /code/
RUN pip install -r requirements.txt
COPY . /code/
EXPOSE 8000
CMD ["sh", "on-container-start.sh"]
Dockerrun.aws.json
{
"AWSEBDockerrunVersion": "1",
"Image": {
"Name": "myname/dockername:latest",
"Update": "true"
},
"Ports": [
{
"ContainerPort": "8000"
}
]
}
docker-compose.yml
version: '3'
services:
db:
image: postgres
hostname: db
networks:
- some_network
web:
restart: always
build: .
volumes:
- .:/code
hostname: web
expose:
- "8000"
depends_on:
- db
links:
- db:db
networks:
- some_network
nginx:
image: nginx
hostname: nginx
ports:
- "8000:8000"
volumes:
- ./config/nginx:/etc/nginx/conf.d
depends_on:
- web
networks:
- some_network
networks:
some_network:
One thing I realize is that when I use docker-compose up on my machine, I get 3 different containers running. However on EB, I see only one container running.
I think it's because I'm fetching the image from docker hub that I built with those files and that somehow caused these 3 containers to be one and it's messing up with recognizing host names? I am quite not sure still. Help will be greatly appreciated. Thanks!
Dockerrun.aws.json should correlate with docker-compose.yml
The reason of issue that host name ”db“ could not be translated to address is that the docker-compose.yml and Dockerrun.aws.json files describe a different architecture:
There are 3 containers in docker-compose.yml
There is only 1 container in Dockerrun.aws.json
Therefore, the application tries to resolve the db hostname and cannot find it, because db not declared in Dockerrun.aws.json
Fix Dockerrun.aws.json
So, update your Dockerrun.aws.json. You can do it either manually or using convenient tool micahhausler/container-transform:
a) either update it manually
You can use samples, such as:
k2works/aws-eb-docker-multi-container-sample**
b) or update it using micahhausler/container-transform
You can try micahhausler/container-transform:
Transforms docker-compose, ECS, and Marathon configurations
Transforms docker-compose, ECS, and Marathon configurations
Here is what it outputs for your case:
$ container-transform docker-compose.yml > Dockerrun.aws.json
Dockerrun.aws.json
{
"containerDefinitions": [
{
"essential": true,
"image": "postgres",
"name": "db"
},
{
"essential": true,
"image": "nginx",
"mountPoints": [
{
"containerPath": "/etc/nginx/conf.d",
"sourceVolume": "_ConfigNginx"
}
],
"name": "nginx",
"portMappings": [
{
"containerPort": 8000,
"hostPort": 8000
}
]
},
{
"essential": true,
"links": [
"db:db"
],
"mountPoints": [
{
"containerPath": "/code",
"sourceVolume": "_"
}
],
"name": "web"
}
],
"family": "",
"volumes": [
{
"host": {
"sourcePath": "."
},
"name": "_"
},
{
"host": {
"sourcePath": "./config/nginx"
},
"name": "_ConfigNginx"
}
]
}
Note:: Of course, you should fix missing settings such as memory for db and nginx containers.
You can omit networks at all
According to Networking in Compose | Docker Documentation:
For example, suppose your app is in a directory called myapp, and your docker-compose.yml looks like this:
docker-compose.yml
version: "3"
services:
web:
build: .
ports:
- "8000:8000"
db:
image: postgres
ports:
- "8001:5432"
When you run docker-compose up, the following happens:
A network called myapp_default is created.
A container is created using web’s configuration. It joins the network myapp_default under the name web.
A container is created using db’s configuration. It joins the network myapp_default under the name db.
So, since all your containers linked to the same some_network, you can omit it.
docker-compose.yml
version: '3'
services:
db:
image: postgres
hostname: db
web:
restart: always
build: .
volumes:
- .:/code
hostname: web
expose:
- "8000"
depends_on:
- db
links:
- db:db
nginx:
image: nginx
hostname: nginx
ports:
- "8000:8000"
volumes:
- ./config/nginx:/etc/nginx/conf.d
depends_on:
- web
And $ container-transform docker-compose.yml > Dockerrun.aws.json will produce:
Dockerrun.aws.json
{
"containerDefinitions": [
{
"essential": true,
"image": "postgres",
"name": "db"
},
{
"essential": true,
"image": "nginx",
"mountPoints": [
{
"containerPath": "/etc/nginx/conf.d",
"sourceVolume": "_ConfigNginx"
}
],
"name": "nginx",
"portMappings": [
{
"containerPort": 8000,
"hostPort": 8000
}
]
},
{
"essential": true,
"links": [
"db:db"
],
"mountPoints": [
{
"containerPath": "/code",
"sourceVolume": "_"
}
],
"name": "web"
}
],
"family": "",
"volumes": [
{
"host": {
"sourcePath": "."
},
"name": "_"
},
{
"host": {
"sourcePath": "./config/nginx"
},
"name": "_ConfigNginx"
}
]
}

Docker Compose env_file in Multicontainer Elastic Beanstalk

I have the following docker compose file:
version: '2'
services:
app:
build: .
command: >
bash -cex "
export LC_ALL=C.UTF-8
export LANG=C.UTF-8
/virtualenv/bin/flask run -h 0.0.0.0 -p 5050
"
env_file: env
links:
- postgres
ports:
- 8080:8080
As you can see I'm using the env_file option to load my environment variables from the file env.
Now I'm trying to deploy this container to Elastic Beanstalk.
This is my file Dockerrun.aws.json so far:
{
"AWSEBDockerrunVersion": 2,
"containerDefinitions": [
{
"name": "app",
"image": "myorg/myimage",
"essential": true,
"memory": 256,
"command": [
"/bin/bash",
"export LC_ALL=C.UTF-8",
"export LANG=C.UTF-8",
"/virtualenv/bin/flask run -h 0.0.0.0 -p 5050"
],
"portMappings": [
{
"hostPort": 8080,
"containerPort": 8080
}
],
"links": [
"postgres",
]
}
In the AWS Elastic Beanstalk documentation just mention the environment option to pass an array of env variables, but I can't find how to pass a file instead of an array of variables.
Does someone knows how to translate this docker-compose file to Dockerrun.aws.json file properly?
Regards.
Try container-transform.
$ pip install container-transform
$ cat docker-compose.yml | container-transform -v
and it will print the ECS format to STDOUT.