I have a ECS cluster in AWS which uses the EC2 launch type, with a Task Definition which runs a container image my-app:latest which it pulls from a ECR repository. I've configured a Service that runs this Task Definition. Now, whenever I push a new image, I want it to run the new image.
So I saw in this thread How do I deploy updated Docker images to Amazon ECS tasks?
That I can use the aws ecs update-service command. The problem is that I can't get the container instance to pull the new image.
I saw in the docs in Container agent configuration that there is the ECS_IMAGE_PULL_BEHAVIOR configuration, which you can se to always, so when launching the EC2 container instance I've configured the user_data as such:
#!/bin/bash
echo "ECS_CLUSTER=my-cluster" >> /etc/ecs/ecs.config
echo "ECS_IMAGE_PULL_BEHAVIOR=always" >> /etc/ecs/ecs.config
Now after pushing a new image to ECR, I run the aws ecs update-service command, but it doesn't seem to pull the new image...
Can someone help?
Related
I am currently using ECS to deploy a web application. This Application is pushed on my ECR with the latest tag but putting new images doesn't seem to change the code at deployment.
I tried to use the answer found here How do I deploy updated Docker images to Amazon ECS tasks? using aws ecs update-service --cluster <cluster name> --service <service name> --force-new-deployment.
I also put ECS_IMAGE_PULL_BEHAVIOR=always in my ecs config and deployment_minimum_healthy_percent = 0 and deployment_maximum_percent = 200 in my deployment settings.
I notice that the image digest is matching the latest image, but by logging on the container, the code is not different from the previous version. But by executing docker pull <my_image:latest> the changes are there.
You need to update task definition first (create new one) and then deploy it.
ECS task definition
CLI for register task
I have ec2 instance for testing. I deployed using OpsWorks, and now I'm making new job on Jenkins to deploy automatically. what I want to do is
when someone push to branch
Jenkins server build docker image
push image to ecr
ec2 instance pull ecr image and build docker container and run
I made a job that using ecr and deploy ECS Fargate, but never done using ecr and deploy pre existed ec2 instance.I wonder this is possible to make it.
Pre-requisite
On your EC2 you first have to install docker.
There are many ways you can do it.
Once Jenkin build & push docker image to ECR you can further add the step in Jenkin build steps. Jenkin will do SSH inside EC2 and pull and run the docker image.
Once Jenkin build & push docker image to ECR you can further add the step in Jenkin build steps. Jenkin will trigger shell script file on EC2. That sh file having all logic to pull the latest one and stop existing etc.
From Jenkins also you can do it via ansible script.
The use case is like - developer makes some code changes and the below things happen automatically -
build runs, application artifact created, docker image generated with the artifact, image pushed to Docker registry, AWS ECS tasks and ECS services updated.
I want to know what are the ways to achieve the above automation of update of AWS ECS services. Till now I have implemented AWS ECS update from Jenkins build using -
1>run post build AWS CLi scripts from Jenkins to update ECS
2>post build action or pipeline step to invoke AWS Lambda function. I have created one Lambda function in Java to implement that.
Please let me the other ways we can achieve the above. Thanks.
I'm continuously deploying Docker containers from CircleCI to AWS ECS.
The outline of the deployment flow is as follows:
Build and tag a new Docker image
Login to AWS ECR and push the image
Update task definitions and services of ECS with ecs-deploy
ecs-deploy is a useful script that updates Docker images in ECS.
https://github.com/silinternational/ecs-deploy
You could use a shell script that calls aws cli commands to create cloudformation stacks or directly call the create commands in the aws cli for the ECR repository, Task Definition, Events rule and target(for scheduling).
then you just call this script on your terminal using this command: ./setup.sh and it should execute all your commands at once.
aws ecr create-repository \
--repository-name tasks-${TASK_NAME}-${TASK_ENV} \
;
or if you want to set up your resources via cloudformation templates, you can launch them using this command as long as the template exists at file://name.yml:
aws cloudformation create-stack \
--stack-name stack-name \
--capabilities CAPABILITY_IAM \
--template-body file://name.yml \
--parameters
ParameterKey=ParamName,ParameterValue=${PARAM_NAME} \
;
Take a look at Codefresh - https://docs.codefresh.io/docs/amazon-ecs
You can build your pipeline
Build Step
Push to Registry
Deply to ECS
That easy
While there are a ton of CI/CD tools out there, since I am early in my rollout, I decided to write a small script instead of having CI/CD pipelines do it.
Here is a one-click deploy script I wrote using the ecs-deploy script as a dependency to achieve a rolling deploy of a docker image to ECS.
You can run this locally from your dev or build/deployment box or use Jenkins or some local build tool.
#!/bin/bash
# automatically login to AWS
eval $(aws ecr get-login)
# build local docker image and push repo to AWS
docker build -t <yourlocaldockerimagetag> .
docker tag <yourlocaldockerimagetag>:latest <yourECSRepoURL>:latest
docker -D -l debug push <yourECSRepoURL>:latest
# deploy to ECS
ecs-deploy/ecs-deploy -m 50 -k <access-key> -s <secret-key> -r <aws-region> -c <cluster-name> -n <service-name> -i <yourECSRepoURL>:latest
Parameters:
cluster-name: Your cluster name in ECS
service-name: Your service name that you had created in ECS
yourECSRepoURL: ECS Repository URL
yourlocaldockerimagetag: Any local image tag name
access-key: your AWS access key for deployments
secret-key: your AWS secret key
Make sure you install ecs-deploy before this script.
The -m 50 tells it that it can deploy even if the number of nodes drops to 50%. Ideally you would have an extra node to do deployments, but if you can't afford that setting this would ensure that deployments continue to happen.
If you are also using an ELB (load balancer), then the default deregistration delay for target groups is 5 minutes which is a bit excessive. The deregistration delay is the time to wait for existing requests to complete BEFORE ECS sends a SIGTERM or SIGINT to your docker container. You should lower this by going to the Target Groups in EC2 dashboard and click the Edit Attributes to edit it. Otherwise your deployments may take forever.
I think nobody has mentioned CodePipeline from AWS, it really integrates easilly with many AWS Services including ECS and CodeCommit:
Push commit to CodeCommit Repo, triggering the pipeline execution.
(Optional) Configure a Manual Approval step that needs you to take an action before Build.
Run a CodeBuild Project that builds your Dockerfile and push the image to an ECR Repo.
Run a "Deploy" step that deploys to a specific ECS Service. It updates the services with a new Task Definition that points to the new ECR Image.
I have used this flow with BitBucket also, just configure a BitBucket pipeline that pushes all new code to a CodeCommit Repo as a previous step.
Exactly as #minamiyojo and #astav answers, we ended up glueing ecs-deploy with a template engine to power up our CD pipeline with some reusable component, we just open-sourced as well:
https://github.com/GuccioGucci/yoke
Please refer to Motivation section in README, hope this would help your scenario too.
I am setting up a CI/CD pipeline for my micro-services. Currently I use TravisCI to pull the code from Github upon check-in, build the docker image and push it to DockerHub. I tried using docker cloud(previously knows as Tutum), which provides automatic deployment feature to AWS EC2 instance but the deployment sometimes recreates the container and the service endpoint URL changes, which is not desirable.
I am exploring amazon's ECS and its tasks , but I can not find any reference for how to setup continuos deployment to ECS when a new image is pushed to docker hub.
Anybody has any experience doing the setup ?
with ECS you would basically have CI detect a change to docker hub and update your task definition/service.
For this I use the wonderful ecs-deploy script from here:
https://github.com/silinternational/ecs-deploy
After my container has been built and deployed to dockerhub it's simply a matter of:
ecs-deploy -k $AWS_KEY -s $AWS_SECRET -r $AWS_REGION -c $CLUSTER_NAME -n $SERVICE_NAME -i $DOCKER_IMAGE_NAME
and that does it.
I’m new to both Amazon’s ECS and docker, and I don’t know how to deploy new images.
I currently create a new image in ECR with
NAME_TAG=my-image-name:my-tag-v1
ECR=my-acct-number.dkr.ecr.us-east-1.amazonaws.com
docker build -t $NAME_TAG .
docker tag -f $NAME_TAG $ECR/$NAME_TAG
$(aws ecr get-login --region us-east-1) #log in
docker push $ECR/$NAME_TAG
At this point I don't know how to deploy the new container from ECR to my cluster.
I created the cluster, task and service using a Cloud Formation template, but updating the TaskDefinition image to $ECR/$NAME_TAG and running a stack update eventually times out and fails with a “service did not stabilize” error.
If I push to my-image-name:latest, my cluster instances do pull down the new image, but they don’t run it, and in any case I want to avoid using the mysterious latest tag.
How am I supposed to deploy new images to ECS?
You should be able to deploy your image using a new task definition every time you deploy.
The task definition lets you set the image version using the attribute "image"
"image":"my-acct-number.dkr.ecr.us-east-1.amazonaws.com/my-image-name:my-tag-v1"
In case you want to use only one task definition, you will have to build you image and tag it with whatever is defined in the the definition.