I'm using GitHub, Jenkins, AWS ECR, AWS ECS.
I want to deploy automatically when GitHub has a new commit.
When GitHub, have new commit, GitHub, sent webhook to Jenkins, Jenkins build images and push to ECR with tag 'latest'.
I wonder how can I make my ECS service restart task and redeploy images automatically when ECR image changed?
Don't use latest in this setup. Have Jenkins pick a tag for the image (maybe based off a source control commit ID, a source control tag name, or a timestamp). Give it the ability to update the ECS tasks, and then (once a build has happened and gone through appropriate pre-launch testing) have Jenkins change the image tag in the task to what it's just built. ECS will see that the image has changed, pull the new image, and launch containers accordingly.
Two other good reasons to do things this way: if you have explicit versions, you can have a pre-production cluster, deploy things there, run tests, and then deploy the same version to production; and if a deploy goes bad, you can straightforwardly roll back by manually setting the tag back to yesterday's build, which is impossible if the only version you have is latest.
Related
We have pushed our docker images (built on .net core) to AWS ECR repository. These docker images are pulled by AWS ECS (we updated task definitons, task service) and new tasks are created. The initial deployment process worked fine. But the following day after we pushed new updated docker images into the ECR repository, and created revisions of the task definitions and updated them on the service, our changes are not reflecting.
We deleted the ECR repository, ECS service and task definition and re-created all of these anew, but still the issue persists.
(we have killed the old tasks and new tasks have been started with the help of ECS service)
Requesting assistance on this.
It's hard to imagine what's going on with this background. In general if you update your task definition with a new revision and make the new revision point to the new image tag, if you deploy the new revision you should see the changes included in the code that belongs to the image with the new tag. If you see the old code is because either the new revision point to an old tag and/or your new revision has the proper new tag but you are deploying an old revision.
If you are new to ECS perhaps a good way to setup all these plumbing would be to use Copilot. You don't have to use it to deploy to production (even though you can) but it could be a good learning exercise to explore how it sets up things. Basically it's a guided way to setup your ECS cluster as well as pipelines.
I have running my cluster and task is running.
My need is want to update container image in running task in cluster how to do?
My Image is with latest tag and every time any new changes come will push to ecr on latest tag.
Deploying with the tag latest isn't a best practice because you loose a lot of visibility into what you are doing (e.g. scale out events where you deploy more tasks as part of a service will all end up using LATEST but will be effectively running different versions of the code, etc.).
This pontificating aside, you didn't say if you started your task(s) as standalone using the run-task API or if you started your task(s) as part of a service.
If the former, you need to stop your task and run it again. If the latter, you need to redeploy your service using the --force-new-deployment flag.
My goal is simple. Commit a change to the application and have it running live on AWS.
I am using Circleci and I have built all my infra with Terraform, so I want to only use Terraform to make changes to AWS. The question is, how do I update my ECS Service and keep it in sync with Terraform. The solution I came up with, not sure if I’m reinventing the wheel here, is the following:
Use circleci/aws-ecr to push the newly built image to ECR.
Use circleci/aws-ecs -> update-task-definition-from-json to update the task-definition.json
Since I have an updated task-definition.json in the Terraform dir holding the newly built image, it’s only a matter of a terraform apply. I have set it with backend so it should be possible to run it from circleci with circleci/terraform.
I should have the latest container up and running, so the first goal is achieved.
Now, I need to have it in sync. Circleci made changes to the task-definition.json and also at the terraform state. I can give an access key to my github account for the ci to commit the changes.
Is there an easier way to do this?
I know that AWS CodePipeline supports updating ECS services.
What if I want to instead update an ECS scheduled task, which does not contain a service definition?
It turns out that CodePipeline doesn't support deploying to scheduled tasks. Instead, you have to specify a static tag (such as latest) in the task definition for the ECS scheduled task, and then make sure it always pulls the latest image by setting the image pull policy accordingly. (The default policy should work, but not guarantee that a cached image will not be run if the pull fails).
Since your ECS scheduled task always uses the latest image, all you need to do in the CodePipeline is include CodeCommit and CodeBuild, then skip CodeDeploy.
Your CodeBuild should include a buildspec.yml file that builds the latest image. The CodeBuild pushes the latest image to your ECR.
So when you git push a commit to the repo, the pipeline triggers the CodeBuild which builds the new image, so next time your ECS scheduled task runs, it uses the new image from ECR.
I'm setting up automated deployments using Azure DevOps to deploy to AWS ECS running services on an EC2 instance (not Fargate). Docker images are loaded into ECR and ECS is fully setup using CloudFormation templates.
After deployments are done (new image loaded and/or any CloudFormation changes applied) I need to restart the service only if it's not running the latest version of the image that the Task Definition uses. If CloudFormation updated the service or task definition then it will automatically restart and I don't want to restart it again. Also, if the docker image was not deployed then I don't want to restart it. So, is there a way to check what version of an image a service task is currently running?
I can't check if the version of a task definition changed because most of the time it would just be the image that changes and not the task definition. I am loading the image using the Latest tag so I can just call ForceNewDeployment on the service to have it deploy the new version, but if it's possible I don't want to do that if nothing has changed.
I initially thought of trying to keep track of it in the deployment process but due to how our release pipeline is setup and some of the limits in Azure DevOps that's not something I'll be able to do.
I think the best solution, if it's possible, would be to check to see what version of an Image is currently running for each task in a service and if an image has an updated version then do a ForceNewDeployment on the service. But...is there a way to accomplish that? Or would there be a better way to do what I'm trying to do?
Edit: When I load the Docker image to ECR I'm adding two tags: "Latest" and the DevOps release number ("1234"). I was initially hoping to just query the ECR images at the end of a release and if one has a tag with the current release number then I know the image has been updated. The issue is that I don't know if the service was already restarted due to a CloudFormation change.