I know that AWS CodePipeline supports updating ECS services.
What if I want to instead update an ECS scheduled task, which does not contain a service definition?
It turns out that CodePipeline doesn't support deploying to scheduled tasks. Instead, you have to specify a static tag (such as latest) in the task definition for the ECS scheduled task, and then make sure it always pulls the latest image by setting the image pull policy accordingly. (The default policy should work, but not guarantee that a cached image will not be run if the pull fails).
Since your ECS scheduled task always uses the latest image, all you need to do in the CodePipeline is include CodeCommit and CodeBuild, then skip CodeDeploy.
Your CodeBuild should include a buildspec.yml file that builds the latest image. The CodeBuild pushes the latest image to your ECR.
So when you git push a commit to the repo, the pipeline triggers the CodeBuild which builds the new image, so next time your ECS scheduled task runs, it uses the new image from ECR.
Related
I have a react application and using AWS CodeDeploy to deploy the changes. Deployment is working fine with ECS but the Install lifecycle event takes too much time. According to the AWS Documentation:
Install – During this deployment lifecycle event, the CodeDeploy agent copies the revision files from the temporary location to the final destination folder. This event is reserved for the CodeDeploy agent and cannot be used to run scripts.
I want to debug why it's taking too long for this specific step.
I'm trying to figure out a way to come up with a CI/CD pipeline for CloudFormation. We use Cloudformation Console directly to deploy our infrastructure and app to the cloud.
Does anyone have any examples of how they have created a CI/CD pipeline using Jenkins or other types of CI tools to do some type of linting, CI, version control, and artifact deployment to Artifactory (or similar toolset)? I'd like to execute a pipeline once a new version of the cloud formation templates is uploaded to Artifactory.
You can always use CodePipeline.
see docs:
CodePipeline
CI:
I am using GitHub, so before i can merge a pull request, my code must pass 3 tests.
Those tests are 3 Codebuilds containers that run tests.
CD:
After my code merged it invoke a CodePipeline that use mainly CodeDeploy and CodeBuild.
About your goal:
I'd like to execute a pipeline once a new version of the cloud formation templates is uploaded to Artifactory.
I don't really think you need a pipeline for this.
Let assume your artifacts uploaded to s3 bucket called artifact-bucket.
You can create a CloudWatch rule that will execute StepFunctions state machine when file added to
artifact-bucket.
see docs:
Trigger StepFunctions from S3
You can easily deploy stack with StepFunctions.
I know that there are already countless questions in this direction, but unfortunately I was not able to find the right answer yet. If a post already exists, please just share the link here.
I have several gitlab CI / CD pipelines. The first pipeline uses Terraform to build the complete infrastructure for an ECS cluster based on Fargate. The second / third pipeline creates nightly builds of the frontend and the backend and pushes the Docker Image with the tag "latest" into the ECR of the (staging) AWS account.
What I now want to achieve is that the corresponding ECS tasks are redeloyed so that the latest Docker images are used. I actually thought that there is a way to do this via CloudWatch Events or whatsoever, but I don't find a really good starting point here. A workaround would be to install the AWS CLI in the CI / CD pipeline and then do a service update with "force new deployment". But that doesn't seem very elegant to me. Is there any better way here?
Conditions:
The solution must be fully automated (either in AWS or in gitlab CI / CD)
Switching to AWS CodePipeline is out of discussion
Ideally as close as possible to AWS standards. I would like to avoid extensive lambda functions that perform numerous actions due to their maintainability.
Thanks a lot!
Ok, for everybody who is interested in an answer. I solved it that way:
I execute the following AWS CLI command in the CICD pipeline
aws ecs update-service --cluster <<cluster-name>> --service <<service-name>> --force-new-deployment --region <<region>>
Not the solution I was looking for but it works.
As a general comment it is not recommended to always push the same container tag because then rolling back to a previous version in case of failure becomes really difficult.
One suitable option would be to use git tags.
Let's say you are deploying version v0.0.1
You can create a file app-version.tf which will contain the variable backend-version = v0.0.1 that you can reference on the task definition of the ecs service.
Same thing can be done for the container creation using git describe.
So, you get a new task definition for every git tag and the possibility of rolling back just by changing a value in the terraform configuration.
It is beneficial to refer to images using either digests or unique immutable tags. After the pipeline pushes the image, it could:
Grab the image's digest/unique tag
Create a new revision of the task definition
Trigger an ECS deployment with the new task definition.
As sgramo93 mentions, the big benefit is that rolling back your application can be done by deploying an older revision of the task definition.
I am building a CI pipeline with AWS CodePipeline. I'm using CodeBuild to fetch my code from a repo, build a docker image and push the image to ECR. The source for my CodePipeline is my ECR repo and is triggered when an image is updated.
Now, here’s the functionality I am looking for. When a new image is pushed to ECR, I want to create an EC2 instance and then deploy the new image to that instance. When the app in the image has completed its task, I.e done something and pushed the results to S3, I want to terminate the instance. It could take hours to days before the task is complete.
Is CodeDeploy the right tool to use to deploy the ECR image to an EC2 instance for this use case? I see from the docs that CodeDeploy requires an already running instance to deploy to. I need to create one on the fly before CodeDeploy is initiated. Should I add a step in the CodePipeline to trigger a lambda that creates an instance before CodeDeploy gets run?
Any guidance would be much appreciated!
CloudTrail supports logging a PutImage event that you can use to do stuff with your pipeline. I prefer producing artifacts after specific steps in your build pipeline and then have a lambda function that reacts to an object created event. Your lambda function could then make the necessary calls to spin up ec2 instances. Your instance could then run a job and then call lambda again, which could tear it down. It sounds like you need an on-demand worker. Services like AWS Batch or ECS might be able to provide you with this functionality out of the box.
I'm using GitHub, Jenkins, AWS ECR, AWS ECS.
I want to deploy automatically when GitHub has a new commit.
When GitHub, have new commit, GitHub, sent webhook to Jenkins, Jenkins build images and push to ECR with tag 'latest'.
I wonder how can I make my ECS service restart task and redeploy images automatically when ECR image changed?
Don't use latest in this setup. Have Jenkins pick a tag for the image (maybe based off a source control commit ID, a source control tag name, or a timestamp). Give it the ability to update the ECS tasks, and then (once a build has happened and gone through appropriate pre-launch testing) have Jenkins change the image tag in the task to what it's just built. ECS will see that the image has changed, pull the new image, and launch containers accordingly.
Two other good reasons to do things this way: if you have explicit versions, you can have a pre-production cluster, deploy things there, run tests, and then deploy the same version to production; and if a deploy goes bad, you can straightforwardly roll back by manually setting the tag back to yesterday's build, which is impossible if the only version you have is latest.