I have a react application and using AWS CodeDeploy to deploy the changes. Deployment is working fine with ECS but the Install lifecycle event takes too much time. According to the AWS Documentation:
Install – During this deployment lifecycle event, the CodeDeploy agent copies the revision files from the temporary location to the final destination folder. This event is reserved for the CodeDeploy agent and cannot be used to run scripts.
I want to debug why it's taking too long for this specific step.
Related
Elastic Beanstalk application version is updated automatically daily during specific time. Don't how it is getting triggered and source is upload. Can someone suggest how can we find why application version is updated and source is uploaded.
**Version label** **Source**
87f62452d3h673377a1d331502d8f8 class-time-service-hzavks12/program-time-service-87f62452d3h673377a1d331502d8f8-jenkins-hzavk12-branch-deploy-program-time-service-100.zip
-jenkins-hzavk12-branch-deploy-program-time-service-100
To investigate who/what triggers the deployments of your app daily during a specific time can have a look at:
event history in CloudTrial for API calls related to EB.
rules in CloudWatch Events, especially scheduled expressions which cloud execute a lambda-based or CodePiepline deployments of your application.
Jenkins setup. From the file name it seems the deployment zip originates from Jenkins. Maybe Jenkins is performing the deployment on a recurring schedule.
I have a Jenkins job and custom-built pipeline already functioning to deploy my Java application using Jboss to an EC2 server in AWS(by using the static IP of the instance). Now I plan to enable the Autoscaling feature for my EC2 instance.
What would be the best practices to make continuous deployments to these scaled EC2 instances maintaining the application's stability?
Here is the workflow I follow using Jenkins.
Create a Jenkins Pipeline that does a build and pushes to Amazon S3 - In an artifact bucket
Create a CodeDeploy application that deploys to the autoscaling group (not tag)
Create a CodePipeline application that polls for the S3 object, and when receved triggers CodeDeploy.
The great thing about CodeDeploy when attached to autoscaling groups is it creates a lifecycle hook. Whenever a new instance is deployed to the autoscaling group it will automatically deploy the latest version of code.
I am building a CI pipeline with AWS CodePipeline. I'm using CodeBuild to fetch my code from a repo, build a docker image and push the image to ECR. The source for my CodePipeline is my ECR repo and is triggered when an image is updated.
Now, here’s the functionality I am looking for. When a new image is pushed to ECR, I want to create an EC2 instance and then deploy the new image to that instance. When the app in the image has completed its task, I.e done something and pushed the results to S3, I want to terminate the instance. It could take hours to days before the task is complete.
Is CodeDeploy the right tool to use to deploy the ECR image to an EC2 instance for this use case? I see from the docs that CodeDeploy requires an already running instance to deploy to. I need to create one on the fly before CodeDeploy is initiated. Should I add a step in the CodePipeline to trigger a lambda that creates an instance before CodeDeploy gets run?
Any guidance would be much appreciated!
CloudTrail supports logging a PutImage event that you can use to do stuff with your pipeline. I prefer producing artifacts after specific steps in your build pipeline and then have a lambda function that reacts to an object created event. Your lambda function could then make the necessary calls to spin up ec2 instances. Your instance could then run a job and then call lambda again, which could tear it down. It sounds like you need an on-demand worker. Services like AWS Batch or ECS might be able to provide you with this functionality out of the box.
I know that AWS CodePipeline supports updating ECS services.
What if I want to instead update an ECS scheduled task, which does not contain a service definition?
It turns out that CodePipeline doesn't support deploying to scheduled tasks. Instead, you have to specify a static tag (such as latest) in the task definition for the ECS scheduled task, and then make sure it always pulls the latest image by setting the image pull policy accordingly. (The default policy should work, but not guarantee that a cached image will not be run if the pull fails).
Since your ECS scheduled task always uses the latest image, all you need to do in the CodePipeline is include CodeCommit and CodeBuild, then skip CodeDeploy.
Your CodeBuild should include a buildspec.yml file that builds the latest image. The CodeBuild pushes the latest image to your ECR.
So when you git push a commit to the repo, the pipeline triggers the CodeBuild which builds the new image, so next time your ECS scheduled task runs, it uses the new image from ECR.
When using AWS, it seems a nice way to deploy an application to a newly created instance is via AWS CodeDeploy. This works as follows:
Set up an auto-scaling group for the application
Write a user-data bash script for the auto-scaling group which pulls the CodeDeploy agent from S3, installs it and starts it
Set up a CodeDeploy deployment group which deploys to the auto-scaling group
Now, when an application bundle (e.g. jar or debian package) is deployed to the deployment group, it will be deployed automatically to new instances launched in the auto-scaling group.
My question is: how can this deployment strategy fit with a CI tool like Travis CI?
Specifically:
How can CodeDeploy pick up a package built by a CI tool like Travis CI? Does the build job need to upload the package to S3?
How can CodeDeploy be used to deploy the application gradually (e.g. one instance at a time)?
Does this deployment strategy require each running instance to be shut down and replaced, or is the new version of the application deployed on the existing instances? If it is the former, machine IP addresses would change during deployment, so how can other services discover the newly deployed application (i.e. without hardcoded IP addresses)?
tl;dr version:
The build job needs to upload the package to S3.
Use the one at a time deployment config.
The new version of the application is deployed on the existing instances.
Ok, here's the long version:
I recommend you try the Deployment Walkthrough or take a looks at Concepts in the documentation. It should help you get familiar with CodeDeploy faster.
You don't have to use an AutoScaling group with CodeDeploy if you don't want to. CodeDeploy with AutoScaling integration allows you to manage fleets that need to change in size dynamically separately from the code that is deployed to them, but that is not a requirement to use CodeDeploy. You can also launch some EC2 instances manually, install the host agent, and then tag them into a deployment group - but they won't get deployed to automatically on launch like the AutoScaling instances would. In either case, you can always create fleet wide deployments.
You'll have to do some work to integrate it with your CI tool. CodeDeploy doesn't directly manage your build artifacts, so your build process will need to do that. To have automatic deployments, your will need to:
Create a archive bundle with an appspec.yml, any scripts you need to handle the install/upgrade, and your build artifacts.
Upload the bundle to S3.
Create a deployment in CodeDeploy.
You might want to look at CodePipeline as an example of a continuous delivery system that's integrated with CodeDeploy.
CodeDeploy uses deployment configs to control how aggressively it deploys to the instances in your fleet. (This config gets ignored for automatic deployments, since each instance is handled separately.) CodeDeploy will fail your deployment and stop deploying to new instances if it cannot potentially fail another instance without violating the constraints in the deployment config.
There are three built in deployment configs, and you can create your own via the CLI or API if you need a different one. To deploy to only one instance at a time, you can use the CodeDeployDefault.OneAtATime deployment config, which allows at most one unhealthy host at any given time.
For anyone else (like me) looking for an example on how to actually integrate Travis-CI with CodeDeploy:
Configure the Application, DeploymentGroups and instances, as explained in the CodeDeploy walkthrough.
Use aws-cli commands to deploy your first revision successfully to the CodeDeploy target instance.
After you have the application deployed and running, configure Travis to trigger the deployments.
The CodeDeploy appspec.yml file and any scripts used for the deployment should be packaged inside your application bundle (latest.zip in the below example).
The following .travis.yml config worked for me:
script: npm run build
before_deploy:
- zip -r latest dist/*
- mkdir -p dpl_cd_upload
- mv latest.zip dpl_cd_upload/latest.zip
deploy:
- provider: s3
access_key_id: "XXXX"
secret_access_key: "YYYYY"
bucket: "deployments-bucket-name"
local_dir: dpl_cd_upload
skip_cleanup: true
- provider: codedeploy
access_key_id: "ZZZZZ"
secret_access_key: "WWWW"
bucket: "deployments-bucket-name"
key: latest.zip
bundle_type: zip
application: CodeDeployAppName
deployment_group: CodeDeployDeploymentGroupName
This examples were really useful:
https://github.com/travis-ci/cat-party/blob/master/.travis.yml