When using AWS, it seems a nice way to deploy an application to a newly created instance is via AWS CodeDeploy. This works as follows:
Set up an auto-scaling group for the application
Write a user-data bash script for the auto-scaling group which pulls the CodeDeploy agent from S3, installs it and starts it
Set up a CodeDeploy deployment group which deploys to the auto-scaling group
Now, when an application bundle (e.g. jar or debian package) is deployed to the deployment group, it will be deployed automatically to new instances launched in the auto-scaling group.
My question is: how can this deployment strategy fit with a CI tool like Travis CI?
Specifically:
How can CodeDeploy pick up a package built by a CI tool like Travis CI? Does the build job need to upload the package to S3?
How can CodeDeploy be used to deploy the application gradually (e.g. one instance at a time)?
Does this deployment strategy require each running instance to be shut down and replaced, or is the new version of the application deployed on the existing instances? If it is the former, machine IP addresses would change during deployment, so how can other services discover the newly deployed application (i.e. without hardcoded IP addresses)?
tl;dr version:
The build job needs to upload the package to S3.
Use the one at a time deployment config.
The new version of the application is deployed on the existing instances.
Ok, here's the long version:
I recommend you try the Deployment Walkthrough or take a looks at Concepts in the documentation. It should help you get familiar with CodeDeploy faster.
You don't have to use an AutoScaling group with CodeDeploy if you don't want to. CodeDeploy with AutoScaling integration allows you to manage fleets that need to change in size dynamically separately from the code that is deployed to them, but that is not a requirement to use CodeDeploy. You can also launch some EC2 instances manually, install the host agent, and then tag them into a deployment group - but they won't get deployed to automatically on launch like the AutoScaling instances would. In either case, you can always create fleet wide deployments.
You'll have to do some work to integrate it with your CI tool. CodeDeploy doesn't directly manage your build artifacts, so your build process will need to do that. To have automatic deployments, your will need to:
Create a archive bundle with an appspec.yml, any scripts you need to handle the install/upgrade, and your build artifacts.
Upload the bundle to S3.
Create a deployment in CodeDeploy.
You might want to look at CodePipeline as an example of a continuous delivery system that's integrated with CodeDeploy.
CodeDeploy uses deployment configs to control how aggressively it deploys to the instances in your fleet. (This config gets ignored for automatic deployments, since each instance is handled separately.) CodeDeploy will fail your deployment and stop deploying to new instances if it cannot potentially fail another instance without violating the constraints in the deployment config.
There are three built in deployment configs, and you can create your own via the CLI or API if you need a different one. To deploy to only one instance at a time, you can use the CodeDeployDefault.OneAtATime deployment config, which allows at most one unhealthy host at any given time.
For anyone else (like me) looking for an example on how to actually integrate Travis-CI with CodeDeploy:
Configure the Application, DeploymentGroups and instances, as explained in the CodeDeploy walkthrough.
Use aws-cli commands to deploy your first revision successfully to the CodeDeploy target instance.
After you have the application deployed and running, configure Travis to trigger the deployments.
The CodeDeploy appspec.yml file and any scripts used for the deployment should be packaged inside your application bundle (latest.zip in the below example).
The following .travis.yml config worked for me:
script: npm run build
before_deploy:
- zip -r latest dist/*
- mkdir -p dpl_cd_upload
- mv latest.zip dpl_cd_upload/latest.zip
deploy:
- provider: s3
access_key_id: "XXXX"
secret_access_key: "YYYYY"
bucket: "deployments-bucket-name"
local_dir: dpl_cd_upload
skip_cleanup: true
- provider: codedeploy
access_key_id: "ZZZZZ"
secret_access_key: "WWWW"
bucket: "deployments-bucket-name"
key: latest.zip
bundle_type: zip
application: CodeDeployAppName
deployment_group: CodeDeployDeploymentGroupName
This examples were really useful:
https://github.com/travis-ci/cat-party/blob/master/.travis.yml
Related
I have a react application and using AWS CodeDeploy to deploy the changes. Deployment is working fine with ECS but the Install lifecycle event takes too much time. According to the AWS Documentation:
Install – During this deployment lifecycle event, the CodeDeploy agent copies the revision files from the temporary location to the final destination folder. This event is reserved for the CodeDeploy agent and cannot be used to run scripts.
I want to debug why it's taking too long for this specific step.
I have a large, multi-component django application I am trying to deploy to elastic beanstalk. I am using the multi-docker environment. THis is my current workflow
Git commit triggers AWS code pipeline
AWS Codebuild builds docker image (docker-compose build), runs some tests, and pushes this image to AWS Elastic Container Registry
AWS Code Build calls eb deploy
The issue I am running into is that when I call eb deploy from my local box, the it simply upgrades the application, but when I call it from Code Build, it is upgrading the environment every time, which takes about 30 minutes for some reason
I run the deploy command with -v and confirmed that the same files are being zipped. Any ideas on what is going on here, is my setup incorrect?
I also tried to deploy the application from Code Deploy in the pipeline and can confirm that it also always upgrades the entire environement.
I think that if you use CB to update your EB env, it just replaces it as it is being considered as a new environment. In your local workstation you are using only one single environment, but with new application version.
I would consider replacing CB for updating your EB environment, with the EB deploy provider in your CP. This should successful just upload your new application version to an existing EB environment.
CP natively supports a number of deploy action providers, one of the being Elastic Beanstalk:
You can configure CodePipeline to use Elastic Beanstalk to deploy your code. You can create the Elastic Beanstalk application and environment to use in a deploy action in a stage either before you create the pipeline or when you use the Create Pipeline wizard.
I have a Jenkins job and custom-built pipeline already functioning to deploy my Java application using Jboss to an EC2 server in AWS(by using the static IP of the instance). Now I plan to enable the Autoscaling feature for my EC2 instance.
What would be the best practices to make continuous deployments to these scaled EC2 instances maintaining the application's stability?
Here is the workflow I follow using Jenkins.
Create a Jenkins Pipeline that does a build and pushes to Amazon S3 - In an artifact bucket
Create a CodeDeploy application that deploys to the autoscaling group (not tag)
Create a CodePipeline application that polls for the S3 object, and when receved triggers CodeDeploy.
The great thing about CodeDeploy when attached to autoscaling groups is it creates a lifecycle hook. Whenever a new instance is deployed to the autoscaling group it will automatically deploy the latest version of code.
I use CodeShip to deploy my app to AWS EC2 instances when a new app version is out.
CodeShip first packages my app into .zip and puts it in S3.
Each deployment package is being generated with different file name.
When I deploy with CodeShip, they create a new revision in CodeDeploy and deploys it to all current instances.
On the one direction, every time I deploy new version it's being deployed to all my already up instances, but how can I deploy the latest revision of my app from CodeDeploy to the new born instance?
Is there any way using the aws-sdk (Ruby or cli) to achieve this?
I use OpsWorks to maintain my instances and I use custom Chef cookbook to build the environment.
Thanks
Since you have a group of EC2 instances which you wish to have the same application running, consider to use the AutoScaling Group (ASG). Then, create a deployment group based on the ASG (not EC2 instance tags). Next time when any new machine is launched in this ASG, Code Deploy will automatically run and deploys the last successful deployment package. I've been using this method for many months now and it works perfectly.
From CodeDeploy http://aws.amazon.com/codedeploy/faqs/ , if you go to the concepts section and look at question "What is a deployment group" , in the answer you will get more idea about how deployment group works.
CodeDeploy uses the tags in the deployment group to find EC2 instances when creating a new deployment. So for new born instances, basically you just need to tag them with the same tag(s) in that deployment group, and then kick off a new deployment with the revision you want. CodeDeploy will find all the EC2 instances that are tagged with the tags in the deployment group.
But you should notice this, if you want to manually start up a new EC2 instance, there are several things you need to do. You can follow the steps here: http://docs.aws.amazon.com/codedeploy/latest/userguide/how-to-prepare-instances.html to launch a instance that works with CodeDeploy.
After the instance has been tagged and set up correctly, you can just kick off a new deployment with the latest revision as the current way you do it. The revision should be deployed to the new instance by CodeDeploy automatically.
I want to autoscale the infrastructure when load gets high. I am running my infrastructure on AWS. I have a requirement where I need to pull the application code from Github when autoscaling happens. As the code changes frequently, we can't take an AMI and launch an instance from that AMI. So I want to pull the latest code from repositories. AWS just launched a service called AWS CodeDeploy. How can I use this service to automate the process of pulling the code when the instances start?
P.S. I have written an init script to automatically attach an EIP, whitelist that IP on different Security Groups and put the instance under a load-balancer when the instance boots and revoking everything when instance is terminated in autoscaling.
CodeDeploy is a great solution to your problem. If configured correctly, it can automatically deploy to new EC2 instances that are spun up by Autoscaling. To get this working you'll need three things:
An Auto Scaling group that launches instances with the latest host agent installed and running.
A Code Deploy deployment group configured to deploy to that Auto Scaling group (as a Auto Scaling group not by tags).
A target revision to deploy automatically to new instances. If there is no target revision, Code Deploy will see the instance launch, but won't deploy anything.
Here's a tutorial you can use to help get started: Tutorial: Using AWS CodeDeploy to Deploy an Application to an Auto Scaling Group. That tutorial will walk you through baking an AMI with the agent installed and setting up the deployment group to deploy your code to new instances.
If you do bake a AMI with the agent pre-installed, you would need to update that image regularly with agent releases. Once the agent is installed it will update itself, but Auto Scaling might fail your instance launches if the agent version is no longer supported by Code Deploy. For actual production use, I would recommend not baking an AMI and instead installing the latest agent when your instances are launched. (The tutorial should be updated to use this method soon.)
You can setup your instances to automatically download and run the latest installer on boot. Essentially, you paste in a shell script as user data when creating the Auto Scaling group.
For example, I tested the following script on Amazon Linux (taken from Set Up a New Amazon EC2 Instance to Work with AWS CodeDeploy):
#!/bin/bash
yum -y update
yum install -y aws-cli
cd /home/ec2-user
aws s3 cp s3://aws-codedeploy-us-east-1/latest/install . --region us-east-1
chmod +x ./install
./install auto
You should be able to paste this in as user data when you are creating the Auto Scaling group. For Auto Scaling, you set that up when creating the launch configuration under configure details -> advanced details.
To set up the deployment group and set the target revision:
Create or edit a deployment group to include the Auto Scaling group. (Note: the Auto Scaling group must exist first for this to work.)
Make sure there is at least 1 instance in the Auto Scaling group.
Deploy the artifact bundle you want to be automatically deployed to the deployment group.
Assuming step 3 was successful, the deployment group will now have that bundle set as it's target revision. For automatic deployments, this becomes the known good revision to deploy.
New instances that are launched by Auto Scaling in that Auto Scaling group will have the target revision of the deployment group automatically deployed to them. Revisions from failed manual deployments won't be automatically deployed automatically.