This error is coming up when I try to deploy to my staging environment. I have a running ec2 instance and I have it correctly set to code deploy, but I see that in the error I have no replacement instances. Do I need one?
The code-pipeline path is -
Source (Github) -> CodeBuild -> Staging (codeDeploy) -> ManualApproval -> CodeBuild (for production) -> Production (codeDeploy)
Picture of the error
As per AWS CodeDeploy User Guide:
A blue/green deployment is used to update your applications while minimizing interruptions caused by the changes of a new application version. CodeDeploy provisions your new application version alongside the old version before rerouting your production traffic.
All AWS Lambda and Amazon ECS deployments are blue/green.
An EC2/On-Premises deployment can be in-place or blue/green.
You have the blue side of the deployment a.k.a. your current version of the application on your instances but you don't have the green fleet a.k.a. where the new version of the application should be deployed.
One option is to specify the green fleet and create those instances yourself (manual provisioning of instances).
Another option is that you allow CodeDeploy to provision these instances for you, which will also create an ASG (I would recommend this option as blue/green deployments are normally better). You've selected manual provision as step 1 shows Instance provisioning skipped.
Alternatively, do an in-place deployment, which is supported for EC2 instances in the cloud/onprem.
Related
I have a Jenkins job and custom-built pipeline already functioning to deploy my Java application using Jboss to an EC2 server in AWS(by using the static IP of the instance). Now I plan to enable the Autoscaling feature for my EC2 instance.
What would be the best practices to make continuous deployments to these scaled EC2 instances maintaining the application's stability?
Here is the workflow I follow using Jenkins.
Create a Jenkins Pipeline that does a build and pushes to Amazon S3 - In an artifact bucket
Create a CodeDeploy application that deploys to the autoscaling group (not tag)
Create a CodePipeline application that polls for the S3 object, and when receved triggers CodeDeploy.
The great thing about CodeDeploy when attached to autoscaling groups is it creates a lifecycle hook. Whenever a new instance is deployed to the autoscaling group it will automatically deploy the latest version of code.
So I have an auto scale group setup behind a loadbalancer. I have circle ci setup to run tests and then push to the auto scale group on aws. But for the life of me I cannot find how to tell the auto scale group how to pull the latest commit from master branch when spinning up new instances. Any help is appreciated. Thanks!
It actually does it automatically by the code deploy application you set for auto scale group. If anyone ever searches for this.
First, Integrating CodeDeploy with Amazon EC2 Auto Scaling
When new Amazon EC2 instances are launched as part of an Amazon EC2 Auto Scaling group, CodeDeploy can deploy your revisions to the new instances automatically. You can also coordinate deployments in CodeDeploy with Amazon EC2 Auto Scaling instances registered with Elastic Load Balancing load balancers
See
https://docs.aws.amazon.com/codedeploy/latest/userguide/integrations-aws-auto-scaling.html
Then, give CodeDeploy the commit id of the commit you want to deploy, via the CodeDeploy application console, in the deployment settings.
Click Connect With GitHub, and then step through the OAuth process. A few different things might happen next. First, if you are not logged into GitHub in your browser, you will be asked to log in. Next, if you haven’t already granted AWS CodeDeploy access to your GitHub repositories, you will be asked to authorize that now. Once this is done, you’ll return to the AWS Management Console and CodeDeploy will have the permissions required to access your repository. All that’s left is to fill in the Repository Name and Commit ID. The repository name will be in the format “GITHUB_USERNAME/REPOSITORY_NAME”. The commit ID will be the full SHA (a 40-digit hex number) that can be copied through the GitHub UI. You can find this information from the commit history page of your repository.
https://aws.amazon.com/blogs/devops/automatically-deploy-from-github-using-aws-codedeploy/
When configured as above, instances that meet the deployment who criteria will automatically receive a new deployment on first launch and of course on subsequent manual deployments.
I've been working on a cloud formation template for my environment. I end up with a
VPC
Subnet x2
Autoscaling group
Launch configuration (EC2 instances on AWS Linux AMI)
Application load balancer
Codedeploy (for deployments)
But I incurred problem with CodeDeploy configuration with Cloud Formation, as not all features are possible for EC2 instances. After configuring manually CodeDeploy, I get an error while deploying such as "too few unhealthy instances" after which created instances are not destroyed even if rollback is enabled. I'm using right now only one EC2 instance for application, but planning in future to scale.
Is there an alternative for CodeDeploy? I'm interested to trigger deploy from Jenkins Machine.
For above your requirements, I strongly suggest that using aws elastic beanstalk is better way to deploy codes to aws. Because we could manage those in elastic beanstalk and for code deployment, use codeship is also better way to mange deployment integrated with github instead of aws code deployment.
Ensure that you have assigned the correct IAM role for the EC2 instance by going to the "Instance Settings". This will ensure that your deployment occurs smoothly without throwing that error.
You can also configure the deployment to EC2 using CodeDeploy through jenkins.
Steps to follow:
AWS CodeDeploy:
Create a new CodeDeploy application.
Enter a suitable application name and choose "EC2/On premises" as the compute pleatform.
Add a deployment group under the application. For eg: "test".
Choose in-place deployment.
Add service role as "Codedeploy development".
This will allow codedeploy to interact with other AWS services.
Choose a suitable deployment configuration preferably : "OneAtATime"
if deploying to a single EC2 instance.
Environment configuration :
Choose the EC2 instance in which you want to deploy the application
Jenkins:
On Jenkins, create a job with a suitable application name.
In the "Post Build Action" section, click on "Add Post Build Action"
Jenkins - post build configuration
Choose : "Deploy an application to AWS CodeDeploy"
Enter the CodeDeploy and S3 details in the section
S3 bucket will contain all the builds which is used to deploy onto EC2 using Codedeploy
When using AWS, it seems a nice way to deploy an application to a newly created instance is via AWS CodeDeploy. This works as follows:
Set up an auto-scaling group for the application
Write a user-data bash script for the auto-scaling group which pulls the CodeDeploy agent from S3, installs it and starts it
Set up a CodeDeploy deployment group which deploys to the auto-scaling group
Now, when an application bundle (e.g. jar or debian package) is deployed to the deployment group, it will be deployed automatically to new instances launched in the auto-scaling group.
My question is: how can this deployment strategy fit with a CI tool like Travis CI?
Specifically:
How can CodeDeploy pick up a package built by a CI tool like Travis CI? Does the build job need to upload the package to S3?
How can CodeDeploy be used to deploy the application gradually (e.g. one instance at a time)?
Does this deployment strategy require each running instance to be shut down and replaced, or is the new version of the application deployed on the existing instances? If it is the former, machine IP addresses would change during deployment, so how can other services discover the newly deployed application (i.e. without hardcoded IP addresses)?
tl;dr version:
The build job needs to upload the package to S3.
Use the one at a time deployment config.
The new version of the application is deployed on the existing instances.
Ok, here's the long version:
I recommend you try the Deployment Walkthrough or take a looks at Concepts in the documentation. It should help you get familiar with CodeDeploy faster.
You don't have to use an AutoScaling group with CodeDeploy if you don't want to. CodeDeploy with AutoScaling integration allows you to manage fleets that need to change in size dynamically separately from the code that is deployed to them, but that is not a requirement to use CodeDeploy. You can also launch some EC2 instances manually, install the host agent, and then tag them into a deployment group - but they won't get deployed to automatically on launch like the AutoScaling instances would. In either case, you can always create fleet wide deployments.
You'll have to do some work to integrate it with your CI tool. CodeDeploy doesn't directly manage your build artifacts, so your build process will need to do that. To have automatic deployments, your will need to:
Create a archive bundle with an appspec.yml, any scripts you need to handle the install/upgrade, and your build artifacts.
Upload the bundle to S3.
Create a deployment in CodeDeploy.
You might want to look at CodePipeline as an example of a continuous delivery system that's integrated with CodeDeploy.
CodeDeploy uses deployment configs to control how aggressively it deploys to the instances in your fleet. (This config gets ignored for automatic deployments, since each instance is handled separately.) CodeDeploy will fail your deployment and stop deploying to new instances if it cannot potentially fail another instance without violating the constraints in the deployment config.
There are three built in deployment configs, and you can create your own via the CLI or API if you need a different one. To deploy to only one instance at a time, you can use the CodeDeployDefault.OneAtATime deployment config, which allows at most one unhealthy host at any given time.
For anyone else (like me) looking for an example on how to actually integrate Travis-CI with CodeDeploy:
Configure the Application, DeploymentGroups and instances, as explained in the CodeDeploy walkthrough.
Use aws-cli commands to deploy your first revision successfully to the CodeDeploy target instance.
After you have the application deployed and running, configure Travis to trigger the deployments.
The CodeDeploy appspec.yml file and any scripts used for the deployment should be packaged inside your application bundle (latest.zip in the below example).
The following .travis.yml config worked for me:
script: npm run build
before_deploy:
- zip -r latest dist/*
- mkdir -p dpl_cd_upload
- mv latest.zip dpl_cd_upload/latest.zip
deploy:
- provider: s3
access_key_id: "XXXX"
secret_access_key: "YYYYY"
bucket: "deployments-bucket-name"
local_dir: dpl_cd_upload
skip_cleanup: true
- provider: codedeploy
access_key_id: "ZZZZZ"
secret_access_key: "WWWW"
bucket: "deployments-bucket-name"
key: latest.zip
bundle_type: zip
application: CodeDeployAppName
deployment_group: CodeDeployDeploymentGroupName
This examples were really useful:
https://github.com/travis-ci/cat-party/blob/master/.travis.yml
I want to autoscale the infrastructure when load gets high. I am running my infrastructure on AWS. I have a requirement where I need to pull the application code from Github when autoscaling happens. As the code changes frequently, we can't take an AMI and launch an instance from that AMI. So I want to pull the latest code from repositories. AWS just launched a service called AWS CodeDeploy. How can I use this service to automate the process of pulling the code when the instances start?
P.S. I have written an init script to automatically attach an EIP, whitelist that IP on different Security Groups and put the instance under a load-balancer when the instance boots and revoking everything when instance is terminated in autoscaling.
CodeDeploy is a great solution to your problem. If configured correctly, it can automatically deploy to new EC2 instances that are spun up by Autoscaling. To get this working you'll need three things:
An Auto Scaling group that launches instances with the latest host agent installed and running.
A Code Deploy deployment group configured to deploy to that Auto Scaling group (as a Auto Scaling group not by tags).
A target revision to deploy automatically to new instances. If there is no target revision, Code Deploy will see the instance launch, but won't deploy anything.
Here's a tutorial you can use to help get started: Tutorial: Using AWS CodeDeploy to Deploy an Application to an Auto Scaling Group. That tutorial will walk you through baking an AMI with the agent installed and setting up the deployment group to deploy your code to new instances.
If you do bake a AMI with the agent pre-installed, you would need to update that image regularly with agent releases. Once the agent is installed it will update itself, but Auto Scaling might fail your instance launches if the agent version is no longer supported by Code Deploy. For actual production use, I would recommend not baking an AMI and instead installing the latest agent when your instances are launched. (The tutorial should be updated to use this method soon.)
You can setup your instances to automatically download and run the latest installer on boot. Essentially, you paste in a shell script as user data when creating the Auto Scaling group.
For example, I tested the following script on Amazon Linux (taken from Set Up a New Amazon EC2 Instance to Work with AWS CodeDeploy):
#!/bin/bash
yum -y update
yum install -y aws-cli
cd /home/ec2-user
aws s3 cp s3://aws-codedeploy-us-east-1/latest/install . --region us-east-1
chmod +x ./install
./install auto
You should be able to paste this in as user data when you are creating the Auto Scaling group. For Auto Scaling, you set that up when creating the launch configuration under configure details -> advanced details.
To set up the deployment group and set the target revision:
Create or edit a deployment group to include the Auto Scaling group. (Note: the Auto Scaling group must exist first for this to work.)
Make sure there is at least 1 instance in the Auto Scaling group.
Deploy the artifact bundle you want to be automatically deployed to the deployment group.
Assuming step 3 was successful, the deployment group will now have that bundle set as it's target revision. For automatic deployments, this becomes the known good revision to deploy.
New instances that are launched by Auto Scaling in that Auto Scaling group will have the target revision of the deployment group automatically deployed to them. Revisions from failed manual deployments won't be automatically deployed automatically.