I have a Jenkins job and custom-built pipeline already functioning to deploy my Java application using Jboss to an EC2 server in AWS(by using the static IP of the instance). Now I plan to enable the Autoscaling feature for my EC2 instance.
What would be the best practices to make continuous deployments to these scaled EC2 instances maintaining the application's stability?
Here is the workflow I follow using Jenkins.
Create a Jenkins Pipeline that does a build and pushes to Amazon S3 - In an artifact bucket
Create a CodeDeploy application that deploys to the autoscaling group (not tag)
Create a CodePipeline application that polls for the S3 object, and when receved triggers CodeDeploy.
The great thing about CodeDeploy when attached to autoscaling groups is it creates a lifecycle hook. Whenever a new instance is deployed to the autoscaling group it will automatically deploy the latest version of code.
Related
This error is coming up when I try to deploy to my staging environment. I have a running ec2 instance and I have it correctly set to code deploy, but I see that in the error I have no replacement instances. Do I need one?
The code-pipeline path is -
Source (Github) -> CodeBuild -> Staging (codeDeploy) -> ManualApproval -> CodeBuild (for production) -> Production (codeDeploy)
Picture of the error
As per AWS CodeDeploy User Guide:
A blue/green deployment is used to update your applications while minimizing interruptions caused by the changes of a new application version. CodeDeploy provisions your new application version alongside the old version before rerouting your production traffic.
All AWS Lambda and Amazon ECS deployments are blue/green.
An EC2/On-Premises deployment can be in-place or blue/green.
You have the blue side of the deployment a.k.a. your current version of the application on your instances but you don't have the green fleet a.k.a. where the new version of the application should be deployed.
One option is to specify the green fleet and create those instances yourself (manual provisioning of instances).
Another option is that you allow CodeDeploy to provision these instances for you, which will also create an ASG (I would recommend this option as blue/green deployments are normally better). You've selected manual provision as step 1 shows Instance provisioning skipped.
Alternatively, do an in-place deployment, which is supported for EC2 instances in the cloud/onprem.
I have created a Multiservices Spring/Python project. What's the easiest way to deploy it on the AWS cloud with 4 machines?
You can use multiple Services to achieve the same :
ElasticBeanstalk: If you have you code then you upload it on ElasticBeanstalk and any newer version just upload it on the Beanstalk and choose the deployment method it will automatically be deployed on the machine. You can choose the whatever number of instances you want to spin along with LoadBalancer and more.
Documentation here
CodePipeline: Have your code pushed into CodeCommit or Github or S3 and let it use CodeCommit, CodeBuild and CodeDeploy to deploy it on your EC2 server.
Documentation here
CloudFormation: This service you can use to spin up your services just through code. It is called Infrastructure as Code. Write code and spin up the instances.
Documentation here
I've been working on a cloud formation template for my environment. I end up with a
VPC
Subnet x2
Autoscaling group
Launch configuration (EC2 instances on AWS Linux AMI)
Application load balancer
Codedeploy (for deployments)
But I incurred problem with CodeDeploy configuration with Cloud Formation, as not all features are possible for EC2 instances. After configuring manually CodeDeploy, I get an error while deploying such as "too few unhealthy instances" after which created instances are not destroyed even if rollback is enabled. I'm using right now only one EC2 instance for application, but planning in future to scale.
Is there an alternative for CodeDeploy? I'm interested to trigger deploy from Jenkins Machine.
For above your requirements, I strongly suggest that using aws elastic beanstalk is better way to deploy codes to aws. Because we could manage those in elastic beanstalk and for code deployment, use codeship is also better way to mange deployment integrated with github instead of aws code deployment.
Ensure that you have assigned the correct IAM role for the EC2 instance by going to the "Instance Settings". This will ensure that your deployment occurs smoothly without throwing that error.
You can also configure the deployment to EC2 using CodeDeploy through jenkins.
Steps to follow:
AWS CodeDeploy:
Create a new CodeDeploy application.
Enter a suitable application name and choose "EC2/On premises" as the compute pleatform.
Add a deployment group under the application. For eg: "test".
Choose in-place deployment.
Add service role as "Codedeploy development".
This will allow codedeploy to interact with other AWS services.
Choose a suitable deployment configuration preferably : "OneAtATime"
if deploying to a single EC2 instance.
Environment configuration :
Choose the EC2 instance in which you want to deploy the application
Jenkins:
On Jenkins, create a job with a suitable application name.
In the "Post Build Action" section, click on "Add Post Build Action"
Jenkins - post build configuration
Choose : "Deploy an application to AWS CodeDeploy"
Enter the CodeDeploy and S3 details in the section
S3 bucket will contain all the builds which is used to deploy onto EC2 using Codedeploy
When using AWS, it seems a nice way to deploy an application to a newly created instance is via AWS CodeDeploy. This works as follows:
Set up an auto-scaling group for the application
Write a user-data bash script for the auto-scaling group which pulls the CodeDeploy agent from S3, installs it and starts it
Set up a CodeDeploy deployment group which deploys to the auto-scaling group
Now, when an application bundle (e.g. jar or debian package) is deployed to the deployment group, it will be deployed automatically to new instances launched in the auto-scaling group.
My question is: how can this deployment strategy fit with a CI tool like Travis CI?
Specifically:
How can CodeDeploy pick up a package built by a CI tool like Travis CI? Does the build job need to upload the package to S3?
How can CodeDeploy be used to deploy the application gradually (e.g. one instance at a time)?
Does this deployment strategy require each running instance to be shut down and replaced, or is the new version of the application deployed on the existing instances? If it is the former, machine IP addresses would change during deployment, so how can other services discover the newly deployed application (i.e. without hardcoded IP addresses)?
tl;dr version:
The build job needs to upload the package to S3.
Use the one at a time deployment config.
The new version of the application is deployed on the existing instances.
Ok, here's the long version:
I recommend you try the Deployment Walkthrough or take a looks at Concepts in the documentation. It should help you get familiar with CodeDeploy faster.
You don't have to use an AutoScaling group with CodeDeploy if you don't want to. CodeDeploy with AutoScaling integration allows you to manage fleets that need to change in size dynamically separately from the code that is deployed to them, but that is not a requirement to use CodeDeploy. You can also launch some EC2 instances manually, install the host agent, and then tag them into a deployment group - but they won't get deployed to automatically on launch like the AutoScaling instances would. In either case, you can always create fleet wide deployments.
You'll have to do some work to integrate it with your CI tool. CodeDeploy doesn't directly manage your build artifacts, so your build process will need to do that. To have automatic deployments, your will need to:
Create a archive bundle with an appspec.yml, any scripts you need to handle the install/upgrade, and your build artifacts.
Upload the bundle to S3.
Create a deployment in CodeDeploy.
You might want to look at CodePipeline as an example of a continuous delivery system that's integrated with CodeDeploy.
CodeDeploy uses deployment configs to control how aggressively it deploys to the instances in your fleet. (This config gets ignored for automatic deployments, since each instance is handled separately.) CodeDeploy will fail your deployment and stop deploying to new instances if it cannot potentially fail another instance without violating the constraints in the deployment config.
There are three built in deployment configs, and you can create your own via the CLI or API if you need a different one. To deploy to only one instance at a time, you can use the CodeDeployDefault.OneAtATime deployment config, which allows at most one unhealthy host at any given time.
For anyone else (like me) looking for an example on how to actually integrate Travis-CI with CodeDeploy:
Configure the Application, DeploymentGroups and instances, as explained in the CodeDeploy walkthrough.
Use aws-cli commands to deploy your first revision successfully to the CodeDeploy target instance.
After you have the application deployed and running, configure Travis to trigger the deployments.
The CodeDeploy appspec.yml file and any scripts used for the deployment should be packaged inside your application bundle (latest.zip in the below example).
The following .travis.yml config worked for me:
script: npm run build
before_deploy:
- zip -r latest dist/*
- mkdir -p dpl_cd_upload
- mv latest.zip dpl_cd_upload/latest.zip
deploy:
- provider: s3
access_key_id: "XXXX"
secret_access_key: "YYYYY"
bucket: "deployments-bucket-name"
local_dir: dpl_cd_upload
skip_cleanup: true
- provider: codedeploy
access_key_id: "ZZZZZ"
secret_access_key: "WWWW"
bucket: "deployments-bucket-name"
key: latest.zip
bundle_type: zip
application: CodeDeployAppName
deployment_group: CodeDeployDeploymentGroupName
This examples were really useful:
https://github.com/travis-ci/cat-party/blob/master/.travis.yml
I use CodeShip to deploy my app to AWS EC2 instances when a new app version is out.
CodeShip first packages my app into .zip and puts it in S3.
Each deployment package is being generated with different file name.
When I deploy with CodeShip, they create a new revision in CodeDeploy and deploys it to all current instances.
On the one direction, every time I deploy new version it's being deployed to all my already up instances, but how can I deploy the latest revision of my app from CodeDeploy to the new born instance?
Is there any way using the aws-sdk (Ruby or cli) to achieve this?
I use OpsWorks to maintain my instances and I use custom Chef cookbook to build the environment.
Thanks
Since you have a group of EC2 instances which you wish to have the same application running, consider to use the AutoScaling Group (ASG). Then, create a deployment group based on the ASG (not EC2 instance tags). Next time when any new machine is launched in this ASG, Code Deploy will automatically run and deploys the last successful deployment package. I've been using this method for many months now and it works perfectly.
From CodeDeploy http://aws.amazon.com/codedeploy/faqs/ , if you go to the concepts section and look at question "What is a deployment group" , in the answer you will get more idea about how deployment group works.
CodeDeploy uses the tags in the deployment group to find EC2 instances when creating a new deployment. So for new born instances, basically you just need to tag them with the same tag(s) in that deployment group, and then kick off a new deployment with the revision you want. CodeDeploy will find all the EC2 instances that are tagged with the tags in the deployment group.
But you should notice this, if you want to manually start up a new EC2 instance, there are several things you need to do. You can follow the steps here: http://docs.aws.amazon.com/codedeploy/latest/userguide/how-to-prepare-instances.html to launch a instance that works with CodeDeploy.
After the instance has been tagged and set up correctly, you can just kick off a new deployment with the latest revision as the current way you do it. The revision should be deployed to the new instance by CodeDeploy automatically.