We use AWS cloudformation service to initialize our stack, and set up the auto scaling service to bring up new app servers when load is rising.
My understanding is that Auto Scaling can only start predefined AMI as new instances. These instances could be different from other running instances, because we may have updated packages/source code deployed on those instances.
How can I bring the new instances up-to-date?
Should I update the AMIs everytime I deploy something new to the running instances? Or is there anyway to trigger auto-deployment on new instances (Opsworks) when auto scaling?
I am new to AWS, so pardon me if my question is rudimentary.
There are multiple ways of doing this. My preferred approach is never to touch the servers directly, but instead create a new AMI whenever I deploy a new version of the software.
To do this, use the AutoScalingRollingUpdate property for the auto-scaling group. When you then change ImageId for the launch configuration, AWS will automatically replace your old servers with new ones as a rolling upgrade.
I have a simple deploy script that creates a new AMI, replaces ImageId in the template, and then does a stack update - AWS takes care of the rest.
When creating EC2 instances from Beanstalk, it automatically creates a AutoScaling Group and Launch Configuration based on the specified environment selections. Creating the instance from base AMI is done using a custom code call user data which includes the shell script to create folders and install relevant software.
You can add a new shell scripts or commands there to do your custom work before starting a new instance. This way it is much simpler. e.g. you can run yum update before starting a instance
To find user data section
Go to EC2 Console -> Go to launch configurations section (on left) -> Select the correct launch configuration and copy it -> Click view user data -> Add your scripts and commands as required -> Modify the relevant Auto Scaling group to point to the new launch configuration
Related
I have completely understood the concept of Auto-Scaling in AWS. My only question is, what AMI will the launch configuration use while in production environment?
According to my understanding Image of existing instance should be used. Lets say I have used an image of existing instance.
What if there are any changes in existing instance in future? In this scenario we have to update the AMI.
Is there any process to automate this process?
When you create new AMI and set it in a new launch configuration (LC; LC can't be edited) or new version of a launch template (LT), then you will have to update the ASG configuration with the new LC/LT.
However, ASG by default will not update existing instances with new LC/LT. Only new instance that ASG launches will have the new LC/LT, and subsequently, the new AMI. Therefore, you will end up with ASG in which part of instances is running old AMI, and the other part is running new AMI.
You can deal with this in two commonly used ways:
Create your LC/LT and ASG using CloudFormation and specify UpdatePolicy. The update policy will be triggered when LC/LT changes, and existing instances in ASG will be updated based on the rules you specify in the policy.
Perform blue/green deployment of your ASG. How to perform the deployment is described and explained in details in an excellent AWS white paper:
Blue/Green Deployments on AWS
Auto scaling uses AMIs which are a point in time snapshot of your instance. Any changes made thereafter will not be applied to the AMI.
If you want any change to your base image you will need to recreate an image and roll it out across your Launch Configuration/Launch Template again.
There are many tools people use to provision the configuration of instances for AMIs such as Ansible, Chef and Puppet.
AWS also launched an automation tool for building images last year, the EC2 Image Builder
For some additional reading take a look at the golden ami pipeline.
I have setup a process for installing and setting up my application stack on a ubuntu base image and then creating an AMI from the machine. All of this is automated using packer. Now once I have the ami-id once packer is done I manually create a new launch configuration and update my ASG configuration and then schedule scale up and scale down action which gets rid of the old instances.
So what I am looking for is:
Please suggest a better way to update my application stack whenever there is a new update for any software version (e.g ngnix ruby etc)
How can I automate the roll out process so the new ami is picked up and old instance should degrade.
There is a good strategy for this in Faster Auto Scaling in AWS CloudFormation Stacks with Lambda-backed Custom Resources
To orchestrate this process, you bootstrap a reference instance with a
user data script, use wait conditions to trigger an AMI capture, and
finally create an Auto Scaling group launch configuration that
references the newly created AMI. The reference instance that is used
to capture the AMI can then be terminated, or it can be repurposed for
administrative access or for performing scheduled tasks.
The process does not use Packer and does not require a dedicated server for creating the AMI, and instead uses a Lambda-backed custom resource.
Second Option
As you already have your AMI creation in Packer working, you should consider using Lambda to copy your existing Launch Configuration with the updated AMI. You can see a good approach to this from Patch an AMI and Update an Auto Scaling Group:
The following example builds on the Simplify AMI Patching Using
Automation, Lambda, and Parameter Store example by adding a step that
updates an Auto Scaling group with the newly-patched AMI. This
approach ensures that new images are automatically made available to
different computing environments that use Auto Scaling groups.
The final step of the Automation workflow in this example uses an AWS
Lambda function to copy an existing launch configuration and set the
AMI ID to the newly-patched AMI. The Auto Scaling group is then
updated with the new launch configuration. In this type of Auto
Scaling scenario, users could terminate existing instances in the Auto
Scaling group to force a new instance to launch that uses the new
image. Or, users could wait and allow scale-in or scale-out events to
naturally launch newer instances.
In my auto scaling group I want to copy my existing AMI and copy it to bring up a new instance.
I can easily use a script to create the new instance but the developers change the code without committing to git hub so the new instance will not have the changes.So bootstrap or user script is out of the windows.
Creating a snapshot and making an instance manually is been ruled out.
I there any way using AWS CLI or SDK so that I can copy an existing running AMI and bring it up in auto scaling group.
I want to modify / update the user-data for an EC2 instance. This is attached to an autoscaling cluster.
I understand that the instance needs to be stopped before the user-data can be updated. The problem I am facing is, when I stop the instance to update user-data autoscaler automatically brings a new instance back up.
Is there a way to update user-data without removing the EC2 instance from the autoscaling group?
For instances in an autoscaling group, the user data is generally updated by creating a new launch configuration with your new user data.
Your AutoScaling group should be associated with a launch configuration already. There is an easy option to copy launch configurations from the AWS web console that will replicate all of your existing options. Simply find this launch configuration, copy it, and then replace the old user data before you save the new configuration.
Once the new launch configuration is created, apply it to your autoscaling group. You can begin using it immediately by increasing the desired size of the group to launch a new instance with the new configuration, and then detach the old instance once you're satisfied that the new instance (and any hosted applications) are operational.
You can likewise use this method to change any property of a launch configuration without causing an interruption to your application.
Further Resources:
AWS Documentation - Creating a Launch Configuration
The only way can achieve this is by disabling autoscaling temporarily using programmatic invocation using aws sdk.
You can restart the servers after the autoscaling is disabled.
(node API http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/AutoScaling.html#suspendProcesses-property)
I have created Auto scaling group in AWS using a customized AMI. Now to rollout my new code I need to either update all instances running but then if a new instance comes up it won't be updated. So, I need a way to update AMI. One way could be creating new AMI and Autoscaling group.
Thanks in advance.
This is one way to go about it:
Spin up a stand-alone instance using the AMI
Make changes
Stop instance
Create new AMI from this instance
Create a new Launch Configuration that uses the new AMI
Update the Autoscaling Group to use the new Launch Configuration
Slowly terminate the old instances in the Autoscaling Group, and let them be automatically replaced with instances using the new AMI
Of course all this is a pain to deal with manually every time you need to make a change. Elastic Beanstalk and CloudFormat both provide mechanisms to deal with this in a more automated way.
If you are just changing the code you are deploying to your servers, then there are other ways to handle this, such as using AWS CodeDeploy. You could also update the running servers in some automated or manual fashion, and configure the AMI such that any new instances that are created will go get the latest code on startup.