I read the docs but still didn't understand anything. The AWS forum takes time to answer. Here is my scenario:
I had code in an EC2 instance but needed to join the instance to a scaling group. There was one already created so I just joined mine to that since it was "unused". In a moment notice my instance terminates and I lost all my code within the instance. Only the original instance from the scaling group maintained itself.
Now my question: Is scaling supposed to help the same service performance? Because:
the dynamically created instances DO NOT possess the same address.
This means I can SSH to one but if I SSH to another one will there be the same code?
the dynamically created instances DO NOT possess the same security group
the original instance (that could hold the code) ALSO CLOSES if replaced by the new one (I don't know the criteria). So, the one holding the code can always close...
So, between trying to understand if need to try and recover my terminated EC2 instance and understanding all this process, I must admit I'm quite lost because nothing really inspires "merely incrementing process power" with confidence so I really don't know what to do.
Sorry to hear that you lost valuable information. One thing to learn about Auto Scaling is that instances can be terminated, so they should be designed with this in mind.
Purpose of Auto Scaling
Auto Scaling is designed to dynamically adjust the number of instances in an auto scaling group to handle a given compute/processing requirement. Instances can be automatically added when things are busy, and terminated when things are less busy.
Auto Scaling automatically creates new instances by using a Launch Configuration, which describes the type of instance to launch, which disk image (AMI) to use, security group, network settings, startup script, etc. Each new instance would, of course, receive its own private IP address and (if configured) its own public IP address.
The idea is that a new instance can be started with no manual intervention, and all instances within an Auto Scaling group would normally be configured identically (since they are doing an identical job).
When an instance is no longer required, the group is "scaled-in", meaning that an instance is terminated. All data on the instance is lost. (Actually, disks can be kept around after termination, but they are a pain to handle later, so it is best not to do this.)
It's okay for instances to be terminated because they can easily be recreated when Auto Scaling launches a new instances. Unfortunately in your case, you added an instance to an existing Auto Scaling group with your own configuration. It would therefore be your responsibility to make sure you can handle the loss of the instance. It's quite unusual to manually add an instance to an existing Auto Scaling group -- usually it's to send some traffic to a test instance or for doing A/B trials.
Why did it terminate?
The scaling policies attached to your Auto Scaling group probably decided that you had too much capacity (as a result of increasing the number of instances), and therefore decided to terminate an instance.
Recovering your terminated instance
Your terminated instance cannot be "recovered", but there is a small possibility that your disk is still available. By default, when launching an Amazon EC2 instance, boot disks are marked as Delete on Termination, but additional disks default to not deleting on termination. Therefore, if your code was on the non-boot disk, it may still be available in your Volumes tab.
If, however, your code was on a disk volume that was deleted, then the contents cannot be recovered. (Well, if you have Support, you could ask them but it's unlikely to be easy, especially if time has passed since deletion.)
See: AWS Auto Scaling documentation
The short answer. Autoscaling uses horizontal scaling (adding more instances) and not vertical scaling (increasing CPU/memory allotment)
In order to successfully use auto scaling, you need to design your application using the principle of shared nothing architecture. No persistent data can be stored on the instance itself. Instead you would store it on S3, or other instances not part of autoscaling group.
Related
I am trying to spin up 2 ec2 instances, that are essentially the exact same server, except that they are differentiated by name (ie. server-01 and server-02). I'm using an autoscaling group to make sure that if an instance crashes, that a new one spins up automatically. Also, the launch script that is passed to each instance includes their hostname (server-01 etc.) that is needed for a configuration file. It would be a lot cleaner to just have one asg with 2 concurrent instances that passes this variable into the individual instances, but then there is the issue of first, is it even possible to pass in a variable into an asg launch script, and then if one instance goes down, wouldn't the variable then go to server-03 for example?
To me, it seems that I need to create 2 asgs each with a single instance, but I was wondering of the above is possible? Thanks!
If your two Amazon EC2 instances are performing different functions then, yes, it would be correct to use two different Auto Scaling groups.
When multiple instances are launched under one Auto Scaling group, it is expected that each instance is performing an identical function. There should certainly be no need to give each instance a specific reference number (eg 1, 2) and have a replacement instance assume that number.
The alternative is to have each instance inspect the environment at startup. It could DescribeInstances() with a tag of the Auto Scaling group to discover currently-running instances in the group. Then, you could use Name tags to track what each instance is doing. For example, if it sees that there is only Server-2 running, then it should assume that it needs to take the place of Server-1.
Much better to either use separate Auto Scaling groups or make each instance identical.
Using Auto Scaling with Load balance in AWS, We can do the following things, according to my understanding:
we can scale up and scale down according to load.
all instances have the same image.
But I have a different problem:
if we have less load then we should terminate a big machine and start a small machine and vice versa.
small machine and the big machine has a different image
but I am not getting any help from AWS UI.
Can anyone help me on this issue?
Amazon EC2 Auto Scaling can launch new instances and can terminate instances. It only ever adds or removes instances -- it never changes the size of an instance. This is why you'll often see it referred to as "scale-out and scale-in" rather than "scale-up and scale-down".
When a scaling policy is triggered and Auto Scaling needs to launch a new instance, it uses the provided Launch Configuration or Launch Template to determine what type of instance to launch, which network to use, etc.
Therefore, an Auto Scaling group typically consists of all the same size instances since they are all launched from the same Launch Configuration. This is actually a good thing because it makes it easier for the scaling alarms to know when to add/remove instances and it also helps Load Balancers distribute load amongst instances since they assume that all instances are sized the same.
So, rather than "terminate a big machine and start a small machine and vice versa", Auto Scaling simply launches the same sized instance or terminates an instance.
Also, all instances should use the same AMI since load balancers will send traffic to every instance, expecting them to behave the same way.
You could, if you wish, modify the Launch Configuration associated with the Auto Scaling group so that, when it next launches an instance, it launches a different-sized instance. However, Auto Scaling and Load Balancers will not 'know' that it is a different-sized instance.
Basically John answered this question.
As alternative we can have some sophisticated scaling logic in any computing resource. For example AWS Alarms can send SNS notification, that Lambda reads and scale in or out using sophisticated logic you have (big or small instances etc.).
I'm considering using spot instances in an auto-scaling group. While I'm aware that I'll receive a 'Spot instance interruption notice' if my spot instances are going to get terminated, what is unclear from the documentation is if my auto-scaling group will spin up new on-demand instances to replace these when the notice occurs, or if they only get replaced on termination. I'm aware that I could listen for these notices manually, but it seems like something that an auto-scaling group should be able to handle automatically.
I've tried testing this out on an existing auto-scaling group that had spot instances by changing the launch configurations 'spot price' to be lower than the current price. This did not work as it would only effect new instances and not currently running ones. I'm unsure of how to change an existing spot request's price.
What I'm hoping will happen is that on-demand instances will be spun up in the two minutes I have from the interruption notice till the time of termination.
If the Launch Configuration in your Auto Scaling Group is configured to use Spot instances then the new instance will indeed be a Spot instance.
The situation you describe is one of the challenges of using Spot instances; although the cost is very low, Spot instances can be terminated and the underlying resources used for a paying customer to fulfill an on-demand or Reserved Instance at anytime.
One way to avoid this is to use Reserved Instances. If you have a predictable long-term need for an instance, or are running a production workload, using Reserved instances is an effective way to lower your costs (albeit, not as low as a spot instance) without having to worry that you could lose your instance(s) at anytime.
Regarding changing the price, updates to pricing are applied to new instances only. After updating pricing simply terminate your existing instances and they’ll be replaced by your ASG with instances at the new price.
I just got a word in below link that aws have unprotected instance what is it mean in actualy?
Reference link - https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-instance-termination.html
When using autoscaling you can protect instances from termination during scale in events by selecting them in the Auto Scaling Console and then choosing Instance Protection from the Actions menu.
You might want to do this for several reasons. First, an instance
might be handling a long-running work task, perhaps pulled from an SQS
queue. Protecting the instance from termination will avoid wasted
work. Second, the instance might serve a special purpose within the
group. It could be the master node of a Hadoop cluster or a “canary”
that flags the entire group of instances as up and running.
In most cases, you will want to leave at least one instance in each of
your auto scaling groups unprotected. If all of the instances are
protected, no scale in action will be taken.
Link
I am looking at using AWS auto-scaling to scale my infrastructure up and down based on various performance metrics (CPU, etc.). I understand how to set this up; however, I don't like that instances are terminated rather than stopped when it is scaled down. This means that when I scale back up, I have to start from scratch with a new instance and re-install my software, etc. I'd rather just start/stop my instances as needed rather than create/terminate. Is there a way to do this?
No, it is not possible to Stop an instance under Auto Scaling. When a Scaling Policy triggers the removal of an instance, Auto Scaling will always Terminate the instance.
However, here's some ideas to cope with the concept of Termination...
Option 1: Use pre-configured AMIs
You can configure an Amazon EC2 instance with your desired software, data and settings. Then, select the EC2 instance in the Management Console and choose the Create Image action. This will create a new Amazon Machine Image (AMI). You can then configure Auto Scaling to use this AMI when launching a new instance. Each new instance will contain exactly the same disk contents.
It's worth mentioning that EBS starts up very quickly from an AMI. Instead of copying the whole AMI to the boot disk, it copies it across on "first access". This means the new instance can start-up immediately rather than waiting for the whole disk to be copied.
Option 2: Use a startup (User Data) script
Each Amazon EC2 instance has a User Data field, which is accessible from the instance. A script can be passed through the User Data field, which is then executed when the instance starts. The script could be used to install software, download data and configure the instance.
The script could do something very simple, like download a configuration script from a source code repository, then execute the script. This means that machine configuration can be centrally managed and version-controlled. Want to update your app? Just launch a new instance with the updated script and throw away the old instance (which is much easier than "updating" an app).
Option 3: Add/Remove instances to an Auto Scaling group
Rather than using Scaling Policies to Launch/Terminate instances for an Auto Scaling group, it is possible to attach/detach specific instances. Thus, you could 'simulate' auto scaling:
When you want to scale-down, detach an instance from the Auto Scaling group, then stop it.
When you want to add an instance, start the instance then attach it to the Auto Scaling group.
This would require your own code, but it is very simple (basically two API calls). You would be responsible for keeping track of which instance to attach/detach.
You can suspend scaling processes, see documentation here:
https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-suspend-resume-processes.html#as-suspend-resume
Add that instance to Scale in protection and then stop the instance then it will not delete your instance as it's having the scale in protection.
Actually you have three official AWS options to reboot or even stop an instance which belongs to an Auto Scaling Group:
Put the instance into the Standby state
Detach the instance from the group
Suspend the health check process
Ref.: https://aws.amazon.com/premiumsupport/knowledge-center/reboot-autoscaling-group-instance/
As of April 2021:
Option 4: Use Warm Pools and an Instance Reuse Policy
By default, Amazon EC2 Auto Scaling terminates your instances when your Auto Scaling group scales in. Then, it launches new instances into the warm pool to replace the instances that were terminated.
If you want to return instances to the warm pool instead, you can specify an instance reuse policy. This lets you reuse instances that are already configured to serve application traffic.
This mostly automates option 3 from John's answer.
Release announcement: https://aws.amazon.com/blogs/compute/scaling-your-applications-faster-with-ec2-auto-scaling-warm-pools/
Documentation: https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-warm-pools.html
This is to expand a little on #mwalling's answer, because that is the right direction, but needs a little extra work to prevent instance termination.
There is now a way to stop or hibernate scaled in instances!
By default AWS Autoscaling scale in policy is to terminate an instance. Even if you have a warm pool configured. Autoscaling will create a fresh instance to put into the warm pool. Presumably to make sure you start with a fresh machine every time. However, with a instance reuse policy you can make AWS Autoscaling either stop or hibernate a running instance and store that instance in the warm pool.
Advantages include:
Local caches stay populated (use hibernate for in memory cache).
Burstable EC2 instances (those types with T*) keep built up burst credits instead of the newly created instance that have limited or no credits.
Practical example:
We use a burstable EC2 instance for CI/CD work that we scale to 0 instances outside working hours. With a reuse policy our local image repository stays populated with the most important Docker images. Also we keep the built up credit from the previous day and that speeds up automatic jobs we run first thing every morning.
How to implement:
There's currently no way of doing this completely via the management console. So you will need to use AWS CLI or SDK.
First create a warm pool as described in the AWS Documentation
Then execute this command to add a reuse policy:
aws autoscaling put-warm-pool --auto-scaling-group-name <Name-of-autoscaling-group> --instance-reuse-policy ReuseOnScaleIn=true
Reference docs for the command: AWS CLI Autoscaling put-warm-pool documentation
Flow diagram of possible life cycles of EC2 instances:
Image from AWS Documentation: Lifecycle state transitions for instances in a warm pool