Is there a way to instruct AWS to:
create X (number) spot instances from specific AMI? (I guess this is the easy part)
create something periodic so that when these spot instances are terminated, create X new ones from the same AMI?
When launching Amazon EC2 spot instances, you must specify an AMI. Therefore, choose the desired AMI when launching the spot instances.
Your desire to keep a certain number of spot instances running sounds like an excellent use-case for an Amazon EC2 Spot Fleet. A Spot Fleet allows you to specify the types of EC2 instances you would like to be available (instance type, AZ, price) and any instances terminated by Spot capacity can be automatically replaced with other spot instances of a different instance type, AZ, etc, that is not affected by the capacity issue that terminated the previous instances.
Related
As we know, when a spot instance is about to be terminated , we get notified 2 minutes in advance.Now i want my services to be running by replacing the spot instance that is notified to be deleted, by a new spot instance.
Are there any existing ways to automatically replace an EC2 spot instance by a new spot instance?.
From Spot Fleet - Amazon Elastic Compute Cloud:
A Spot Fleet is a set of Spot Instances and optionally On-Demand Instances that is launched based on criteria that you specify. The Spot Fleet selects the Spot capacity pools that meet your needs and launches Spot Instances to meet the target capacity for the fleet. By default, Spot Fleets are set to maintain target capacity by launching replacement instances after Spot Instances in the fleet are terminated. You can submit a Spot Fleet as a one-time request, which does not persist after the instances have been terminated. You can include On-Demand Instance requests in a Spot Fleet request.
We are running g548x AWS EC2 instance to train our ML model. We have observed that sometimes the capacity is not available even with different subnet.
We are considering to use an on-demand capacity reservation but would like to use the same EC2 instance which we have configured to train our ML model.
Should we just create an AMI for the same instance and use this AMI to launch another similar instance in the reserved capacity?
Is there any other better way to achieve this?
In autoscaling group, I have an EC2 instance (with two EBS storage) which could terminate due to any fault and a new EC2 instance is spun in it's place inside the autoscaling group.
My question is how the two EBS storage attached to old EC2 instance be attached to new EC2 instance.
if this is manual process, could reference to terraform be provided.
#rukan I guess you need to make 'DeleteOnTermination' value as false, as you would need EBS of old instances for newly created one.
Reference :
https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_Ebs.html
https://francescoboffa.com/aws-stateful-service-ebs/
Update :
I tried to do some research on your requirement and I can conclude my answer that there is no standard method where we can re-use EBS volume in AWS auto-scaling group. Moreover It is not recommended one, as autoscaling groups start multiple EC2 instances. Each EBS volume can only be attached to a single EC2 instance. For now, I can suggest to use AWS EFS instead of EBS.
But, if this is must do requirement with EBS, then we need think of some complex logic where writing some kind of startup script which associated the volume with the EC2 instance then mounted it. You can refer this answer
Reference :
https://serverfault.com/questions/831974/can-i-re-use-an-ebs-volume-with-aws-asg
I have AWS ECS cluster but spot instance type I selected is too small.
I can't find way to adjust Spot Fleet request ID or change Instance type(s) for Spot Fleet request cluster is using.
Do I have to create a new cluster with a new spot fleet request?
Is there any cli option to adjust cluster?
Do I have manually order EC2 with ECS optimized AMI ?
UPDATE In question How to change instance type in AWS ECS cluster? that sounds similar advised to copy Launch Configuration. But I have no Launch Configuration
There is no way of changing the instance types' requested by a spot fleet after its been created.
If you want to run you ECS workload on another instance type, create a new spot fleet (with instances which are aware of your ECS cluster).
When the spot instances spin up, they will register with your ECS Cluster.
Once they are registered, you can find the old instances (in the ECS Instances tab of the cluster view) and click the checkbox net to them.
Then, go to Actions -> Drain instances
This tells ECS that you no longer wish to use these instances. New tasks will now be scheduled on the new instances.
Once all the tasks are running on the new instances, you can delete the old spot fleet.
On the subject of launch configurations. There are two ways of creating collections of spot instances.
Through a Spot Fleet (which is what you're doing)
Through and Auto Scaling Group (ASG)
ASGs allow you to supply a launch configuration (basically a set of instructions to set up an EC2 instances.
Spot Fleets only allow you to customise the instance on creation via User Data.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html
Because you're using Spot Fleets, Launch Configurations are really a consideration for you.
I have an auto-scaling group with 2 instances.
Every time an instance is launched an EBS volume is attached to it. When it is replaced/terminated is the EBS volume deleted ?
I want to keep a tight budget on my account and I dont want to have volumes lingering and pay for them.
You can configure it both ways. If you are using the web interface to configure the launch configuration when you get to Storage you will see this:
You just need to check the "Delete on Termination" checkbox on the right, whenever an instance is terminated the EBS volume associated will be deleted as well.