My AWS solution spans over 3 availability zones. In my backend the user is able to trigger a heavy compute job with beefy px instances. Therefore I wrote a CFN template, which provision all resorucess to execute the compute job (secret store, IAM Role, EC2 instance, log group). However when I try to create the template, it returns with a 500 and states that no capacity for my instance type is available for the availability zone i choose. My template provides a subnet for the EC2 instance and an availability zone for the attached volume. In the end I don't care in which availability zone the ec2 is provisioned as long it is in one of my subnets. Does someone know a way to provision an EC2 instance and it's volume (with cloudforamtion) by not specifically choosing one availability zone, but rather provide a range of subnets/availability zones ?
TLDR:
Does someone know a way to provision an EC2 instance and it's volume (with cloudforamtion) by not specifically choosing one availability zone, but rather provide a range of subnets/availability zones ?
Related
I'm trying to create a deployment where as many tasks as possible are on each EC2 instance, but ECS still places one task per instance which then makes them heavily underutilized.
Here's all the settings I think are relevant:
ASG capacity provider uses managed scaling, target capacity is set to 100%, instances are t3a.micro (2048 CPU, 960 memory)
Task uses 480 memory, 1024 CPU but for testing I tried to go to 200 memory and 500 CPU and nothing changed
The service placement strategies are binpack(cpu) and binpack(memory)
The tasks are in a public VPC within subnets that have access to nat gateway, public ip assign is disabled, the networking mode is awsvpc
I'm changing the desired count to test this
On every deploy tasks are placed into empty instances and if there aren't any ASG creates new ones, there's never more than one task per instance.
It seems that the micro EC2 instance you require doesn't have enough ENI capacity to scale more than 1 task per container instance.
According to the documentation:
With the awsvpc network mode, Amazon ECS creates and manages an
Elastic Network Interface (ENI) for each task and each task receives
its own private IP address within the VPC. This ENI is separate from
the underlying hosts ENI. If an Amazon EC2 instance is running
multiple tasks, then each task’s ENI is separate as well.
Since "this ENI is separate from the underlying hosts ENI", running 1 ECS task requires at least 2 interfaces. In case of running 2 ECS tasks you would need 3 ENI and so on.
When I get a description of EC2 instance types (you can just run aws ec2 describe-instance-types --filters "Name=instance-type,Values=t3a.micro") I see that t3a.micro has a maximum limit of only 2 available network interfaces ("MaximumNetworkInterfaces": 2). So, you need to make better use of the existing maximum number of network interfaces or to get a container instance with capacity to attach more network interfaces.
A solution might be to use an instance type with more ENI capability or to increase task density with ENI trunking. Please read about ENI trunking considerations before. ENI trunking might look something like this:
Does anyone have any guidance on what the most efficient way to launch an EMR cluster using EC2 spot instances? I'm using a 10-node r5d.8xlarge (1 master 9 core) and usually I will clone a previous instance. What's been happening more often than not is that I'll get an "out of capacity" error. When that happens I change the subnet and try again. It's really a guessing game at this point and I'm sure there's a much better way to do this, unfortunately I haven't found one.
There's no need for it to be a guessing game. You can use Instance Fleets - Amazon EMR to specify multiple:
Instance Types
For each fleet, you specify up to five EC2 instance types. Amazon EMR
chooses any combination of these EC2 instance types to fulfill your
target capacities.
Availability Zones
When you use instance fleets, you can specify multiple EC2 subnets
within a VPC, each corresponding to a different Availability Zone.
Instances are always provisioned in only one Availability Zone.
See also: New – Amazon EMR Instance Fleets | AWS News Blog
How can I make sure ASG is scaling EC2 instances in a correct Zone sequence, i.e when I scale ASG from 3 instances to 5 instances, it needs to have 2 nodes in Zone-A, 2 in Zone-B and 1 in Zone-C. But in our case, it ends up in 2 nodes in Zone-A, 1 node in Zone-B and 2 nodes in Zone-C.
AWS ASG launches new instances in all Availability zones you enabled for that particular ASG. This is an extract from the official documentation.
Amazon EC2 Auto Scaling attempts to distribute instances evenly between the Availability Zones that are enabled for your Auto Scaling group. Amazon EC2 Auto Scaling does this by attempting to launch new instances in the Availability Zone with the fewest instances. If the attempt fails, however, Amazon EC2 Auto Scaling attempts to launch the instances in another Availability Zone until it succeeds
If you increase the desired capacity to say 9 (and you have 3 AZ's), you'll see there's a high chance there will be 3 instances on each AZ.
There is no way to control which AZ the AutoScaling Group will launch instances in.
The only work around I can think of is that you could make 1 ASG per AZ and then control the desired on your own via a script instead of using a scaling policy. I would recommend trying to make sure your application is as ephemeral as possible without zonal dependencies so that instances can be added in any zone
So currently I have an application running on am m3.medium instance, its just an single instance running in EU/Ireland area.
If I go to EC2 and buy a reserved instance of m3.medium when will my Elastic beanstalk instance starting using it. Does it just happen or do I have to redeploy or do something else ?
Your EC2 instance will start billing at the discounted price immediately. It "just happens" like you said. Reserved instances are just a billing construct. They are just a discount on your monthly bill. You don't have to assign the reservation to a specific EC2 instance or anything like that.
Once you reserve an instance type, AWS checks for running instances in the region/availability zone where you purchased the reserved instances and applies the discount from the moment you purchased the reserved instances.
For example, you have five m3.medium instances running and you purchased three reserved m3.medium instances, AWS will apply discounted rates for three instances and on-demand rates for two instances.
A region has multiple AZs: Regions and Availability Zones. AWS used to offer RIs in AZs, now they made it flexible by offering it in regions and you can convert the RIs. See: Amazon EC2 Convertible Reserved Instances and the Reserved Instance Regional Benefit
Capacity reservation is the capacity or the instance reserved for you. If you choose AZ based reservation, then capacity is guaranteed. ie., if you reserved a m4.large, you are guaranteed to get a m4.large for the entire reservation period - even if you stop the instance. Always a m4.large instance is set aside for you.
By choosing the region based reservation - which is flexible than the more rigid AZ based reservation - your capacity is not guaranteed. ie., if you reserved a m4.large, you are not guaranteed to get a m4.large for the entire reservation period - in case you stop the instance and start it after sometime or launch a new m4.large instance, your request can be denied because AWS may not have m4.large instance to fulfill your request.
Reserved Instances are more of a billing construct. If you have Reserved Capacity in an Availability Zone, you just need to provision your instance in that Availability Zone. If your Reserved Instance is set to Region, you just need to provision your instance in that region. When it comes to billing, it basically credits you for each instance that matches a reservation.
Check out How Do RIs Work for more information
I'm trying to run an AWS Instance for as cheap as possible and I can see from the pricing history that in availability zone us-east-1d the current price is at $0.25 but in us-east-1a the price is $9.60.
I put in a request for $0.3 and it's saying my request is lower than the minimum fulfillment price of $9.60. There was nowhere to specify the availability zone in the request wizard but there must be a way to get it cheaper than $9.60 which is way over the on-demand price!
Am I doing it wrong somehow or is there a way to do this?
Edit: added image of wizard, where's the option to specify availability zone?
On step 3 you should have the option to set your availability zone preference. Default option is "No Preference".
Faced the same question recently, AWS spot instance wizard is not very intuitive in this regard.
My case was a default VPC, the same as is in the initial question. Therefore to "select availability zone" one needs just to select a corresponding subnet from drop down field.
The official AWS documentation states:
Launching Spot Instances in an Availability Zone Group
Specify an Availability Zone group in your Spot Instance request to
tell the Spot service to launch a set of Spot Instances in the same
Availability Zone. Note that the Spot service need not terminate all
instances in an Availability Zone group at the same time. If the Spot
service must terminate one of the instances in an Availability Zone
group, the others remain running.
Note that although this option can be useful, adding this constraint
can lower the chances that your Spot Instance request is fulfilled.
If you specify an Availability Zone group but don't specify an
Availability Zone in the Spot Instance request, the action taken by
the Spot service depends on whether you specified the EC2-Classic
network, a default VPC, or a nondefault VPC. For more information
about EC2-Classic and EC2-VPC, see Supported Platforms.
EC2-Classic
The Spot service finds the lowest-priced Availability Zone in the
region and launches your Spot Instances in that Availability Zone if
the lowest bid for the group is higher than the current Spot Price in
that Availability Zone. The Spot service waits until there is enough
capacity to launch your Spot Instances together, as long as the Spot
Price remains lower than the lowest bid for the group.
Default VPC
The Spot service uses the Availability Zone for the specified subnet,
or if you don't specify a subnet, it selects an Availability Zone and
its default subnet, but it might not be the lowest-priced Availability
Zone. If you deleted the default subnet for an Availability Zone, then
you must specify a different subnet.
Nondefault VPC
The Spot service uses the Availability Zone for the specified subnet.