How AWS resources limit and usage are calculated - amazon-web-services

Going through the AWS Service Limits documentation, I can't understand how the AWS resources usage is calculated. Does it decrease every time you launch a new instance? Does it increase when you terminate it? Is it a monthly limit? Annual?
The AWS documentation is unclear.
Update:
In many regions, I have Running On-Demand EC2 instances (max number of EC2s limit is 0).
In other regions I have some instances types limit equal to 0.
In other regions, I have instance type limit > Running On-Demand EC2 instances.
Am I missing something?


The limit is anytime limit, not monthly or annual. Suppose your limit is 20 EC2 instances, at any time you can have a max of 20 instances (running + stopped).
When you launch a new instance, your instance count (running + stopped) is checked and only if it is less than 20 (your limit) AWS will continue to launch a new instance. Otherwise you will get a message to increase the limit and no instance is launched.
So your limit is checked when you launch a new instance.
(Current running + stopped) + 1 > instance_limit ==> Cannot launch
(Current running + stopped) + 1 <= instance_limit ==> Launched
This limit is per region and per instance type.


The Amazon EC2 Management Console has a Amazon EC2 Service Limits Report section that displays the current limits on EC2 resources.
The Running On-Demand EC2 instances limit is described as:
The total number of running On-Demand instances that you can have in this region. Some instance types have different limits for this region that count against your total limit; these are listed below. Check the Current Limit column to find out how many instances per instance type you can run.
The limits can be increased by clicking the Request limit increase link and providing a use-case. AWS Customer Service then evaluates the case and adjusts the limits accordingly.
In some situations, customers request a decrease of their limit to avoid accidentally using some of the more-expensive instance types, or to prevent instances from running in unwanted Regions.

Related

Not able to launch 2nd Instance instance in AWS Free tier

While launching 2nd instance getting below error
you have requested more vcpu capacity than your current vcpu limit of 1 allows for the instance bucket that the specified instance type belongs to. please visit http://aws.amazon.com/contact-us/ec2-request to request an adjustment to this limit.

First, please note that the AWS Free Tier is a billing discount. It does not impact your ability to use any services. If the resources you use are within the limits imposed by the AWS Free Tier, you simply won't be charged for those resources. Thus, the AWS Free tier is not the cause of this error.
From On-Demand Instances - Amazon Elastic Compute Cloud:
There is a limit on the number of running On-Demand Instances per AWS account per Region. On-Demand Instance limits are managed in terms of the number of virtual central processing units (vCPUs) that your running On-Demand Instances are using, regardless of the instance type.
You did not tell us what Instance Type you are running, or attempted to run, but it sounds like you have exceeded this vCPU limit. This is more likely to happen with large or expensive instance types, or instance types that use GPU resources. You can view the limits in the Amazon EC2 management console by clicking Limits and then searching for vCPU.
If this is impacting your use of AWS, you can request an increase to these default limits.
See: Calculate a vCPU limit increase request for an Amazon EC2 On-Demand Instance

AWS spare EC2 compute capacity, how it is calculated?

Spot Instance Definition:
Spot Instances are spare EC2 compute capacity in the AWS Cloud available to you at savings of up to 90% off compared to On-Demand prices
My Confusions are:
How this Spare EC2 compute capacity is calculated
This Spare Capacity calculation is based on the Current Account OR Current Region OR Entire AWS Regions ?

Anybody with an AWS Account can launch an Amazon EC2 instance as a Spot Instance. If there is available capacity for the selected Instance Type and Availability Zone, the Spot Instance will be launched and you will be charged a price that is at the current Spot Price (normally much less than On-Demand prices).
However, AWS can stop/terminate that Spot Instance at any time when they need the capacity for customers who are using On-Demand instances. Therefore, the Spot Instance is not guaranteed to keep running.
You have no visibility into the available capacity or the likelihood that the Spot Instance will be stopped/terminated. If you want to maintain a level of capacity using Spot Instances, then you can use a Spot Fleet - Amazon Elastic Compute Cloud, which can automatically replace Spot Instances when they are stopped/terminated.

What AWS service limit prevents a large fleet of instances from launching?

I'm trying to launch a fleet of 700 r4.16xlarge instances via spot request.
I used cfncluster to launch an fleet with initial_queue_size = 10
and max_queue_size = 700. However this fleet scaled up and maxed out at 50 instances (and only $120/hr :P). There are many hundreds of tasks queued up in squeue, but something is preventing more instances from being launched.
After I realized this, I attempted to create another fleet of the same instance type in the same region and received the following error message:
- AWS::AutoScaling::AutoScalingGroup ComputeFleet Received 0 SUCCESS signal(s) out of 10. Unable to satisfy 100% MinSuccessfulInstancesPercent requirement
I do not know what service limit I am maxing out as my limit for on-demand r4.16xlarge is at 20. Is there a limit for spot request instances separately from the on-demand instances?
I checked ec2 limits as well as the trusted advisor service limits (linked below) and nothing seems to be maxed out.
https://console.aws.amazon.com/trustedadvisor/home?#/category/service-limits
Any help is much appreciated!

AWS EC2 - Clarification on number of instances

This is from the amazon ec2 FAQ :
Q: How quickly can I scale my capacity both up and down?
Amazon EC2 provides a truly elastic computing environment. Amazon EC2 enables you to increase or decrease capacity within minutes, not hours or days. You can commission one, hundreds or even thousands of server instances simultaneously. When you need more instances, you simply call RunInstances, and Amazon EC2 will typically set up your new instances in a matter of minutes. Of course, because this is all controlled with web service APIs, your application can automatically scale itself up and down depending on its needs.
Now again as per the same FAQ, I am only allowed to launch 20 instances per region. They said, I have to fill in a request form if I need more than 20 instances. So, in effect, I cant spin up more than 20 programmatically ?
What am I missing here ? how can we launch 100 instances let alone thousands. Sorry if this is the wrong place for such a question.

You cannot launch instances beyond the instance limit. You need to make a request to increase the instance limit. This is a safety feature so that:
A wild loop in your SDK/API script does not launch instances continuously
A malicious user does not launch a large number of instances
A hacker gets access to your account and launches a large number of instances
An incorrectly configured autoscaling group launches huge number of instances
If you require more than your instance limit, you need to submit a request to AWS. See: Amazon EC2 Service Limits. AWS will review your request and approve it.

You are missing the fact that limit increase requests are very easy to make and are almost always granted with no questions asked within a day or two.
To request a limit increase:
Open the AWS Support Center page, sign in if necessary, and choose Create Case.
For Regarding, choose Service Limit Increase.
Complete Limit Type, Use Case Description, and Contact method. If this request is urgent, choose Phone as the method of contact instead of Web.
Choose Submit.

AWS faqs provides a clear answer
You are limited to running up to a total of 20 On-Demand instances across the instance family, purchasing 20 Reserved Instances, and requesting Spot Instances per your dynamic Spot limit per region. New AWS accounts may start with limits that are lower than the limits described here. Certain instance types are further limited per region as follows
For Spot instance limits AWS states
The usual Amazon EC2 limits apply to instances launched by a Spot Fleet, such as Spot request price limits, instance limits, and volume limits. In addition, the following limits apply:
The number of active Spot Fleets per region: 1,000
The number of launch specifications per fleet: 50
The size of the user data in a launch specification: 16 KB
The target capacity per Spot Fleet: 3,000
The target capacity across all Spot Fleets in a region: 5,000
A Spot Fleet request can't span regions.
A Spot Fleet request can't span different subnets from the same Availability Zone.
These limits protect you from a hacker attack, stolen API keys, ETC. If you want to increase these limits, you need to send a form to AWS support team: AWS Support Center

Can not start EC2 instance - "Your quota allows for 0 more running instance(s)"

I have 50 instances. I am running 30 instances (with multi instance type), then I cannot run another instance. The error:
Error starting instances
Your quota allows for 0 more running instance(s). You requested at least 1
I can not start instance with number > 30.
How to increase limit instance running?

There are resource limits to the number of Amazon EC2 instances that be run concurrently -- in total and per instance type.
The easiest way to view these limits and to request an increase is via the Limits option in the Amazon EC2 management console:
The first line shows the limit related to Total number of On-Demand EC2 Instances.