I am trying to launch a AWS spot instance for a duration. Before that I checked price history graph.
However, when I launch an instance, I get below message:
"minimum required Spot request fulfillment price of 0.477. "
Is spot instance price for fixed duration different from just spot instance?
How does prices for spot instance price for fixed duration are determined?
You are correct.
Introducing Amazon EC2 Spot Instances for Specific Duration Workloads says:
You can now request Amazon EC2 Spot instances to run continuously, for up to six hours, at a flat rate that saves you up to 50% compared to On-Demand prices.
The price will be higher than the normal Spot Instance price because there is a duration guarantee. A normal spot instance can be terminated at any time (with 2 minutes notice).
There is no information available about how the price is determined, but presumably it involves a prediction of having sufficient capacity over that time period possibly forcing normal spot instances to terminate in order to provide sufficient capacity for the defined duration.
Related
i want to the difference between spot and on demand instances. I know there is a price difference between these two but other than this i want to know the differences. Please help me
Actually there are three allocation types:
on demand - kind of "default" mode. You request an instance, if there is free capacity, you will get the instance. No long term commitment, but once you get an instance, it's yours. It may happen that you will get a message that there is no free capacity for the specific instance type and AZ (so far it happened to me only once with AWS).
reserved - AWS reserves the capacity for you. You have guarantee that you will get the instance type in the selected region or AZ.
spot instance - it's kind of auction / bidding of unused capacity. You ask for an instance, you provide your maximum price and if there is free capacity and your price is at the current price or higher, you will get an instance. The difference is - if the free capacity is exhausted, or the current price is higher than your maximum bid price, your spot instance is terminated . You can get a termination warning event upfront.
The resources for both are the same, spot instances utilise the spare compute capacity within the AWS availability zone (those that are not reserved or launched on-demand).
Depending on the demand for that instance class in the availability zone the spot price will increase or decrease (even surpassing the on-demand price).
When you use a spot instance you are taking a risk that if demand increases you will lose access to the spot instance (you are given a 2 minute warning before termination). For his reason it is common to use a mixture of on-demand/reserved instances and spot instances so that you can withstand instance terminations.
Commonly in EC2 applications you would use an autoscaling group with a configured proportion between on-demand/reserved nodes and spot instances.
For more information take a look at the Requesting Spot Instances for fault-tolerant and flexible applications documentation.
I have a fresh AWS account with Reserved Capacity of 15 EC2 instances of the same instance type in the same region primarily intended for EMR.
If I spawn a EMR cluster with 10 instances for 15 days and scale it up to 20 instances for the remaining 15 days, will I get charged $0?
If not, how will the final bill for EC2 for the month be calculated?
The reason why I am confused is that based on my understanding EC2 charges based on capacity consumed (in instance hours) and not on actual instances allocated. Since I'm reserving 15 instances worth of capacity, I would expect it to cost me a total of $0 since my overall monthly average consumption is worth 15 instances.
The Reserved EC2 Pricing documentation and most of the AWS price-calculation related material on the internet don't mention the use-case of dynamic scaling of instances in accounts with reserved capacity.
Reserved instances are applied to running instances in real time, not averaged over any longer period of time. They are baseline, not average.
For example, if you own three Reserved Instances with the same instance attributes and region (or Availability Zone if applicable), the billing system checks each hour to see how many total instances you have running that match those parameters. If it is three or less, you will be charged the Reserved Instance rate for each matching instance running that hour. If more than three are running, you will be charged the On-Demand rate for the additional instances.
https://aws.amazon.com/ec2/pricing/reserved-instances/buyer/
At any instant in time when you lack enough reserved capacity to cover your running workload, you're billed for the excess.
I'm confused about setting maximum price. For example, I'm approved to use g2.8xlarge spot instances in us-west region. In Configure Instance Details I am shown the following:
Availability Zone Current price
us-west-2a $1.261 USD
us-west-2b $1.500 USD
us-west-2c $26.00 USD
So I set max price to $7 thinking that should easily get me an instance in us-west-2a or us-west-2b. However, on submitting the request I get
Status: capacity-not-available: There is no Spot capacity
available that matches your request.
Furthermore, the request description specifies:
Availability Zone: us-west-2c
...which suggests to me that my request is only being considered within the vastly more expensive of the three us-west sub-regions I was quoted prices from before. Thanks for any guidance you can offer me!
When looking at spot instances, look into several different instance types. Not all instance types are available at any given time. Availability varies by region. If the region does not matter, then you will have more choices.
Note: just because a spot price is specified, does not mean that there are any hosts with available instances. They could all be taken. Some companies will bid on thousands of instances which will keep utilization high. This typically happens with the very large instances, which by their nature are expensive and powerful and spot pricing makes these instances much cheaper. If you select middle of the road instances (in performance) you may have better luck.
For bidding price, look at the history of the instance by region. If the history price is close to On-Demand, move on. If the history is less the 50% (better 30%) of On-Demand and there are not too many price spikes per week or month going above 2x On-Demand, then look at bidding 2x On-Demand pricing. If you don't care when and how often your instances run, then bid 1.5x On-Demand.
Remember to make your instances stateless (e.g. store everything in a database, S3, etc.). Your instances may be terminated at any time. You can use ASG to keep a specific number of instances running.
[EDIT]
I just noticed that you are bidding on G2 instances. G2 instances are previous generation instances, which means that they are being phased out making spot bidding harder and harder. Bid instead on current generation instances such as the G3. Also unless you need Windows instances, bid on Linux instances.
want to use spot instance reliable(without termination because of spot market price) for some duration, have following question for the same:
is spot-instance with block duration run without interruption for given duration, and can we rely on that to not terminate instance because of market price of that instance?
is the pricing for --block-duration is same as other spot instance?
is spot-instance with block duration run without interruption for
given duration, and can we rely on that to not terminate instance
because of market price of that instance?
Yes. You can run your spot instances for a maximum allowed block duration period of 6 hours without any interruption or termination even if there is any hike in market price.
is the pricing for --block-duration is same as other spot instance?
No. The pricing is different(greater) than the Spot Instances without defined duration. Refer this link for EC2 Spot Pricing.
I am trying to create ec2 spot instances using boto3 api, so far I am able to get the spot instance history price, spin up a spot instance, etc.
But I don't know how to get the price we are paying for spot instance using boto api.
anyone know how to do this ?
Thanks
Update: See: Spot Instance Interruptions - Amazon Elastic Compute Cloud
Old answer:
When launching a spot instance under Amazon EC2, you specify a maximum hourly price, known as a bid. This is the maximum price that will be charged for the instance.
When the instance is launched, and on each hourly anniversary of the instance being in a running state, the instance will be charged at the then-current spot price.
If an instance is terminated due to the spot price rising above the bid price, then there is no charge for the partial hour where the instance was terminated. However, if the user terminates a spot instance, the partial hour will be charged.
Therefore, "the price we are paying for spot instance" is the price at the start of each hour. It is calculated at the time that the charge takes place.
To obtain the price charged for a spot instance, use the Spot Instance Data Feed that provides a data feed that describes your Spot Instance usage and pricing. This data feed is sent to an Amazon S3 bucket that you specify when you subscribe to the data feed. It is updated hourly.