Perhaps this isn't so much a code question as a definitional question, but would someone be able to explain to me what the six line items below represent?
EBS has 3 types of storage (in order from most expensive to cheapest):
Provisioned I/O. These are SSD volumes with a performance guarantee. With these volumes you not only pay for the size of the volume, but also the number of IOPS you have provisioned. These volumes should only be used when performance is very important.
General Purpose SSD. These volumes provide improved performance over Magnetic volumes at a somewhat higher cost. Probably the best choice for most general purpose uses.
Magnetic. This type of storage uses magnetic disks and is the cheapest and slowest. Good for bulk data storage that doesn't have any performance requirement.
The other two items not covered by the above volume types are IO requests, which occur any time data blocks are read or written to any volume. Also snapshots are copies of volumes stored on S3.
Amazon Elastic Block Storage is offered in three flavors: Magnetic, PIOPS SSD and General Purpose SSD.
Each flavor will offer different performance and prices, that you can check in the EBS pricing page.
These lines looks like a budget showing how much of each is consumed by your project :)
Related
Can you point me to some resources on how EBS works behind the scenes for gp2 volumes?
The way I understand it, it is a service, but really it is some form of connecting arrays of SSD drives to the instance, in a redundant way
What is the actual, physical method of connecting?
THe documentation refers to the fact that data is transferred in 16KB or 256KB blocks, but I can't find any more about that.
If for example, in Linux, my partition is formatted with 4KB blocks, does this mean that EBS will transfer data to and from disk with 16KB block, if so wouldn't it make sense to also format the partition with 16KB block and also optimise it upstream?
If I have a set of very random 4k operations, will this trigger the same amount of 16KB block requests?
If anyone's done such testing already, I'd really like to hear it...
The actual, physical means of connection is over the AWS software-defined Ethernet LAN. EBS is essentially a SAN. The volumes are not physically attached to the instance, but they are physically within the same availability zone, the access is over the network.
If the instance is "EBS Optimized," there's a separate allocation of Ethernet bandwidth for communication between the instance and EBS. Otherwise, the same Ethernet connection that handles all of the IP traffic for the instance is also used by EBS.
The SSDs behind EBS gp2 volumes are 4KiB page-aligned.
See AWS re:Invent 2015 | (STG403) Amazon EBS: Designing for Performance beginning around 24:15 for this.
As explained in AWS re:Invent 2016: Deep Dive on Amazon Elastic Block Store (STG301), an EBS volume is not a physical volume. They're not handing you an SSD drive. An EBS volume is a logical volume that spans numerous distributed devices throughout the availability zone. (The blocks on the devices are also replicated within EBS within the availability zone to a second device.)
These factors should make it apparent that the performance of the actual SSDs is not an especially significant factor in the performance of EBS. EBS, by all appearances, allocates resources in proportion to what you're paying for the volume... which is of course directly proportional to the size of the volume as well as which feature set (volume type) you've selected.
16KiB is the nominal size of an I/O that EBS uses for establishing performance benchmarks for gp2. It probably has no other special significance, as it appears to be related as much or more to the processing resources that EBS allocates to your volume as to the media devices themselves -- EBS volumes live in storage clusters that have "resources" of their own (CPU, memory, network bandwidth, etc.) and 16KiB seems to be a nominal value related to some kind of resource allocation in the EBS infrastructure.
Note that the sc1 and st1 volumes use a very different nominal I/O size: 1 MiB. Obviously, that can't be related to anything about the physical storage device, so this lends credence to the conclusion that the 16KiB number for gp2 (and io1).
A gp2 volume can perform up to the lowest of several limits:
160 MiB/second, depending on the connected instance type‡
The current number of instantaneous IOPS available to the volume, which is the highest of
100 IOPS regardless of volume size
3 IOPS per provisioned GiB of volume size
The IOPS credits available for with in your token bucket, capped at 3,000 IOPS
10,000 IOPS per volume regardless of how large the volume is
‡Smaller instance types can't provide 160MiB/second of network bandwidth, anyway. For example, the r3.xlarge has only half a gigabit (500 Mbps) of network bandwidth, limiting your total traffic to EBS to approximately 62.5 MiB/sec, so you won't be able to push any more throughput to an EBS volume than this from an instance of that type. Unless you are using very large instances or very small volumes, the most likely constraint on your EBS performance is going to be the limits of the instance, not the limits of EBS.
You are capped at the first (lowest) threshold in the list above, the impact of the nominal 16 KiB I/O size is this: if your I/Os are smaller than 16KiB, your maximum possible IOPS does not increase, and if they are larger, your maximum possible IOPS may decrease:
an I/O size of 4KiB will not improve performance, since the nominal size of an I/O for rate limiting purposes is established 16KiB, but
an I/O size of 4KiB is unlikely to meaningfully decrease performance with sequential I/Os since, for EBS's accounting purposes, are internally combined. So, if your instance were to make 4 × 4 KiB sequential I/O requests, EBS is likely to count that as 1 I/O anyway
an I/O size of 4KiB and extremely random I/Os would indeed not be combined, so would theoretically perform poorly relative to the same number of 16KiB extremely random I/Os, but instinct and experience tells me this borders on academic and theoretical territory except perhaps in extremely rare cases. It could just as likely hurt as help, since small writes would use the same number of IOPS but transfer more unnecessary data across the wire.
if your I/Os are larger than 16KiB, your maximum IOPS will decrease if your disk bandwidth reaches the 160MiB/s threshold before reaching the IOPS threshold.
A final thought, EBS performs best under load. That is to say, a single thread making a series of random I/Os will not keep the EBS volume's queue filled with requests. When that is not the case, you will not see the maximum possible performance.
See also Amazon EBS Volume Performance on Linux Instances for more discussion of EBS performance.
I am storing users' code in file system, at present EBS in AWS. I am looking improving the availability and want to reduce the chances of outage due to EBS going down. EFS appears to be a reasonable option.
I understand EFS will be slower than EBS and EFS is more expensive than EBS. I want to know, if there is any performance benchmark done to measure the read and write latencies of EFS and comparison with EBS?
This AWS forums thread shows you some of the problems that some customers have had with eFS latency and AWS reaction. Some customers assert they have had 1+ second latency, to which AWS support say that's not normal, they'll investigate.
My current experience in EU-West appears to suggest that for a series of 150,000 small read operations of about 2.5KB each, my EC2<->EFS is maxing out at 200 read ops per second, so we might guess at no more than 1/200th of a second or 5ms for typical effective latency.
I say "effective latency" because that's really reporting a bandwidth, not a latency. I haven't written timing code to measure round-trip latency.
You can improve it by paying for a bigger drive (which includes bigger IOPS in the price) or for reserved IOPS.
EFS is a Network Filesystem(NFS). It provides a file system interface, file system access semantics (such as strong consistency and file locking), and concurrently-accessible storage for up to thousands of Amazon EC2 instances. Ofcourse there would be read/write latency compared to EBS as EBS is designed for low-latency access to data.
EBS provides different volume types, which differ in performance characteristics and price, so that you can tailor your storage performance and cost to the needs of your applications.
EFS is easy to use and offers a simple interface that allows you to create and configure file systems quickly and easily. With Amazon EFS, storage capacity is elastic, growing and shrinking automatically as you add and remove files, so your applications have the storage they need, when they need it.
Perfromance Overview of EFS: http://docs.aws.amazon.com/efs/latest/ug/performance.html
Performance Overview of EBS:http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html
I have some images needed for an app. There are many images (50,000+) but the overall size is small (40 Mb). Initially, I thought I would simply use S3 but it is painfully slow to upload. As a temporary solution, I wanted to attach an EBS containing the images and that would be fine. However, reading a bit about EBS General Purpose (gp2) I noticed the following description:
GP2 is the default EBS volume type for Amazon EC2 instances. These
volumes are backed by solid-state drives (SSDs) and are suitable for a
broad range of transactional workloads, including dev/test
environments, low-latency interactive applications, and boot volumes.
GP2 is designed to offer single-digit millisecond latencies, deliver a
consistent baseline performance of 3 IOPS/GB to a maximum of 10,000
IOPS, and provide up to 160 MB/s of throughput per volume.
It is that 3 IOPS/GB quantity that is worrying me. What does this mean in practical terms? Suppose that you need an e-commerce site for a small amount of users (e.g. < 10,000 requests per minute) and these images need to be retrieved. Amazon describes how IOPS are measured:
When small I/O operations are physically contiguous, Amazon EBS
attempts to merge them into a single I/O up to the maximum size. For
example, for SSD volumes, a single 1,024 KiB I/O operation would count
as 4 operations, while 256 I/O operations at 4 KiB each would count as
256 operations.
Does this actually mean that if I want to retrieve 50 images of 10kB each in under a second, I would require 50 IOPS and easily exceed the baseline of 3 IOPS?
UPDATE:
Thanks to Mark B's suggestion, I was able to use S3 to upload my files. However, I'm still wondering about the amount of IOPS needed to perform common tasks such as running a database or serving other files for a web application. I would be glad to hear some reference values regarding the minimal values of IOPS based on your experience.
You are missing the "/GB" part of that statement. The baseline is 3 IOPS per GB. If your EBS volume is 100GB, then you would have a baseline of 300 IOPS. For a GP2 EBS volume you have to multiple the size of the volume by 3 to get the IOPS.
Note that any GP2 volume under 1TB is also able to burst at up to 3,000 IOPS, so any limited increases in IO should still perform very well.
Also, I will add that S3 sounds like a better fit for your use case. If you are seeing slow upload speeds to S3, that is a problem that can be solved. You can use CloudFront to provide a nearby edge location that you can upload to.
In my experience uploads to S3 are never any slower than uploads to an EC2 instance that your EBS volume would be attached to.
Update:
To answer your additional question, the minimum IOPS needed will depend on many variables such as the amount of RAM available, the type of application you are running, how well the application caches values in memory, the average size of your IO operations, etc. It's really difficult to pin-down an exact number and state that you need exactly X IOPS for an application.
You also need to remember that any volume under 1TB in size can still burst up to 3,000 IOPS for several seconds. So even if your application needs high IOPS when it is in use, if it doesn't see much usage the IOPS burst feature might be all it ever needs.
In general I usually start with something like a 100GB volume with 300 IOPS and test the performance of my app against that. A web server that operates entirely within RAM might never need more than that. For something like a database you would probably start out with the amount of disk space you think you will need and then start performance testing. CloudWatch will show the amount of IOPS your application is using, and if you see it maxing out at the limits of your volume then you would know you need to increase the available IOPS. Rinse and repeat until you no longer max out the available IOPS during your performance tests.
#Mark B's answer is probably correct, in that it points out your IOPs are based on the size of your EBS volume. For what you want, S3 is the best option.
But depending on your use case and requirements, EBS may be needed. This is especially true if you want to run a database. In that case, you have a couple of options.
You can get Provisioned IOPS - if you know you need 5000 IOPS, but only need say 100GB of storage (which with gp2 would normally provide you with around 300 IOPS), you can use io1 volumes. There is an extra cost to this, and you'll want to make sure that it's attached to an EBS optimized instance, but you can get up to 20k IOPS if needed.
If you're doing a lot of sequential reads (reading in a large data set?) then there's a new type of EBS, st1. This is good for 500MB/s, and is less than 1/2 the cost of gp2.
Finally, there's one other scenario you could consider (say, you're a bit of a madman, and want to try doing strange things). If you can grab an archive from somewhere, and all you care about is serving them up from a really fast file system, you could put them on an instance that has instance storage. This is a locally-attached SSD, so it's very fast. The only drawback is that when your instance stops, you data is gone.
To address your update, "how many IOPS do you need for a database", the answer is "it depends". Every database engine has different requirements, and every database use has different usage patterns. Take a look at this if you want more information. But basically, test & monitor. If you're worried, over provision at launch, and scale down as needed. Or take a guess, and increase if you run into problems - is it more important to minimize costs, or provide good performance to your end users?
As per your use case, s3 is a better option but if one wants to use an EBS volume and thinks that they require more IOPS, they can choose gp3 volume type instead of gp2. In gp3 volume, one can increase upto 16,000 IOPS independent of throughput (also, throughput can be increase upto 1000 MiB/s independently of IOPS).
General Purpose SSD (gp2) volumes offer cost-effective storage that is ideal for a broad range of workloads. These volumes deliver single-digit millisecond latencies and the ability to burst to 3,000 IOPS for extended periods of time. Between a minimum of 100 IOPS (at 33.33 GiB and below) and a maximum of 16,000 IOPS (at 5,334 GiB and above), baseline performance scales linearly at 3 IOPS per GiB of volume size. AWS designs gp2 volumes to deliver 90% of the provisioned performance 99% of the time. A gp2 volume can range in size from 1 GiB to 16 TiB.
link:
Link
Sometimes performance also varies:
According to AWS Doc, instance types can support maximum performance for 30 minutes at least once every 24 hours. If you have a workload that requires sustained maximum performance for longer than 30 minutes, select an instance type according to baseline performance
link:
Link
I am trying to understand the fundamental differences between several different types of storage available on AWS, specifically:
SSD
Magnetic
"Provisioned IOPS"
Snapshot storage
I am stunned to find no clear definition of each of these in the AWS docs, so I ask: How are these storage types different, and what use cases/scenarios are appropriate for each?
You're referring to Elastic Block Store (EBS). EBS provides persistent block level storage volumes for Amazon EC2 instances. EBS volumes come in 3 types:
Provisioned IOPS (SSD)
General Purpose (SSD)
Magnetic
Each type has different performance characteristics and costs. See EBS volume types for more details. The list above is ordered from high to low, by both price and by potential IOPS.
EBS snapshots are something else entirely. All EBS volumes, regardless of volume type, can be snapshotted and durably stored.
Instance storage options:
Magnetic - Slowest/cheapest magnetic disk backed storage
SSD - Faster/more expensive solid state backed storage
"Provisioned IOPS" - FastEST/most expensive but guaranteed (at the physical level) speed of input/output operations per second.
from Google:
IOPS (Input/Output Operations Per Second, pronounced eye-ops) is a common performance measurement used to benchmark computer storage devices like hard disk drives (HDD), solid state drives (SSD), and storage area networks (SAN).
This link has more fine grained details on SSD/Magnetic disk comparisons, granted it seems geared towards databases.
Snapshots are backups and are entirely separate from AWS 'hard drive' offerings.
The High I/O instance in EC2 uses SSD. How does one run a database on such an instance while guaranteeing persistance of data?
From my limited understanding, I'm suppose to use Elastic Block Store (EBS) so that even if the machine goes down the data on the disk doesn't disappear. On the other hand the instance storage SSD of a High I/O instance is ephemeral and can't be used for database storage because if, for example, the machine loses power the data image isn't preserved. Is my understanding correct?
Point 1) If your workloads need High IO SSD for DB, then you should have Master Slave setup. Ideally 1 master and 2 slaves spread across 3 AZ's is suggested. Even if there is an outage on single AZ the alternate AZ's can handle the load and serve your High availability needs. Between master - slave you can employ synchronous, semi or async replication depending upon your DB. This solution is costlier.
Point 2) Generally if your DB is OLTP in nature, then Amazon EBS PIOPS + EBS optimized gives you consistent IOPS. A Single EBS Volume can provide 4000 IOPS and you can RAID 0 multiple volumes and gain 10k+ IOPS for performance. Lots of customers are taking this route in AWS. Even though you may use EBS for persistence, it is still recommended to go with Master-Slave architecture for High Availability. I have written detailed articles on this topic in blog, refer them for more information.
It is the same as other ephemeral storage, it does not guarantee persistence. Persistance is handled by replication between instances with at least one instance writing to an EBS volume.
If you want your data to persist, you're going to need to use EBS. Building a database on an ephemeral drive, regardless of performance, seems a dubious design choice.
EBS now offers 4K IOPS volumes, which is, depending on your database requirements, quite possibly more than sufficient.
My next question would really be: Do you want to host/run your own database?
Turnkey products such as RDS and DynamoDB may be sufficient for your needs. Using them is much easier than setting up and managing your own database. RDS is now advertising "You can now provision up to 3TB and 30,000 IOPS per DB Instance". That's enough database horsepower for many, many problem sets.