Can someone help m e to understand with how the throughput works with EBS Optimised Instance. I am in process of benchmarking EBS disk for MSSQL server on top of EC2 instances. Today i did perform a IOPS test using CrystalDiskMark tool and i found that when with perfmon counters the maximum throughput its showing is the one assigned to the instance , as an example I am using X1e.4XLarge which is having a throughput of ~230 MiB/s and during test i am getting this number only , so just wanted to understand why i am not getting EBS throughput as what it supposed to provide , reason being per aws documentation with EBS optimized instances we get dedicated network between ec2 instances and ebs volumes and instance throughput is additional on top of EBS throughput..
Is that something wrong i am performing or seeing data stats with benchmarking. Highly appreciate help.
Related
I am new to Aws, recently I checked my Compute Optimizer
found the following:
"Compute Optimizer found that this instance's EBS throughput is under-provisioned."
EBS read bandwidth (MiB/second)
EBS write bandwidth (MiB/second)
both Under-provisioned from AWS Compute Optimizer.
I am not so sure how to increase, AWS Compute Optimizer showed few different instance type to switch from, but there is no information about the EBS read and write bandwidth.
There is a tab for instance and Attached EBS volumes. So should I update the instance with the recommend one or update the EBS volumes, which one will increase the EBS read/write bandwidth?
any help would be much appreciated.
I am using "m5d.8xlarge" ec2 instance, which comes ready with 2*600G SSD Volumes, directly attached. They are appearing on the OS, however no mention on the console, as I can't retrieve any info about them.
And it is showing as well the serial of the volumes as AWS-*** not as normal EBS volumes vol***.
I read that these are ephemeral or something; I want to have any AWS official docs that thoroughly explain how this local storage works, as we are hosting prod workload on it, appreciate if someone can explain or provide docs.
"m5d.8xlarge" ec2 instances comes with 2 ephimeral storage which are instance store volume.
Instance store volumes (docs) are directly attached to underlying hardware to reduce latency and increase IOPS and data throughput.
However there is a caveat, if you ec2 instance is terminated,stops, hibernated or stopped or underlying hardware gets shutdown due to some glitch all the data stored on on these ephemeral storage will be lost.
Generally instance store volumes are used for buffer,cache.
In order to confirm you can follow this https://aws.amazon.com/premiumsupport/knowledge-center/ec2-linux-instance-store-volumes/ :-
ssh into ec2 instance
install nvme-cli tool -> sudo yum instal nvme-cli
sudo nvme list - to list all instance store volumes
if you want data to persist you should go for EBS or EFS
EBS docs, EFS docs
In short If you want to access data with super low latency and you can afford to loose data go for instance store but if it is business critical data for example database workload go for EBS, YOu can still achieve very high IOPS and throughput using IO1,IO2 volume types or if you have a want to go even further use nitro ec2 instance type which gives maximum 64000 IOPS.
Play with EBS volume types to increase IOPS and throughput https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html
I've got an EBS volume (16GB) attached to a EC2 instance that has full access to an RDS instance. The thing is I've extracted the DB to the RDS instance, so I don't use the EC2 instance for storing the web application database anymore. I did this because I was having a lot of problems with the EBS credits (they were consuming very quickly). I thought that by having the DB on a separate instance (RDS) this will decrease to almost cero the EBS credit consumption because I'm not reading nor writing on the EBS but on the RDS. However, the EBS credits keep consuming (and decrease to 0) every time users access to the web application and I don't understand why. Perhaps is because I still don't fully understand how EBS credit usage works... Can anyone enlighten me with this? Thanks a lot in advance.
You can review volume types including info on their burst credits here. You should also review I/O Characteristics and Monitoring. From that page:
If your I/O latency is higher than you require, check
VolumeQueueLength to make sure your application is not trying to drive
more IOPS than you have provisioned. If your application requires a
greater number of IOPS than your volume can provide, you should
consider using a larger gp2 volume with a higher base performance
level or an io1 volume with more provisioned IOPS to achieve faster
latencies.
You should review that metric and the others it mentions if this is causing you performance problems. If your IOPs are constantly above your baseline and causing them to queue you will always consume credits as fast as they are given.
I have an Amazon RDS instance.
It has Multi AZ enabled.
Some other specs:
Instance and IOPS
Instance Class
db.t2.micro
DB Instance General Purpose (SSD)
IOPS disabled
60 GB
Lately it's been lagging really hard.
Notice the latency on read, write, and queue depth.
There is no change in Read Throughput or Write Throughput.
Can anyone assist in debugging this?
What's your database engine? If you are using MYSQL, check with profiler application to find which parts make latency.
I'm new to AWS and also to Cassandra. I just read about EBS and S3 storage available in AWS. I was trying to figure out if we have Cassandra installed in EC2, which storage would it use? EBS or S3? Or is there other storage? I'm little confused with this. Please help me understand this.
Thanks
Aravind
You shouldn't run Cassandra on EBS, as recommended per Datastax itself :
"EBS volumes are not recommended for Cassandra data volumes for the following reasons:
EBS volumes contend directly for network throughput with standard packets. This means that EBS throughput is likely to fail if you saturate a network link.
EBS volumes have unreliable performance. I/O performance can be exceptionally slow, causing the system to back load reads and writes until the entire cluster becomes unresponsive.
Adding capacity by increasing the number of EBS volumes per host does not scale. You can easily surpass the ability of the system to keep effective buffer caches and concurrently serve requests for all of the data it is responsible for managing."
http://docs.datastax.com/en/cassandra/1.2/cassandra/architecture/architecturePlanningEC2_c.html
The answer above comes from Cassandra 1.2, a relatively old version. Documentation for newer versions of Cassandra indicate that EBS Optimized instances using GP2 SSD can be used for production workloads.
http://docs.datastax.com/en/cassandra/3.x/cassandra/planning/planPlanningEC2.html
Things that changed since then were the creation of EBS Optimized instances, which reduces and/or eliminates noisy neighbor throughput problems, and using GP2 SSD for EBS storage.
If you are just getting started, I would recommend EBS Optimized. The performance should be pretty good, but you gain a critical ability -> creating snapshots. This reduces the risk of your instance becoming unstable because you would have S3-backed volume snapshots for AWS to rebuild data from if a drive died.
This reduces the need to setup your Cassandra cluster across regions. One of the concerns that you have to build around when using Ephemeral is a whole region potentially going down, which could wipe out your entire cluster if you didn't build a multi-region cluster. With EBS, this isn't really a concern.
For Cassandra you need to use EBS. S3 is an object store with and API to store and retrieve objects, but not easy querying mechanisms. The use cases include backup and archiving, Disaster Recovery, Static Website Hosting, etc
However, you can use S3 for Cassandra backup.
You can also consider ephemeral disks (as Jeff mentions) and storage which comes with AWS instance.