I've got an EBS volume (16GB) attached to a EC2 instance that has full access to an RDS instance. The thing is I've extracted the DB to the RDS instance, so I don't use the EC2 instance for storing the web application database anymore. I did this because I was having a lot of problems with the EBS credits (they were consuming very quickly). I thought that by having the DB on a separate instance (RDS) this will decrease to almost cero the EBS credit consumption because I'm not reading nor writing on the EBS but on the RDS. However, the EBS credits keep consuming (and decrease to 0) every time users access to the web application and I don't understand why. Perhaps is because I still don't fully understand how EBS credit usage works... Can anyone enlighten me with this? Thanks a lot in advance.
You can review volume types including info on their burst credits here. You should also review I/O Characteristics and Monitoring. From that page:
If your I/O latency is higher than you require, check
VolumeQueueLength to make sure your application is not trying to drive
more IOPS than you have provisioned. If your application requires a
greater number of IOPS than your volume can provide, you should
consider using a larger gp2 volume with a higher base performance
level or an io1 volume with more provisioned IOPS to achieve faster
latencies.
You should review that metric and the others it mentions if this is causing you performance problems. If your IOPs are constantly above your baseline and causing them to queue you will always consume credits as fast as they are given.
Related
Can someone help m e to understand with how the throughput works with EBS Optimised Instance. I am in process of benchmarking EBS disk for MSSQL server on top of EC2 instances. Today i did perform a IOPS test using CrystalDiskMark tool and i found that when with perfmon counters the maximum throughput its showing is the one assigned to the instance , as an example I am using X1e.4XLarge which is having a throughput of ~230 MiB/s and during test i am getting this number only , so just wanted to understand why i am not getting EBS throughput as what it supposed to provide , reason being per aws documentation with EBS optimized instances we get dedicated network between ec2 instances and ebs volumes and instance throughput is additional on top of EBS throughput..
Is that something wrong i am performing or seeing data stats with benchmarking. Highly appreciate help.
Given What is difference between Lightsail and EC2? and my own testing, I am convinced that LightSail Instances are burstable performance instances.
Are LightSail database instances subject to this kind of burstable performance too? I'm interested in both I/O burst and CPU burst.
If so, is there a way to view the remaining CPU or IO burst credits? Unlike EC2, the web console for LightSail doesn't seem to offer these numbers.
This issue hit me hard today, taking down my instances making them unable to serve any request
Just want to inform you that they include the bursts area into their metric now. But I dont see the way to get remaining credits
Also now the burstable "feature" is documented
To Devs,
I am getting lazy reads when I create an EBS volume from a snapshot and attach it to an EC2 node.
I would like to create an EBS volume with a complete copy so that the first read is not slow.
Is there a way to do this?
Thanks,
Marc
You and everybody else. According to an AWS rep that I talked with at the AWS Summit in NYC, Amazon is well aware of the issue. Of course, there's a difference between "being aware" of an issue and actually fixing it ...
For now, the best you can do is follow the AWS instructions and use dd or fio to touch every block on the device before it's mounted. The benefit of fio is that it will run parallel threads.
Beware that you will be limited by the IO performance of your volume. One IO is 16k on an gp2 volume, so divide your volume size by that to determine how many IOs it will take to touch every block, and then divide that by the IOPS for your volume (taking into account burst IOPS).
For example (and these are rough numbers!), a 1 TB volume will require 67,108,864 IOs to read fully. The default non-provisioned performance of a 1TB gp2 volume is 3,000 IOPS, this will take 22,369 seconds or somewhat more than 6 hours. Smaller volumes will be able to use burst IOPS to get above their basic allotment, but may run into throughput limits.
I have an Amazon RDS instance.
It has Multi AZ enabled.
Some other specs:
Instance and IOPS
Instance Class
db.t2.micro
DB Instance General Purpose (SSD)
IOPS disabled
60 GB
Lately it's been lagging really hard.
Notice the latency on read, write, and queue depth.
There is no change in Read Throughput or Write Throughput.
Can anyone assist in debugging this?
What's your database engine? If you are using MYSQL, check with profiler application to find which parts make latency.
The High I/O instance in EC2 uses SSD. How does one run a database on such an instance while guaranteeing persistance of data?
From my limited understanding, I'm suppose to use Elastic Block Store (EBS) so that even if the machine goes down the data on the disk doesn't disappear. On the other hand the instance storage SSD of a High I/O instance is ephemeral and can't be used for database storage because if, for example, the machine loses power the data image isn't preserved. Is my understanding correct?
Point 1) If your workloads need High IO SSD for DB, then you should have Master Slave setup. Ideally 1 master and 2 slaves spread across 3 AZ's is suggested. Even if there is an outage on single AZ the alternate AZ's can handle the load and serve your High availability needs. Between master - slave you can employ synchronous, semi or async replication depending upon your DB. This solution is costlier.
Point 2) Generally if your DB is OLTP in nature, then Amazon EBS PIOPS + EBS optimized gives you consistent IOPS. A Single EBS Volume can provide 4000 IOPS and you can RAID 0 multiple volumes and gain 10k+ IOPS for performance. Lots of customers are taking this route in AWS. Even though you may use EBS for persistence, it is still recommended to go with Master-Slave architecture for High Availability. I have written detailed articles on this topic in blog, refer them for more information.
It is the same as other ephemeral storage, it does not guarantee persistence. Persistance is handled by replication between instances with at least one instance writing to an EBS volume.
If you want your data to persist, you're going to need to use EBS. Building a database on an ephemeral drive, regardless of performance, seems a dubious design choice.
EBS now offers 4K IOPS volumes, which is, depending on your database requirements, quite possibly more than sufficient.
My next question would really be: Do you want to host/run your own database?
Turnkey products such as RDS and DynamoDB may be sufficient for your needs. Using them is much easier than setting up and managing your own database. RDS is now advertising "You can now provision up to 3TB and 30,000 IOPS per DB Instance". That's enough database horsepower for many, many problem sets.