Currently i have created EBS Backed(30GB) EC2 Instance, data in EBS has reached to almost maximum capacity.
What would be best practice to increase the size of EBS volume with high availability
a) Extend the existing EBS volume by taking snapshot
b) Attach new EBS Volume
In case we extend the EBS volume we have to take snapshot ,then stop instance and recreate ebs volume with this snapshot.
Will EC2 Instance boot from newly created EBS volume.
Ideal Way of doing this !
Yes precisely you are thinking in the right direction.
You could automate it with multiple sequential lambdas or bash would be also an ideal solution to your query.
How much should space should we increase?
Above will depend on your EBS storage filling rate. Make a wise choice accordingly.
Related
Actually I have to take forecast of costing for one my instance, which is having a number of volumes attached... These volumes are different in size and types.
Let's suppose I took the AMI backup and terminated the server.
Now my confusion is how would I calculate the cost. The cost will be calculated based on pricing of Amazon EBS Volumes or Amazon EBS Snapshot. Because the cost difference is just double.
Let me know if you can help me understanding.
pricing of Amazon EBS Volumes or Amazon EBS Snapshot Which I took from AWS Pricing :
https://aws.amazon.com/ebs/pricing/
Amazon EBS snapshots are a complex subject due to the way they work.
There is a detailed explanation in: Amazon EBS snapshots - Amazon Elastic Compute Cloud
A quick summary is:
Snapshots contain only the data that is different to previous snapshots (they are incremental)
An AMI is actually a snapshot. So, if you booted a new Amazon EC2 instance from an AMI and then created a snapshot, the snapshot would contain very little since most of the volume was already contained in the previous snapshot (that was part of the AMI). Confused yet?
Any snapshot can be deleted and information will still be retained to allow any other snapshot to be restored. So, the snapshot is actually an 'index' to the snapshot data, and the snapshot data is stored separately to the snapshot itself. You should be questioning your sanity at this point!
So, the cost of Amazon EBS snapshots is mostly based on how much the contents of the volume changes, and how many snapshots (effectively, points-in-time) you wish to keep. If you only keep the most recent snapshot, then all data will be available, but the cost will be minimised because it won't keep any data that has been deleted from the volume.
Bottom line: Snapshots take less space than the data on a volume due to the incremental natures. The more snapshots ("points-in-time"), the more data will be kept and hence the more cost.
I am preparing for AWS certification and I found following question in mock test.
The Question is as mentioned in below image :
And they have mentioned EBS volume in the question, I selected to choose "Provisioned IOPS SSD Volume" to implement scalable and high throughput.
But the correct answer was EFS with the following justification.
But, I think EBS volume can only be mapped with one EC2 instance at a time. Can we map one EBS volume with feet of multiple EC2 instance ?
No, you can't map an EBS volume to more than one instance at a time.
But EFS doesn't use EBS, and an EFS filesystem had no meaningful limit on the number of EC2 instances that can access it simultaneously.
The question isn't a very good one. In fact, it proposes an initial scenario that you would never use.
EBS volumes attached to members of an auto-scaling group would never be used to store CMS documents uploaded by users, because those volumes will ether be destroyed or left attached to nothing when the cluster scales in and some of the instances are terminated due to the decreased load.
The giveaway to the correct answer lies in the fact that the question asks for a scalable, high-throughput, POSIX-compliant filesystem and this is pretty much the definition of Amazon EFS. EFS will scale larger than the largest provisioned IOPS EBS volume.
I understand that EBS volumes that are restored from snapshots need to be initialized.
Let's say I create an AMI from an EBS snapshot(s).
When I launch a new instance using that AMI, will I need to initialize each volume before I get consistent latency?
When you create an instance from AMI which has EBS based volumes as storage, your instance creates new volume from this volume and continues with launch.
When you use snapshots as source to launch your instance, AWS need to pull these snapshots and write it to volume and then continue with the launch. In this case, pull storage blocks from S3 and write to volume will be the latency some users want to avoid. This is the case where pre-warming is suggested by AWS.
I have the following use case:
I have an EC2 instance running with an 1000GB io1 SSD EBS root volume attached
It is used for a data collection service which accumulates data every minute
I like to change the volume for a cheaper storage system (as the specs of the process are not IOPS intensive)
I've read through the AWS documentation and found the following articles:
https://aws.amazon.com/about-aws/whats-new/2017/02/amazon-elastic-block-store-amazon-ebs-enables-live-volume-modifications-with-elastic-volumes/
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-expand-volume.html#console-modify
Which indicate that (from february 2017) for Current Generation Instances a volume type change is possible on the fly in the console without stopping the instance. However, when I go to my console I can only choose between io1 (current) or gp2 (also SSD), I'd like to choose sc1 (cold storage) for this instance. The video clearly shows this option (go from io1 to sc1).
Am I missing something? Is there a restriction to go from SSD to HDD?
(im in Asia Sydney region).
Alternatively, if the option is not possible and i have the requirement that data collection continues, it seems that my only option is:
re-create a similar instance with the data collection scripts deployed with an EBS sc1 volume as root attached
get the data collection running (now collecting double)
stop the old instance and take a snapshot of the io1 EBS volume
create a EBS sc1 volume with the snapshot
attach the EBS sc1 volume to the new instance (as additional volume, not root)
Would this be the right way to do so? (if I can't change the volume type to sc1 while it's running)
tldr;
The 2 main reasons why amazon will not propose sc1/st1 type when editing the EBS volumes are:
the current volume is used as root device of the ec2 instance
the current storage is less than 500 Gb
I guess the volume you want to change is used as root device of the instance; in this case you cannot have boot volume of magnetic type st1 or sc1.
Note also the sc1 volume must be at least 500 Gb
You can read more about EBS Volume Types
Unfortunately, you cannot change the root device of a running instance, you can do that while the instance is stopped.
so you can follow your steps proposed but you can make the change of the EBS after it will be detached from the first ec2 instance
re-create a similar instance with the data collection scripts deployed with an EBS sc1 volume as root attached (You cannot have sc1/st1 drive as boot volume; if you want magnetic storage you need to select standard Magnetic storage; but ideally you should have a low volume ssd drive as boot volume and the large magnetic drive as additional ebs volume)
get the data collection running (now collecting double)
stop the old instance and detach the EBS volume
make the modification of the EBS volume (type/storage)
attach the EBS sc1 volume to the new instance (as additional volume, not root)
Hi I want to use the recommended storage for Cassandra which is 1-3 TB per node. I am wondering how to increase the VMs to have that much hard drive space with SSD or HDD
What have you tried exactly?
You can specify the EBS volume size you want when you create the EC2 instance.
You can add a new EBS volume to an existing EC2 instance.
You can grow an existing EBS volume: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-expand-volume.html
You can use multiple EBS volumes in a RAID configuration for performance and ease of increasing space: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/raid-config.html