I created a m3.large EC2 instance for data processing, I modified it to a 40GB root drive and it comes with a 3OGB additional drive. I only need to use it for a few days a month so the idea was to stop it when not in use and start it when required.
When I stopped it I was warned that
Emphermal storage data would be lost
on restarting this turned out to be a way of cryptically saying that everything on the 30Gb volume would be wiped which it has been, whereas the root volume that uses the 4OGB is untouched.
So why does it do this and if I hadn't increased the size of my root volume would all my changes on that be lost as well ?
There 2 types of Volumes ( Disk ) provided by AWS
EBS Volumes
Ephemeral ( Instance Store )
EBS Volumes :
Your root volume is always EBS volume ( you can safely assume this point; almost all of the AMIs EBS based these days). The data in EBS volume would persist even if the instance is stopped or terminated; with this you can quickly remove the EBS volumes from one instance and reattached it to other.
Ephemeral ( Instance Store ) :
Ephemeral volumes are the temporary drives provided for the instance, the size ( 10 GB or 80 GB ), the count ( 2 disks or 5 disks ) the type ( Magnetic or SSD ) depends with each instance size.
The data in these drive would be wiped off during the event of instance stop and start. You may use this disk for non critical application activities like temporary file storage and processing, application log. If you need persistence of the content in this drive then you would need to move it to EBS based volumes or store it in S3 from time to time to prevent data loss.
There wont be any loss of data when you increase the size of the volume of an EBS Volume; you need take the snapshot of the volume and restore it again back to a volume of your required size ( increased size ). I have tried this couple of times; I haven't encountered any data loss during the process; but if you little concerned about this, you can upload the content to S3 for backup.
Related
I want to launch an instance with custom size of root volume on Amazon EC2.
At step no.4 - Add Storage, the root volume default setting came with a default snapshot and a default 10 size (GiB). Then I lowered the size into 5GiB.
But at the final step, it wouldn't allow me to launch the instance because of only 5GB root volume.
Any idea or solution that I can lower the size when launching an instance?
You cannot create an Amazon EBS volume that is smaller than the snapshot that you want loaded.
You could attempt to make your own AMI by launching with 10GB, attaching a 5GB volume, copying files across, turning it into an AMI, etc but frankly it isn't worth the bother.
If you are merely wishing to save money, then at 10¢ per GB per month, it would only save 50¢ per month.
I got my website running on an EC2 instance with 1 EBS of 50 GB attached as root volume. My requirement is to reduce the size of this EBS volume to 20 GB from currently 50 GB. I tried lot of ways like creating an image and launching a new instance with EBS volume as 20GB but in the end launch fails with message saying that volume size can only be increased from current size (i.e. 50GB) and cannot be reduced.
May I know if there is any workaround to handle this issue. I have data in root volume but I do not require whole bunch of 50GB space on my root volume. Please assist.
Thank you,
You would need to create a new volume and copy everything across, making sure that the volume remains bootable.
Then: Stop instance, detach volume, attach new volume, boot.
At 10c/GB/month, the reduction of 30GB will save you $3/month. It isn't worth your time.
I have the following use case:
I have an EC2 instance running with an 1000GB io1 SSD EBS root volume attached
It is used for a data collection service which accumulates data every minute
I like to change the volume for a cheaper storage system (as the specs of the process are not IOPS intensive)
I've read through the AWS documentation and found the following articles:
https://aws.amazon.com/about-aws/whats-new/2017/02/amazon-elastic-block-store-amazon-ebs-enables-live-volume-modifications-with-elastic-volumes/
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-expand-volume.html#console-modify
Which indicate that (from february 2017) for Current Generation Instances a volume type change is possible on the fly in the console without stopping the instance. However, when I go to my console I can only choose between io1 (current) or gp2 (also SSD), I'd like to choose sc1 (cold storage) for this instance. The video clearly shows this option (go from io1 to sc1).
Am I missing something? Is there a restriction to go from SSD to HDD?
(im in Asia Sydney region).
Alternatively, if the option is not possible and i have the requirement that data collection continues, it seems that my only option is:
re-create a similar instance with the data collection scripts deployed with an EBS sc1 volume as root attached
get the data collection running (now collecting double)
stop the old instance and take a snapshot of the io1 EBS volume
create a EBS sc1 volume with the snapshot
attach the EBS sc1 volume to the new instance (as additional volume, not root)
Would this be the right way to do so? (if I can't change the volume type to sc1 while it's running)
tldr;
The 2 main reasons why amazon will not propose sc1/st1 type when editing the EBS volumes are:
the current volume is used as root device of the ec2 instance
the current storage is less than 500 Gb
I guess the volume you want to change is used as root device of the instance; in this case you cannot have boot volume of magnetic type st1 or sc1.
Note also the sc1 volume must be at least 500 Gb
You can read more about EBS Volume Types
Unfortunately, you cannot change the root device of a running instance, you can do that while the instance is stopped.
so you can follow your steps proposed but you can make the change of the EBS after it will be detached from the first ec2 instance
re-create a similar instance with the data collection scripts deployed with an EBS sc1 volume as root attached (You cannot have sc1/st1 drive as boot volume; if you want magnetic storage you need to select standard Magnetic storage; but ideally you should have a low volume ssd drive as boot volume and the large magnetic drive as additional ebs volume)
get the data collection running (now collecting double)
stop the old instance and detach the EBS volume
make the modification of the EBS volume (type/storage)
attach the EBS sc1 volume to the new instance (as additional volume, not root)
Is AWS root storage ephemeral? The documents say they are ephemeral. But, I still can find my files after stopping the instance for some time and reboot it, I can still view my files
When you created your instance, I assume you chose the EBS backed instance. This means that the root device volume is actually an EBS volume, so when the instance restarts, you still have all of your data. If you were to choose the instance-store backed instance, then you would lose your data when restarting.
See http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/RootDeviceStorage.html for more information.
What do you mean by root storage?
Instance Stores are ephemeral. EBS volumes are not.
When launching an instance, if you've selected Instance Store as the root volume, a restart will still retain the data. But a stop and start will delete all data.
If you've selected an EBS volume, the data will be retained.
Using ec2 Windows instance with Instance storage (let's say 32GB SSD) - where OS and its settings are stored? Like Program Files, User profiles. Are they all stored on Instance Storage? As far as I understood from other topics Instance storage is not-persistent and doesn't survive shutdowns/terminations. Does that mean I will lose everything under C: drive if I turn it off?
Can I use EBS storage as a default storage for OS (C drive)? Can I map multiple EBS storages to one Windows storage?
If above is true, then I will be charged for the capacity used by OS on EBS instance? It would be around 20GB I believe. Is that correct?
I am quite new in aws, and before paying for such instances or EBS I would like to know how this technical and billing model is working.
Thank you!
The Storage for the Root device is dependent on the AMI (EBS-Backed or Instance Store-Backed) used to launch the instance.
As far as I understood from other topics Instance storage is
not-persistent and doesn't survive shutdowns/terminations.
If the Root storage device is Instance Store, Stopping (shutdown) the instance is not possible. On termination, Both the storage and Instance does not survive. The Instance does not survive once terminated even if the AMI is EBS-Backed, but you can persist the Root Volume by setting the DeleteOnTermination flag set to False.
Does that mean I will lose everything under C: drive if I turn it off?
You cannot turn off (shutdown) an Instance Store-backed instance.
Can I use EBS storage as a default storage for OS (C drive)?
Yes, Choose an EBS backed Windows AMI.
Can I map multiple EBS storages to one Windows storage?
Yes, multiple EBS Volumes can be attached to one EC2 Windows Instance.
If above is true, then I will be charged for the capacity used by OS
on EBS instance?
You will be charged for the total size of the EBS volumes attached to the instance including the Root Device.
It would be around 20GB I believe. Is that correct?
The EBS Volume Size is adjustable. The upper Size limit is 16TiB.
Read Storage for Root Device and Ec2 Root Device Volume
Please spend more time on the AWS documentation, I don't think here is enough to cover all your question.
Only for specify EC2 instance come with attached SSD storage AKA instance storage. Bare in mind that, this instance storage doesn't come with Snapshot capabilities, so you must backup the file yourself. This is mean for people who need fastest disk access to process their data.
Only EBS allow you do multiple snapshot.
You can always create an AMI image for your instance after complete the deployment. AMI image is store inside EBS, so you will not lost the initial instance if you do this, so for new instance, you just trigger load it from AMI.
If you "Terminate" an instance, it will delete the virtual image. There is no way to recover it even with EBS, unless you make a snapshot. However, attached EBS storage will not be deleted.
EBS is calculate by Per GB and give you 1GB x 3 IOPS, with base 100 IOPS given. This is not enough if anyone want to carry out disk I/O intensive task.