I am new to AWS functionality and is confused in these 3 things.
To me if I consider
Memory as the RAM
Instance Storage as the hard disk space
then what is size and volume type means?
When you launch an instance on EC2, Amazon has to look for a physical server that will host your instance with enough unallocated capacity to be able to run your instance.
In the case of an m1.medium instance, this physical host would need to have enough unallocated resources so that the m1.medium instance specs would fit into it:
at least 1 unallocated core
at least 3.7 GiB of unallocated RAM
at least 1 disk with 410 GiB of unallocated space
So, from this description you see that "Memory" is the amount of RAM and "Instance Storage" is the amount of disk space inside the physical host that is running your instance.
Note that, I insist, this "Instance Store" is disk space local to the physical host. What does this imply? Well, if you stop the instance, you will deallocate all those resources so that another customer can use them. This means that you will deallocate the cores, the RAM and the disks. Which means the data saved on Instance Store is lost when you stop/terminate the instance, or whenever the physical host running the instance fails for whatever reason ("Everything fails, all the time" -- Werner Vogels, CTO at Amazon). This is why Instance Store is called ephemeral storage.
If you want persistent storage, then you'd need to use a service called Amazon EBS -- Amazon Elastic Block Store. In EBS, you create volumes. EBS Volumes are a kind of network attached storage. You can attach a Volume to any EC2 Instance in the same Availability Zone, then you can detach without losing data, then attach to another instance, and so on. When you stop an Instance, you don't lose data stored on EBS Volumes -- this is why they are called persistent storage.
In the screenshot in your question, what you see is that the Root volume (i.e., the "disk" where your operating system will be running) is an standard EBS volume (there's another kind of EBS volume, called PIOPS). This implies that any OS settings you change (and save to the root volume) will be persisted and survive an stop-start sequence, or instance restarts due to crashes.
There are some AMIs (Amazon Machine Images) that use Instance Store as the Root Volume. Instances launched with those AMIs won't persist any changes to the OS settings that are saved to the root volume -- therefore, if you stop them and start them again, you'd get a fresh OS.
I hope this answers your questions.
Related
Obviously, hibernation and stop are two different actions that I can select.
What's the difference?
Benefit of Hibernating over Stopping
The memory state is preserved
Since the memory state is perserved and loaded again when the instance start, this reduce the boot time of the instance.
The long running process can continue without interuption
A great benefit if you have some services that take a great amount of time to fully initialized
Under the hood
The whole hibernation process in visual:
When the instance is in Stopping state, the instance memory is persisted in the instance's EBS root volume, and is loaded again when the instance start.
Reference
AWS Instance Hibernate Overview
From the docs
When you hibernate an instance, Amazon EC2 signals the operating
system to perform hibernation (suspend-to-disk). Hibernation saves the
contents from the instance memory (RAM) to your Amazon Elastic Block
Store (Amazon EBS) root volume. Amazon EC2 persists the instance's EBS
root volume and any attached EBS data volumes. When you start your
instance:
The EBS root volume is restored to its previous state
The RAM contents are reloaded
The processes that were previously running on the instance are resumed
Previously attached data volumes are reattached and the instance retains its instance ID
Read more
TL;DR
When you stop your instance, the data stored in memory (RAM) is lost.
When you stop-hibernate an instance, AWS signals the OS to perform hibernation (suspend-to-disk), which saves the contents from the instance memory (RAM) to the Amazon EBS root volume.
From the charging perspective, AWS does not charge usage or data transfer fees for your instance after you stop it, but storage for any Amazon EBS volumes is still charged.
A practical example
Suppose you want to build a caching layer (e.g. on top of your DB) in an EC2 instance. For such a case, the stop-hibernate feature would be instrumental in persisting storage. It would prevent you from having to manually create scripts to save the RAM data before shutting down the server.
We have my staging environment set up on the Linux machine with type t2 micro.
Now we are trying to ramp up the instance but worried about the dependencies and the file structure which is present on the machine.
What happens when we change the instance type?
Does it have a new EBS attached to it( 1gb will be switched by 16gb)?
OR
Does it add to the already existing EBS volume (1gb + 16gb)?
Whenever you update the instance type it will perform the following process:
Shutdown the instance
Update the instance type (if its possible, some are incompatible).
Restart the instance.
Any EBS volumes will persist this process, but if you have a local volume such as NVMe or instance store this disk will be replaced with another local disk on the new hardware.
I believe what you're referring too isn't the EBS volume storage it is the memory (RAM) that is available to the instance. If you wanted to increase the amount of storage you would instead need to increase the EBS size.
I have one service running on my t2.micro instance. How can i confirm if there is any bandwidth and storage limit or not.
Attached is that status screen shot of it.
Please guide me on this.
Thanks
Bandwidth
The only bandwidth limitation in AWS is related to the Instance Type of Amazon EC2 instances.
Put simply, smaller instances have less bandwidth than larger instances. You'll see this in the Launch Instance screen (right column):
The documentation doesn't specifically say what bandwidth you are given, but you can run some performance tests to determine available throughput.
Storage
There are two types of disk storage for Amazon EC2 instances:
Instance Storage
Amazon Elastic Block Store (EBS)
Instance Storage is disk storage that is directly-attached to the instance (or, more accurately, to the host computer running the instance). As you'll see in the Instance Storage column in the above picture, not all EC2 instances have Instance Storage -- some of them just say "EBS only", meaning that it is not available.
For instance types that provide Instance Storage, the size is fixed and is again based on the Instance Type.
The most important thing to know about Instance Storage is that it is lost when an instance is stopped/terminated. This is because the virtual machine is deleted, which gives back the CPU, RAM and Instance Storage. Thus, it is only useful for temporary files, virtual memory swap files and local cache. Do not store the only copy of important data on Instance Storage.
Amazon EBS is network-attached storage. Data is retained when instances are stopped. When they are later started, the disks are exactly the same as when the instance was turned off. When an instance is Terminated, the EBS volume can optionally be kept or deleted.
EBS volumes have the advantage that you can configure a disk of any size up to 16TB and there are various different types of volumes to trade-off cost/performance.
Bottom line: Your t2.micro instance has no Instance Storage. It has EBS volumes that you have attached (at whatever size you configured). It has Low to Moderate network bandwidth.
Using ec2 Windows instance with Instance storage (let's say 32GB SSD) - where OS and its settings are stored? Like Program Files, User profiles. Are they all stored on Instance Storage? As far as I understood from other topics Instance storage is not-persistent and doesn't survive shutdowns/terminations. Does that mean I will lose everything under C: drive if I turn it off?
Can I use EBS storage as a default storage for OS (C drive)? Can I map multiple EBS storages to one Windows storage?
If above is true, then I will be charged for the capacity used by OS on EBS instance? It would be around 20GB I believe. Is that correct?
I am quite new in aws, and before paying for such instances or EBS I would like to know how this technical and billing model is working.
Thank you!
The Storage for the Root device is dependent on the AMI (EBS-Backed or Instance Store-Backed) used to launch the instance.
As far as I understood from other topics Instance storage is
not-persistent and doesn't survive shutdowns/terminations.
If the Root storage device is Instance Store, Stopping (shutdown) the instance is not possible. On termination, Both the storage and Instance does not survive. The Instance does not survive once terminated even if the AMI is EBS-Backed, but you can persist the Root Volume by setting the DeleteOnTermination flag set to False.
Does that mean I will lose everything under C: drive if I turn it off?
You cannot turn off (shutdown) an Instance Store-backed instance.
Can I use EBS storage as a default storage for OS (C drive)?
Yes, Choose an EBS backed Windows AMI.
Can I map multiple EBS storages to one Windows storage?
Yes, multiple EBS Volumes can be attached to one EC2 Windows Instance.
If above is true, then I will be charged for the capacity used by OS
on EBS instance?
You will be charged for the total size of the EBS volumes attached to the instance including the Root Device.
It would be around 20GB I believe. Is that correct?
The EBS Volume Size is adjustable. The upper Size limit is 16TiB.
Read Storage for Root Device and Ec2 Root Device Volume
Please spend more time on the AWS documentation, I don't think here is enough to cover all your question.
Only for specify EC2 instance come with attached SSD storage AKA instance storage. Bare in mind that, this instance storage doesn't come with Snapshot capabilities, so you must backup the file yourself. This is mean for people who need fastest disk access to process their data.
Only EBS allow you do multiple snapshot.
You can always create an AMI image for your instance after complete the deployment. AMI image is store inside EBS, so you will not lost the initial instance if you do this, so for new instance, you just trigger load it from AMI.
If you "Terminate" an instance, it will delete the virtual image. There is no way to recover it even with EBS, unless you make a snapshot. However, attached EBS storage will not be deleted.
EBS is calculate by Per GB and give you 1GB x 3 IOPS, with base 100 IOPS given. This is not enough if anyone want to carry out disk I/O intensive task.
I thought I clearly understood the difference between instance-store and EBS backed AMIs.
But http://aws.amazon.com/maintenance-help/ says "if you are running an EBS-backed AMI, you can stop and then restart your instance in order to easily re-launch it. This will cause the loss of any data you have saved on the local instance store of the instance,"
Stop/start does NOT lose the sysvol data, so this confuses me.
I'm assuming that here, by "local instance store", they mean the backing EBS volume (the sysvol), and I'm thinking that they meant to say "terminate" instead of stop. Am I correct?
Terminating an EBS-backed instance will not cause your data to be deleted. You can still access the EBS volume until you delete it (unless you set it to delete when your instance is terminated).
Local instance store refers to hard drive space on the actual physical server that is running your instance. You can see the available instance store by doing sudo fdisk -l. Some images come with some instance store volumes already mounted (see df -h). Otherwise you'll have to mount and format the instance store volumes before you can use them.
Data on an instance store volume is lost when you stop (not terminate) your instance because it is local to a physical server, and your instance might start up on a new server.
Quite simply, EC2 is running your virtual server on some physical server. The root filesystem can either be on a local disk (ephemeral storage) or on network attached storage (EBS). With EBS, they can snapshot it for backups or to make a copy, so EBS is far more flexible, although not as fast as a local disk in the server where your instance is running.
In order to make this all work, when you shutdown an ephemeral server, amazon wipes the disk in order to reallocate it to the next customer. There is no need or reason for them to do that with EBS, since it was not physically attached to that server in the first place.
You might note, that even EBS backed instances (depending on size) come with an allocation of ephemeral storage (2-500gig+) which can be used for swap, logs, or whatever else you want to do with them. The only issue of course is that should the server be shutdown, or should there be a catastrophic disk or hardware error, you'll lose that data. You can still manually back it up, in the same way people have backed up traditional servers over the years.
Making your own AMI from an EBS backed server is trivial now, and can be done easily through the AWS web interface. Making a non-EBS backed AMI is a very complicated task the last time I tried to do it. With that said, there are certain use cases where it makes a lot of sense to consider using purely ephemeral storage. Computation or memory/cache nodes that have no need to persist data will be faster and cost less.