I am having a problem here in AWS Instance I already created the volume but when I check in the Remote Desktop it only shows the 100GB storage.
Check this Image for reference,
Volume Storage
Attaching new volume to an instance is not enough to be able to use it. You have to format it for your operating system. For example, for linux:
Make an Amazon EBS volume available for use on Linux
Related
I am learning about aws and using ec2 instances. I am trying to understand what a volume is.
I have read from the aws site that:
An Amazon EBS volume is a durable, block-level storage device that you
can attach to your instances. After you attach a volume to an
instance, you can use it as you would use a physical hard drive.
Is it where things are stored when I install things like npm and node? Does it function like the harddrive o my server?
AWS EBS is block storage volume, and for the ease of understanding, yes you can consider it same as hard drive, however with more benefits over traditional hard drive. few of them are:
You can increase/decrease size of the storage as per your requirement
(Hence name Elastic)
You can add multiple ebs to your instances, for example 20 GB of volume1 and 30 GB of volume2
And for the question you asked if you can install npm & node yes you
can as it would be attached to your EC2 instance and your instance
can easily utilised attached data, modules,etc
For further explanation you can refer this user guide from AWS on EBS: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volumes.html
Yes it is exactly like a hard drive on your server and you can have multiple devices.
The cool thing is that you also can expand them if you need extra space.
where things are stored when I install things like npm and node
Yes, technically ebs volume is virtual storage drive which is
connected to your instance through network, ( flash drive connected over network).
Since network is involved which implicitly means there will be some latency because of data transfer through network.
The data is persistent even if instance stops,terminates, hibernates or hardware failure.
Since it is network drive it can be attached or detached to any other instance.
Adding to this there is another type of storage which you will find called as instance store
You can specify instance store volumes for an instance only when you launch it. You can't detach an instance store volume from one instance and attach it to a different instance.
it gives very high IOPS because it is directly (physically) attached to instance.
The use case for instance store would where data changes rapidly like for cache or buffers.
Your data will be lost if any of these events happens like The underlying disk drive fails The instance stops, instance hibernates instance terminates or drive failure.
I created two virtual disks for AWS EC2. One is 16G and the other is 100G.
When I connect to EC2 with puTTy it does not show the disks that I created.
Anyone know what the problem is or how to fix it? Thanks.
Make an Amazon EBS volume available for use on Linux
After you attach an Amazon EBS volume to your instance, it is exposed as a block device. You can format the volume with any file system and then mount it. After you make the EBS volume available for use, you can access it in the same ways that you access any other volume. Any data written to this file system is written to the EBS volume and is transparent to applications using the device.
Depending on your requirement you can choose two ways:
Format and mount an attached volume
Automatically mount an
attached volume after reboot
I have a VMware VM whose OS raw disk is backed up to AWS S3. I can create AMI from the OS disk raw using import-image. I cannot use import-image everytime because it is extremely slow and because I am creating an application where you can backup your VM to AWS cloud, where in the first backup will be FULL backup which will take longer, but the consequent INCREMENTAL backup should take very less time(depending on the amount of data changed).I am creating AMI during every backup i.e. FULL or INCREMENTAL backup.
Hence, it is OK and explainable that FULL backup is taking time but for INCREMENTAL it should take less time.
The problem is, while creating AMI from RAW data during incremental backup, AWS is not aware that there is already an AMI (and also corresponding EBS snapshot) created during FULL backup which should be used(or compared) with latest data to find data changes and hence should create AMI out of the changed data only, which will in turn take less time.
So, I have following options :
1) import-snapshot API = converts the raw OS disk to EBS snapshot file.
2) copy OS Disk data = create a EBS volume and attach it to a running EC2 instance. Then copy all OS disk raw data to the volume. Then create snapshot from the EBS volume. From the EBS snapshot, we can create AMI.
I have tried both options but everytime I try to launch EC2 instance from the AMI, I get below error :
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,0)
After going through various forums, I came to know that the above error occurs if there is mismatch in AKI and ARI while creating AMI from snapshot. Correct AKI and ARI is fetched from source EC2 instance from which the snapshot is created (as this is expected by AWS).
In my case, I have not created snapshot from a running EC2 instance but from a VMWare VM OS disk.
I figured out that import-image API also creates snapshots while creating AMI. So, I compared the snapshot created by import-image and the snapshot created by me using option-1 and option-2.
I compared the list of files in /boot/ and their md5sum. I found out the snapshot created by AWS import-image API has "initramfs-3.10.0-327.36.3.el7.x86_64.img.vmimport" file and has modified many files in /boot/grub2 directory.
As per AWS documentation, https://docs.aws.amazon.com/vm-import/latest/userguide/vm-import-ug.pdf,
AWS modifies filesystem :
- installs Citrix PV drivers either directly in OS or modifies initrd/initramfs to contain them,
- modifies /etc/fstab,
- modifyies grub bootloader settings such as the default entry and timeout.
So, do I need to do above changes in my EBS volume as well ? How to do these changes (code, script, tool, etc) ?
Please suggest any better option if someone has.
I explored Packer but found out that it needs source_ami to create AMI, hence not applicable to me as I am not creating AMI from source AMI, Please correct me if I am wrong.
Using ec2 Windows instance with Instance storage (let's say 32GB SSD) - where OS and its settings are stored? Like Program Files, User profiles. Are they all stored on Instance Storage? As far as I understood from other topics Instance storage is not-persistent and doesn't survive shutdowns/terminations. Does that mean I will lose everything under C: drive if I turn it off?
Can I use EBS storage as a default storage for OS (C drive)? Can I map multiple EBS storages to one Windows storage?
If above is true, then I will be charged for the capacity used by OS on EBS instance? It would be around 20GB I believe. Is that correct?
I am quite new in aws, and before paying for such instances or EBS I would like to know how this technical and billing model is working.
Thank you!
The Storage for the Root device is dependent on the AMI (EBS-Backed or Instance Store-Backed) used to launch the instance.
As far as I understood from other topics Instance storage is
not-persistent and doesn't survive shutdowns/terminations.
If the Root storage device is Instance Store, Stopping (shutdown) the instance is not possible. On termination, Both the storage and Instance does not survive. The Instance does not survive once terminated even if the AMI is EBS-Backed, but you can persist the Root Volume by setting the DeleteOnTermination flag set to False.
Does that mean I will lose everything under C: drive if I turn it off?
You cannot turn off (shutdown) an Instance Store-backed instance.
Can I use EBS storage as a default storage for OS (C drive)?
Yes, Choose an EBS backed Windows AMI.
Can I map multiple EBS storages to one Windows storage?
Yes, multiple EBS Volumes can be attached to one EC2 Windows Instance.
If above is true, then I will be charged for the capacity used by OS
on EBS instance?
You will be charged for the total size of the EBS volumes attached to the instance including the Root Device.
It would be around 20GB I believe. Is that correct?
The EBS Volume Size is adjustable. The upper Size limit is 16TiB.
Read Storage for Root Device and Ec2 Root Device Volume
Please spend more time on the AWS documentation, I don't think here is enough to cover all your question.
Only for specify EC2 instance come with attached SSD storage AKA instance storage. Bare in mind that, this instance storage doesn't come with Snapshot capabilities, so you must backup the file yourself. This is mean for people who need fastest disk access to process their data.
Only EBS allow you do multiple snapshot.
You can always create an AMI image for your instance after complete the deployment. AMI image is store inside EBS, so you will not lost the initial instance if you do this, so for new instance, you just trigger load it from AMI.
If you "Terminate" an instance, it will delete the virtual image. There is no way to recover it even with EBS, unless you make a snapshot. However, attached EBS storage will not be deleted.
EBS is calculate by Per GB and give you 1GB x 3 IOPS, with base 100 IOPS given. This is not enough if anyone want to carry out disk I/O intensive task.
I have an AMI image that was originally created from an instance where I had an EBS volume with 20 GBs of storage. So the original instance had a C:\ drive (runs Windows OS) of 20 GBs. When I launched another instance with the AMI image created from the original instance, I noticed that the C:\ drive is still only 20 GBs even though I requested the EBS volume on the new instance to be 200 GBs.
When I look at the EBS Volumes section of my EC2 console, I can see the new EBS storage of 200 GBs and it's attached to my new instance.
Why am I not seeing this new EBS storage as a hard drive in my server?
It may be that the space is available, you just need to extend into it. Run diskmgmt.msc in Windows and see if there is space showing on your existing 20GB volume
I followed the guide at http://technet.microsoft.com/en-us/library/cc771473.aspx on how to extend a hard drive and it did the trick