I have a question regarding GCP Persistent Disk (SSD). I created a disk, I created a VM to which I added the disk. Then I ssh'd into the VM, I saw the disk, I formatted it and mounted it to my VM. Everything worked just fine, I was able to create some files there and use it as a normal disk. However, when I deleted the VM and created another VM some time later, mounted the same disk, the data seemed to be lost, I didn't see it.
This contradicts the name "persistent" which means I might be doing something wrong or misunderstanding something. I would be greateful for any help.
It would help if you had posted your VM configuration, but if I had to guess you had the disk deletion rule in your VM set to delete the disk when you deleted your VM.
Edit your VM and make sure the "Deletion rule" setting is set to "Keep disk."
Related
If you use EC2 and launch instances, you can add EBS volumes. So a storage option. However, what I still don't understand exactly is why. Why is there or does EC2 even need a storage option like EBS or Instance Store? What does EC2 store anyway? And why it makes sense that there is EBS?
I know that EBS volume is persistent block storage and data is not lost after exit, unlike instance store. I just don't really understand what EBS is useful for. For which cases and applications is EBS used? Or does using EBS have more to do with creating snapshots that you can create to cache data and then save it to S3?
I've already read a lot and tried to make it understandable somehow, but somehow I can't get any further here. I would be really happy if someone could shed some light on this for me.
Thank you already!
Think of an Amazon EC2 instance as a normal computer. Inside, there is CPU, RAM and (perhaps) a hard disk.
When an EC2 instance has a hard disk, it is called Instance Storage and it behaves just like a normal hard disk in a computer. However, when you turn off the instance and stop paying for it, the EC2 instance can give that computer to somebody else. Rather than giving your data to somebody else, the disk is erased. So, anything you stored on Instance Store is gone! (In truth, instance store is also a virtualised disk, but this is close enough.)
In fact, in the early days of EC2, this was the only storage available. If you wanted to keep data after the instance was turned off, you first had to copy it to Amazon S3. People didn't like this, so they invented Amazon EBS.
If you want to keep your data so that it is still there when you turn on the instance in future, it needs to be stored on a network disk and that is what Amazon EBS provides. Think of it a bit like a USB drive that you can plug into one computer, then disconnect it and plug it into another computer. However, rather than being a physical device, it uses a storage service that keeps multiple copies of the data (in case a disk fails) and lets you modify the size of the disk. You are charged based on the amount of storage space assigned and how long the data is kept ("GB-Month").
Amazon EBS Snapshots are simply a backup of the disk. A snapshot contains all the data currently on the disk, allowing you to create a new disk anytime that will contain an exact copy of the disk as it was when the snapshot was created. This is great for backups, but is also very useful for creating multiple EC2 instances with the same disk content. An Amazon Machine Image (AMI) is actually just an Amazon EBS Snapshot plus a bit of metadata. When a new EC2 instance is launched, it uses an AMI to populate the boot disk rather than loading the operating system from scratch every time.
It is possible to create an AMI that populates an Instance Store disk. This way, you don't actually need to use an Amazon EBS volume. This is good for instances that don't need to permanently keep any data -- they could simply store information in a database or Amazon S3 instead of saving it on disk. Instance Store disks can be very fast since they don't send data across the network, so this is very useful in some situations.
In summary:
Instance Store is a normal disk in a computer (but it gets erased when the instance turns off so nobody else sees your data)
Amazon EBS volumes are network-attached storage that stays around until you delete it
I am working in Google cloud platform and I have a situation where I need to attach local SSD disk to a existing VM machine but I cannot find a way to do it, Basically this is the process I need to do.
1. Attach a new local SSD disk
2. Copy the existing data to new disk
4. Unmount the old disk
5. Mount new disk to old data path
Thanks.
As John said, you can't if you initially created your VM without local SSD. But you can do something like this
Go to the console, select your current VM
Go the the boot disk section and select your boot disk
Create an image of the boot disk
Go back to the current VM page
Click on "create similar" button, on the top
Select the boot disk image that you just created
Add additional disk, with type local scratch SSD
Now you have a similar VM, with the same boot disk image but with local SSD.
Then, you have to detach the existing persistent disk on the old VM and attach it to the new one, make your copy, and delete it.
This is the situation:
I hooked a disk to a VM. Reboot the VM (for any reasons). I have to remount the disk otherwise it is not available (unmounted) after the restart. So I remount the disk with the following command: sudo mount -o discard, defaults /dev/[DEVICE_ID] /mnt/disks/[MNT_DIR]
Does the fact that I have to remount the disk also mean that I have lost all the data inside?
Thanks in advance
The document that you shared with us says:
"If you detach this zonal persistent disk or create a snapshot from the boot disk for this instance, edit the /etc/fstab file and remove the entry for this zonal persistent disk"
Therefore, if you are not creating snapshots from the BOOTDISK you can reboot your instance without having any issue with your data.
However, if you are using snapshots or schedule snapshots of your SSD disk, I would recommend you to follow these best practises to create it :
https://cloud.google.com/compute/docs/disks/snapshot-best-practices
But also you can do persistent disk snapshots at any time without unmounting your disk. These recommendations are only in order to have a greater reliability and create the snapshots more quickly (this is also explained in the documentation: https://cloud.google.com/compute/docs/disks/snapshot-best-practices#prepare_for_consistency)
In the document that you linked, there is a description of how to add your mount point to /etc/fstab. Using the command line sudo mount -o ... mounts the disk temporarily, but the mount will be lost across reboots. Editing the /etc/fstab will cause the mount point to persist across reboots because that file is read during startup.
Thanks a lot for your answers. I'm sorry for my question that's not complete. I'm developer and I new in sysadmin.
As you can see here, I added "zonal persistent disk" (type permanent ssd disk) to my VM on Compute Engine (https://cloud.google.com/compute/docs/disks/add-persistent-disk).
Here says if I have a schedule snaphots it's no possible set automatically mount the disk to my VM after restart (that I may need for any reasons). So the question: how I sure that the restart, in addition to unmount the disc, it won't lose the data?
Having disk snapshots I would still be able to restore the data, but I would still understand what happens in this case. Meantime I read your suggestions on linux mount and I have understand that I will not lose the data on the disk with restart of the machine.
Thanks
I've restarted my VM instance and found that all of my files are deleted. My VM has a 10 GB SSD persistent boost disk and an additional 50GB SSD persistent disk. Do VM's only save files within the current session?
Thanks!
Google does not delete data upon restart. Be sure the disk containing your data is mounted. Perhaps you restarted after manually mounting & the disk is no longer mounted. You may have also been using the cloud shell instead of your VM which "After the instance is terminated, any modifications that you made to it outside your $HOME are lost." As per John Hanley's comment, you should also check the console log to see what happened on restart. Lastly, be sure you don't have any start-up scripts that may have deleted data.
According to the AWS documentation "any data stored in the RAM of the host computer or the instance store volumes of the host computer is gone." Does this mean that the web application I installed on my EC2 instance will be deleted if I stop running my instance?
I apologize for the naive question. I am new to this and worried I might make a mistake.
An Amazon EC2 instance is just like a normal computer. If you turn it off, anything in RAM is lost. Also, if you reboot either computer, the contents of RAM is lost (well, more like 'forgotten', but effectively the same).
Just like your home computer, if you reboot or stop/start an EC2 instance, it boots up again. Whatever software you have installed on the computer is still there. However, applications you were running will only start if you have configured a startup script to run the app again.
Typically, when software such as a web server is installed on a computer, it is configured to automatically start again when the computer is turned on/rebooted in future.
Elastic Block Store (EBS) disk volumes act just like a disk in a normal computer. If an EC2 instance is stopped and later started again, the contents of the disk is still there, unchanged.
Bottom line: It's just like a home computer. Don't panic.
Adding to John's answer, as long as you do not use an Instance Store volume, to store your web application, you are good to go.
The data in an instance store persists only during the lifetime of its
associated instance. If an instance reboots (intentionally or
unintentionally), data in the instance store persists. However, data
in the instance store is lost under the following circumstances:
The underlying disk drive fails
The instance stops
The instance terminates
If however you are using an EBS volume, these conditions do not apply and you are free to start and stop your instance any number of times you want.
Yes unless you have EBS volume attached to EC2. If you are using an EBS-backed instance, you can stop and restart that instance without affecting the data stored in the attached volume