It is my understanding that when you create an EBS Snapshot, any data written to the volume up to the time that the snapshot was started, will be included. I.e, even if the snapshot is in the "pending" state, it is still safe to write to the EBS volume.
Is this still considered the case for taking EBS snapshots of root volumes? In the documentation, it is stated that when you create a snapshot for Amazon EBS volumes that serve as root devices, you should stop the instance before taking the snapshot.
Does this imply that we can stop the instance, begin the snapshot, and then restart the instance before the snapshot has completed? The reason I'm asking is because our snapshots are taking >15 minutes, which is timing out our Snapshot Management Lambda before it can restart the instance.
It should be safe, see https://serverfault.com/questions/548731/can-i-re-attach-my-ebs-while-a-snapshot-is-pending
You should stop the instance when creating a root volume snapshot because you can not unmount root volume beforehand, hence you would risk your snapshot would be corrupted.
Related
I have a few AMIs stored with snapshot associated with their storage definitions. However, the volumes that are associated with those snapshots have been deleted to save costs.
Can the AMIs still be used?
Yes, because snapshot is a backup of your volume. The original volume does not have to exist for the snapshot to be useful. You can always create a volume from a snapshot.
If you want to double check this, you can spin up basic t2.micro instance, take it snapshot, delete the instance and any associated volumes (root volume will be automatically deleted). Then you can launch new instance from the snapshot created.
Scenario
I currently have an EC2 instance with a root EBS volume attached to it of 30gb and i have some files stored in that EBS
If i delete the EC2 instance and have delete on termination false then EBS persists.
Desired outcome
I want to provision a new EC2 (provisioned by auto scaling group) instance such that it uses the old EBS volume as its root volume which was detached as a result of me terminating the old instance
Note
I want to have the liberty of choosing OS of newly provisioned EC2 so creating an AMI does not work
You cannot directly launch a new Amazon EC2 instance with an existing Amazon EBS volume. Instead, you would need to:
Launch a new Amazon EC2 instance with a new root volume
Stop the instance
Detach the root volume
Attach the 'old' EBS volume
Start the instance
Storing data in root EBS volume might be a bad idea to start with.
Consider one of the following:
Mount another EBS volume to the instance to store required data only.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-using-volumes.html
best performance, highest cost/effort, but your application doesn't
change a bit.
Create EFS and mount it to your instances. https://docs.aws.amazon.com/efs/latest/ug/mounting-fs.html -
reasonable effort, minimal if any changes to the application.
Store data in S3. Ideal from price standpoint; requires changes to the application.
I understand that EBS volumes that are restored from snapshots need to be initialized.
Let's say I create an AMI from an EBS snapshot(s).
When I launch a new instance using that AMI, will I need to initialize each volume before I get consistent latency?
When you create an instance from AMI which has EBS based volumes as storage, your instance creates new volume from this volume and continues with launch.
When you use snapshots as source to launch your instance, AWS need to pull these snapshots and write it to volume and then continue with the launch. In this case, pull storage blocks from S3 and write to volume will be the latency some users want to avoid. This is the case where pre-warming is suggested by AWS.
Is AWS root storage ephemeral? The documents say they are ephemeral. But, I still can find my files after stopping the instance for some time and reboot it, I can still view my files
When you created your instance, I assume you chose the EBS backed instance. This means that the root device volume is actually an EBS volume, so when the instance restarts, you still have all of your data. If you were to choose the instance-store backed instance, then you would lose your data when restarting.
See http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/RootDeviceStorage.html for more information.
What do you mean by root storage?
Instance Stores are ephemeral. EBS volumes are not.
When launching an instance, if you've selected Instance Store as the root volume, a restart will still retain the data. But a stop and start will delete all data.
If you've selected an EBS volume, the data will be retained.
I have two amazon machine instances running.Both of them are m3.xlarge instances. One of them has the right software and configuration that I want to use.I want to create a snapshot of the EBS volume for that machine and use that as the EBS volue to boot the second machine from. Can I do that and expect it to work without shutting down the first machine.
It is well described in the AWS documentation...
"You can take a snapshot of an attached volume that is in use. However, snapshots only capture data that has been written to your Amazon EBS volume at the time the snapshot command is issued. This might exclude any data that has been cached by any applications or the operating system. If you can pause any file writes to the volume long enough to take a snapshot, your snapshot should be complete. However, if you can't pause all file writes to the volume, you should unmount the volume from within the instance, issue the snapshot command, and then remount the volume to ensure a consistent and complete snapshot.
I have amazon as well, with 3 different clusters. With one of my clusters after setting up 25 of them I realized there was a small issue in the configuration and had live traffic going to them so I couldn't' shut down.
You can snapshot the first machines volume while it's still running, I had to do this myself. It took a little while, but ultimately it worked out. Please note that amazon cannot guarantee the consistency of the disk when doing this.
I did a snapshot of the entire thing, fixed what needed to be fixed, and spooled up 25 new servers and terminated the other 25 ( easier than modifying volumes, etc ).. But you can create a new volume with the new snapshot, and attach it to an instance and do what needs to be done to get it to boot off that volume without much of a headache.
Being that I went the easy route of spooling up new instances after my snapshot was complete, I can't walk you through on how to get a running instance to boot off a new volume.