I've two VMs created at the Compute Engine session with hourly snapshot as backup copies. I never created any storage bucket, I wonder where do those snapshots stored and how does it count for the storage space charges?
And, is there a way I can backup the VMs to on-prem storage? e.g. can I use any API command to download the VM snapshot to my local storage as a backup of backup just in case Google Cloud screwed up.
When you create a snapshot, you can specify a storage location. The location of a snapshot affects its availability and can incur networking costs when creating the snapshot or restoring it to a new disk. You will find the pricing for snapshot storage here.
This article describes how to create an image from your VM's root disk and export it to Google Cloud Storage or you can directly download it to your remote computer.
Related
gcloud compute instances attach-disk wants a disk name, but it doesn't show up on my Disks page. It seems silly to create and pay for another disk when this one has much more storage than I plan to use.
Notice that the Cloud Shell is intended for interactive usage and that in general the disk is intended to be recycled, as you can't manage it and it will be deleted after 120 days of inactivity. You'll need to consider a different solution, such as Cloud Storage if you wish the data to persist in time. So you'd need to store your data in Cloud Storage and then create a new disk to store the information, as the Cloud Shell is a tool meant for rapid testing and prototyping and not as a development machine for persistent storage.
As per the GCP article enter here, you can attach-detach disk to the VM instance from gcloud shell.
To detach a disk from a instance:
gcloud compute instances detach-disk [INSTANCE_NAME] --disk=[DISK_NAME]
To attach a disk to another instance:
gcloud compute instances attach-disk [INSTANCE_NAME] --disk=[DISK_NAME] --boot
Google Cloud Disk Snapshots are default located in which regions? Is there region same as Disk region?
Whenever you take snapshot in Google cloud, the snapshot data is incremental and the snapshot data is kept in Google cloud object storage and the details of which object storage, where and how is not at all revealed to users.
Because, Google needs to meet some SLAs for those snapshots they would want to disclose details of the snapshot , as to meet SLA s they would be changing and managing snapshots internally that user would be oblivious to
And that's true with other clouds too!!
I have pretty uncommon task - backup all data and configurations from AWS account to local storage (and then potentially restore them).
OR at least EC2 snapshots and then restore them
i m running ec2 instance in 1 region i want to create snapshots of ec2 instances in other region directly without coping and cross region replication in s3, is this possible? if possible then how?
Amazon EBS Snapshots are created in the same region as the original EBS Volume. They can then be used to create a new Volume within the same Region.
If you wish to use an Amazon EBS Snapshot in a different region, the snapshot must first be copied to the other Region. This can done via the Amazon EC2 management console, the AWS Command-Line Interface (CLI) aws ec2 copy-snapshot command, or an AWS API call.
Please note that snapshots are incremental backups. The first snapshot isn't really a full backup. Rather, every snapshot simply copies any blocks that have been modified since any previous snapshot. Blocks are retained while snapshots still require the blocks. This means that blocks made during the initial snapshot could actually be deleted if they are not required by any active snapshots. This is why I say they are not the same as a full backup, which traditionally never has content deleted.
However, when a snapshot is copied to a new region it is copied in full, rather than incrementally.
If you do not with to copy an EBS snapshot between regions, you would need to find a different way to transfer the disk volume (eg filesystem-level synchronisation).
In fact, there should typically be no need to transfer a disk volume -- rather, your systems should be capable of configuring a new server based upon a startup configuration script and data should be stored in a separate database so that it is accessible to multiple instances. It is a very rare case that requires a complete copy of a disk volume.
Currently I am taking manual backup of our EC2 instance by zipping the data and downloading it locally as well as on DropBox.
But I am wondering, can I have an option where I just take a complete copy of the whole system automatically daily so if something goes wrong/crashes, I can replace it with previous copy immediately rather than spending hours installing and configuring things ?
I can see there is an option of take "Image" but can I automated them to have just 1 latest image and replace the system with single click ?
You can create a single Image of your instance as Backup of your instance Configuration.
And
To keep back up of your data you can use snapshots of your volumes.
snapshots store data in incremental format whenever you make any changes.
When ever needed you can just attach the volume from the snapshot to your Instance.
It is not a good idea to do "external backup" for EC2 instance snapshot, before you read AWS pricing details.
First, AWS is charging every GB of data your transfer OUTside AWS cloud. Check out this pricing. Generally speaking, after the 1st GB, the rest will be charge at least $0.09/GB, against S3-standard pricing ~ $0.023/GB.
Second, the snapshot created is actually charges as S3 pricing(Check :
Copying an Amazon EBS Snapshot), not EBS pricing. After offset the transfer cost, perhaps you should consider create multiple snapshot than keep doing the data transfer out backup.
HOWEVER, if you happens to use an instance that use ephemeral storage, snapshot will not help. You need to copy the data out from ephemeral storage yourself. Then it is your choice to store under S3 or other place.
Third. If you worry the AWS region going down, check the multiple AZ option. Or checkout alternate AWS region option.
Fourth. When storing backup data in S3, you can always store them under Infrequent-Access, which save you some bucks, and you don't need to face an insane Glacier bills during emergency restore(Avoid Glacier, unless you are pretty sure about your own requirement).
Fifth, after done your plan of doing everything inside AWS, you can write bash script (AWS CLI) or use boto3, etc API to do the automatic backup.
Lastly , here is way of AWS create and maintain snapshot. Though each snapshot are deem "incremental", when u delete old snap shot :
the snapshot deletion process is designed so that you need to retain
only the most recent snapshot in order to restore the volume.
You can always "test" restore by create another EC2 instance that load the backup snapshot. Or you can mount the snapshot volume from another EC2 instance to check the contents.