I am asking about the GCP Preemptive Instances. I have read that when an instance is terminated on a preemptive instance, an ACPI Soft Power Off occurs.
I am wondering if the hypervisor pauses the instance so that I can continue on my tasks. Or, the VM is shut off and not paused.
I have used preemptive instances in the past, but I cannot seem to remember if the VM was shut down or paused.
When your instance is turned off (or terminated) as stated in the documentation you can still access your data which are stored on a persistent disk.
Preemption process is described in the documentation and it states:
Preempted instances still appear in your project, but you are not charged for the instance hours while it remains in a TERMINATED state. You can access and recover data from any persistent disks that are attached to the instance, but those disks still incur storage charges until you delete them. As with normal instances, persistent disks that are marked for auto-delete are deleted when you delete the preemptible instance.
You can start your instance later and access your data - and if a proper resources are not available then you can attach the disk to other VM's and still access your data.
In short you will retain the disk data and when new vm comes up then it will automatically attach to it
Related
Obviously, hibernation and stop are two different actions that I can select.
What's the difference?
Benefit of Hibernating over Stopping
The memory state is preserved
Since the memory state is perserved and loaded again when the instance start, this reduce the boot time of the instance.
The long running process can continue without interuption
A great benefit if you have some services that take a great amount of time to fully initialized
Under the hood
The whole hibernation process in visual:
When the instance is in Stopping state, the instance memory is persisted in the instance's EBS root volume, and is loaded again when the instance start.
Reference
AWS Instance Hibernate Overview
From the docs
When you hibernate an instance, Amazon EC2 signals the operating
system to perform hibernation (suspend-to-disk). Hibernation saves the
contents from the instance memory (RAM) to your Amazon Elastic Block
Store (Amazon EBS) root volume. Amazon EC2 persists the instance's EBS
root volume and any attached EBS data volumes. When you start your
instance:
The EBS root volume is restored to its previous state
The RAM contents are reloaded
The processes that were previously running on the instance are resumed
Previously attached data volumes are reattached and the instance retains its instance ID
Read more
TL;DR
When you stop your instance, the data stored in memory (RAM) is lost.
When you stop-hibernate an instance, AWS signals the OS to perform hibernation (suspend-to-disk), which saves the contents from the instance memory (RAM) to the Amazon EBS root volume.
From the charging perspective, AWS does not charge usage or data transfer fees for your instance after you stop it, but storage for any Amazon EBS volumes is still charged.
A practical example
Suppose you want to build a caching layer (e.g. on top of your DB) in an EC2 instance. For such a case, the stop-hibernate feature would be instrumental in persisting storage. It would prevent you from having to manually create scripts to save the RAM data before shutting down the server.
I was created an instance in Google Cloud Platform. I use this instance 2 to 4 days in a week. So, I want to close my instance for rest of the days to save my billing charges. So, what is best option for me. Stop the instance or Suspend the instance? What is the difference between them?
Please have a look at the documentation Suspending and resuming an instance:
Suspending an instance differs from stopping an instance in the following ways:
Suspended instances preserve the guest OS memory, device state, and application state.
Google charges for the storage necessary to save instance memory.
You can only suspend an instance for up to 60 days. After 60 days, the instance is automatically moved to the TERMINATED state.
and at the article Stopping and starting an instance:
Instances that are in a TERMINATED state are not charged for
per-second usage and do not count toward your regional CPU quota, so
you can choose to stop instances that you are not using, saving you
from being charged for instances that aren't active. After you are
ready, you can come back and start the same instances again, with the
same instance properties, metadata, and resources.
Your instances are not charged for per-second usage while in the
TERMINATED state but any resources attached to the virtual machine,
such as static IPs and persistent disks, are charged until they are
deleted.
As a result, it's better to stop your VM instance, to avoid extra changes for keeping instance's memory.
In addition, please have a look at the article Save money by stopping and starting Compute Engine instances on schedule.
Ive read: https://cloud.google.com/compute/docs/instances/stopping-or-deleting-an-instance. In it they explain that:
You can stop an instance temporarily so you can come back to it at a
later time. A stopped instance does not incur charges, but all of the
resources that are attached to the instance will still be charged.
Alternatively, if you are done using an instance, delete the instance
and its resources to stop incurring charges.
However, i don't know how things are allocated in google cloud so im not sure what this means.
Say I stop my instance, I guess i am still using their storage, but im not using any CPU right? Does that mean that i will only pay for storage but not for the CPU and GPU hours?
Also how does google stop my instance? I know that if you suspend an instance in something like "Virtualbox" you can start it and start from when you left off. Is that what google does? Can I stop the instance halfway through running something and have it continue where it left off when i start the instance again?
When you stop your instance, you will not pay for CPU or GPU while the instance is stopped as the instance is not using them but you will be charged for resources attached as stated in the link you posted:
Your instances are not charged for per-second usage charges in TERMINATED state but any resources attached to the virtual machine will be charged until they are deleted, such as static IPs and persistent disks.
Google stops your instance by shutting it down, so you will lose data that is not in a persistent disk already.
From their docs:
When you shut down or delete an instance, Compute Engine sends the ACPI Power Off signal to the instance and waits a short period of time for your instance to shut down cleanly. If your instance is still running after this grace period, Compute Engine forcefully terminates it even if your shutdown script is still running.
There is a gcloud command that is in alpha that can suspend your VM: gcloud alpha compute instances suspend, you can read more in the docs here. It will only work on instances not using GPU or CSEK or preemptible VMs.
Compute Engine documentation says:
A stopped instance does not incur charges, but all of the resources
that are attached to the instance continue to incur charges. For
example, you are charged for persistent disks and external IP
addresses according to the price sheet, even if an instance is
stopped. To stop being charged for attached resources, you can
reconfigure a stopped instance to not use those resources, and then
delete the resources.
Simple question for GCE users: are persistent boot disks safe to be used or data loss could occur?
I've seen that I can attach additional persistent disks, but what about the standard boot disks (that should be persistent as well) ?
What happens during maintenance, equipment failures and so on ? Are these boot disks stored on hardware with built-in redundancy (raid and so on) ?
In other words, are a compute instance with persistent boot-disk similiar to a non-cloud VM stored on local RAID (from data-loss point of view) ?
Usually cloud instances are volatile, a crash, shutdown, maintenance and so on, will destroy all data stored.
Obvisouly, i'll have backups.
GCE Persistent Disks are designed to be durable and highly-available:
Persistent disks are durable network storage devices that your instances can access like physical disks in a desktop or a server. The data on each persistent disk is distributed across several physical disks. Compute Engine manages the physical disks and the data distribution to ensure redundancy and optimize performance for you.
(emphasis my own, source: Google documentation)
You have a choice of zonal or regional (currently in public beta) persistent disks, on an HDD or SSD-based platform. For boot disks, only zonal disks are supported as of the time of this writing.
As the name suggests, zonal disks are only guaranteed to persist their data within a single zone; outage or failure of that zone may render the data unavailable. Writes to regional disks are replicated to two zones in a region to safeguard against the outage of any one zone. The Google Compute Engine console, "Disks" section will show you that boot disks for your instances are zonal persistent disks.
Irrespective of the durability, it is obviously wise to keep your own backups of your persistent disks in another form of storage to safeguard other mechanisms for data loss, such as corruption in your application or user error by an operator. Snapshots of persistent disks are replicated to other regions; however, be aware of their lifecycle in the event the parent disk is deleted.
In addition to reviewing the comprehensive page linked above, I recommend reviewing the relevant SLA documentation to ascertain the precise guarantees and service levels offered to you.
Usually cloud instances are volatile, a crash, shutdown, maintenance and so on, will destroy all data stored.
The cloud model does indeed prefer instances which are stateless and can be replaced at will. This offers many scalability and robustness advantages, which can be achieved using managed instance groups, for example. However, you can use VMs for persistent storage if desired.
normally the data boot disk should be ok with restart and other maintenance operation. But it will be deleted with the compute by default.
If you use managed-instance-group, preemptible compute... and you want persistent data, you should use another storage system. If you juste use compute as is, it should be safe enough with backup.
I still think an additional persistent disk or another storage system is a better way to do things. But it's only my opinion.
Last night there was an error on my EB instance. The instance was removed and a new one was added. Because of that I lost data from the non-persistent instance storage. I don't have a backup / snapshot. A big beginner's mistake.
My question: Is there any chance to recover the instance's data from 12 hours ago? Maybe with the help from the AWS staff?
When an instance is stopped or terminated, the ephemeral volumes are gone. Terminating an instance releases the hardware for use by another customer. Stopping an instance does the same thing -- that's part of why you don't pay for stopped instances. The same instance will actually come up on physically different hardware if stopped and started.
Aside from the documentation...
The data in an instance store persists only during the lifetime of its associated instance. If an instance reboots (intentionally or unintentionally), data in the instance store persists. However, data in the instance store is lost under the following circumstances:
The underlying disk drive fails
The instance stops
The instance terminates
Therefore, do not rely on instance store for valuable, long-term data.
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html
...there are also numerous support forum posts on this topic, with responses by AWS personnel, indicating that the answer is "no."
Here's one example:
Once an instance that has Ephemeral (or Instance Store) volumes has been Stopped or Terminated, we are unable to recover the data that was on that volume. When you Stop or Terminate such an instance, those volumes are securely wiped and this is to ensure the security and confidentiality of your data that was on that volume.
This is as per: http://aws.amazon.com/instance-help/
And: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html
https://forums.aws.amazon.com/thread.jspa?messageID=501815񺠷
And another:
Ephemeral (instance-store data) is the local host's hard drive and when an instance is migrated (moved) to new hardware (from a stop/start) the ephemeral data is scrubbed as part of the process as the instance will have new ephemeral storage as part of the new host.
All that said, there is not any way to get the data back from the ephemeral location.
https://forums.aws.amazon.com/thread.jspa?messageID=396680񠶈