My GCP SQL instance is stuck on restart. Is there way to force stop or restart or to access the database backups so I can create a new one from the backup?
After 1 hour the instance restart failed due to an unknown error.
The original instance was db-n1-standard-1. I created a new instance type db-n1-standard-2 (more memory and 2CPU) and I managed to restore a backup to the new instance.
Any other operations to restore backup or clone in the db-n1-standard-1 failed.
Related
I currently have one AWS RDS Mariadb instance stuck at rebooting. In the current state, I cannot either modify or stop the instance.
I contacted AWS requesting to have the instance stopped and they responded We cannot STOP the instance on behalf of our customers. It is available to do actions from your end.
I understand that it might have hit this mariadb bug
I'd like to try update some innodb_* parameters to see if the instance can be started.
But there's nothing I can do at the moment because the instance is stuck at rebooting.
I don't think I am the only one who had this kind of issue.
AWS support is not helping at all. The only solution they suggest is delete the instance and restore from backup. Restoring from backup would be my last option. Luckily, this is not our production database. If restoring from backup is the only option if you hit a mariadb bug and the instance end up stuck at rebooting, I'd reconsider whether I should host mariadb on EC2 instances.
I accidentally deleted my ai notebook vm and I hadn't downloaded the notebooks connected to it. I still have the url. Does anybody know if there's a way for me to recover my work?
According to the documentation, there is a life cycle for the instances. Verify the state of your AI Notebook VM to make sure that it is deleted or just turned off.
Unfortunately, if an AI Notebook instance is deleted and there is no snapshot configured, there is no way to restore that instance neither recover the notebooks stored there. There are three ways to prevent this from happening in the future:
Create snapshots to periodically or schedule back up data from your zonal persistent disks (snapshot can be located in multiple zones) or regional persistent disks (You must indicate the region where the disk is located ).
Edit VM instance, go to the deletion protection checkbox to enable it as this option is disabled by default. This setup will avoid that your Notebook instance was deleted by accident.
In the VM instance, go to boot disk, in the drop down list under “When deleting instance” select “Keep Disk” (or you can use gcloud command to disable set-disk-auto-delete)
I have a scheduled daily snapshot in GCP for one of my instances. I have several snapshots now. The first one is the full snapshot and the rest of the snapshots only contain changed data.
I want to be able to restore and boot the instance but it fails to boot. Checking the serial console I see reference to a blue screen and then it reboots and shows the same errors again, repeating the reboot cycle.
I have followed the guide in GCP on how to restore an instance from a snapshot by creating a new instance, selecting the snapshot tab and then selecting my snapshot. After saving the instance and trying to boot it I get the blue screen message.
Also, if I create a new instance and use a Windows 2008 R2 Datacenter image the system obviously boots fine but if I try to attach the snapshot disk as a secondary disk (non-boot) then I get the error: Editing VM instance failed. Error: Supplied fingerprint does not match current metadata fingerprint. I'm not sure if this is related to my issue with unable to boot the OS from my snapshot.
I did find a workaround:
1) create an image of running instance (my instance is Win'2008 R2 Datacenter)
2) enable scheduled snapshots of this new instance (with VSS)
3) wait for a scheduled snapshot to get created (hourly so must wait 1 hr)
4) create new instance from the scheduled snapshot
After all this work the instance boots just fine with all my data. Obviously not a very good workaround as now I have two instances with the same data. So I have to schedule the production system for maintenance so that I can bring it down and use the new instance so that future scheduled snapshots work if I try to restore it again. A major paint in the butt.
Anyone have any ideas as to why none of my instances boot from scheduled snapshots without my workaround? Manual snapshots work fine. And new instances also work fine with the same snapshot schedule.
I had this exact same problem. I tried multiple scheduled snapshots with the same result UNTIL I made a change in the VM Instance when attaching the restored snapshot. Maybe it's just Windows but if you named the disk something different then it seems to fail.
My original disk was called disk-1 for example. When I restored the snapshot I did it to disk-1-a and attached it to my instance. It failed the same way yours did. When I attach it and under "Device name" for the boot disk, select Custom and entered my original disk name of disk-1, it booted and RDP worked.
I have created a free trial instance in GCP. Now I am seeing 2 instance in VM instance dashboard.
One is the instance that I am using and other is ce-replicator-goo16-2e8##f6b. I tried to delete this instance but it automatically comes up again. I am not sure why this instance is for. Will I be charged for this instance as well.
IF anyone have idea on this, Please help.
Thanks
J
It happens when you are using CloudEndure migration service to bring your instance into GCP. In such cases, Service Account having Owner permission recreating the instance "ce-replicator-goo16-2e8##f6b". Deleting the "service account" used for migration would allow deleting those temporary instance.
In case any VM instance gets deleted accidentally then is there any way to recover it in Dataproc cluster. In case there is no way to recover a deleted VM instance then can we create a new VM instance and connect to an existing DataProc cluster?
Please suggest.
There is no way to recover data on deleted VMs. Also manually removing workers carries a possibility of bricking the cluster. If your goal is to shrink the cluster we recommend option #1 below.
There's a few ways to add back a deleted VM:
Resize the cluster down and up using gcloud dataproc clusters update --num-workers, you'll want to ensure the deleted VM is in the set of VMs to be removed by downsize operation (this is not always possible)
From Developers Console go to Compute > Instances select any regular worker VM and clone it such that name matches the deleted VM.
Both of these approaches carry identical billing implications.