How do I get files from GCP VM? - google-cloud-platform

I currently have a GCP VM where I tried to install something and there was a no memory left error on Ubuntu. I tried opening the SSH again and it is not working.
P.S there is no problem with firewall/connection.
I just want a way to download the files that I had stored in the VM. Is there a way to do this without accessing the Terminal?

If you are not able to login through serial console, then the only option left would be to retrieve the data from your OLD VM by creating a new VM.
You can follow the steps below to copy the data from the affected(OLD) VMs disk.
1 Create a snapshot from the boot disk of the OLD VM
2 Create a new VM. As a boot disk, you should use a Google public
image (important- do not use the snapshot you created).
3 Once that instance is created, try to SSH into it just to test if
you are able to access it. There should be no issue at this point with
this VM instance, as this is a new instance using a fresh operating
system.
4 In the newly created instance, click on the instance name (in the
Console), and then click ‘Edit’ at the top of the page to edit the
machine.
5 In the ‘Additional Disks’ section, click ‘Add item’.
6 In the ‘Name’ drop-down select ‘Create disk’. In the window that
opens add a name for the disk, and in the ‘Source snapshot’ drop-down
select the snapshot you created in Step 1. Now Click ‘Create’
7 Click ‘Save’ to save the instances new configuration.
8 Please SSH into the new instance, and run command $lsblk . You will
be able to see the new disk and partition added (It will most probably
be named sdb1 but you should check this and take note).
9) Please run the following command which will create a mount point at
/mnt/newdisk and then mounts the additional disk partition to that
mount point. Note- substitute /dev/sdb1 in the below command with the
name of the partition if it is different.
$ sudo mkdir /mnt/newdisk | sudo mount -o discard,defaults /dev/sdb1
/mnt/newdisk
The snapshots file system will now be mounted at /mnt/newdisk.
You should now be able to navigate the directories and retrieve any data.
I hope this helps you.

The description and results of your problem do not make sense. However, lets assume that your instance is out of memory and you cannot connect to the instance with SSH.
Reboot the instance and try again. Installing software might cause an out of memory issue. Rebooting should correct this.
Launch the instance with a larger machine type that has more memory. If this is a memory size problem, this will correct it.
Detach the instance's disk and attach to another instance that you can connect to. Mount the file system and copy off the files.
However, if instead your problem is out of disk space, this makes more sense.
Resize the instance disk. In the Google Cloud Console, go to Compute Engine -> Disks. Click on the disk for your instance. Click EDIT. Under Size enter a new larger disk size. Now launch your instance. For most operating systems (Ubuntu, Debian, etc.) the OS will automatically resize the root file system. I wrote an article that covers this in detail.

If you can't connect to the instance you always can take a snapshot of the disk and then create a copy to mount it in a new instance to recover the data from there.

Related

How to read/get files from Google Cloud Compute Engine Disk without connecting into it?

I accidentally messed up the permissions of the file system, which shows the message sudo: /usr/local/bin/sudo must be owned by uid 0 and have the setuid bit set when attempting to use sudo, such as read protected files, etc.
Response from this answer (https://askubuntu.com/a/471503) suggest to login as root to do so, however I didn't setup a root password before and this answer (https://stackoverflow.com/a/35017164/4343317) suggest me to use sudo passwd. Obviously I am stuck in an infinite loop from the two answers above.
How can I read/get the files from Google Cloud Compute Engine's disk without logging in into the VM (I have full control of the VM instance and the disk as well)? Is there another "higher" way to login as root (such as from gcloud tool or the Google Cloud interface) to access the VM disk externally?
Thanks.
It looks like the following recipe may be of value:
https://cloud.google.com/compute/docs/disks/detach-reattach-boot-disk
What this article says is that you can shutdown your VM, detach its boot disk and then attach it as a data disk to a second VM. In that second VM you will have the ability to make changes. However, if you don't know what changes you need to make to restore the system to sanity, then as #John Hanley says, we might want to use this mounting technique to copy of your work and then destroy your tainted VM and recreate a new one fresh and copy back in your work and start from there.

How to deativate UFW from outside the VM on Google Cloud Compute Instance

I accidentaly enabled the UFW on my Google Cloud Compute debian instance and unfortunately port 22 is blocked now. I've tried every way to go inside the VM but i can't...
I'm trying to access trhougth the serial port but it's asking me for user and password that was never set.
Does anyone have any idea what can I do?
If I could 'edit' the files on disk, it would be possible to change firewall rules and disable it. Already thought on mounting the VM disk on another instance but Google doesn't allow to "hot detach" it.
Also tried to create another VM from a snapshot of VM disk, but of course, the new instance came with the same problem.
Lots of important files inside and can't go in...
This is the classical example when you close yourself outside of the house with the key inside.
There are several ways to get back inside a virtual machine when the ssh is not currently working in Google Cloud Platform, from my point of view the easiest is to make use of the startup script.
You can use them to run a script as Root when your machine starts, in this way you can basically change the configuration without accessing the virtual machine.
Therefore you can:
simply launch some command in order to deactivate UFW and then access again the machine
if it is not enough and you rally need to access to fix the configuration, you can set up username and password for the root user making use of the startup script and then accessing through the serial console therefore without ssh (basically it is like you had your keyboard directly connected to the hardware). Note as soon you access the instance remove or at least change the password you have just used was visible to the people having access to the project. A safer way is to write down the password in on a private file on a bucket and download it on the instance with the startup script.
Note that you can redirect the output of command to a file and then upload the file to a bucket if you need to debug the script, read the content of a file, understanding what is going on, etc.
The easiest way is to create a startup-script that disables ufw. which gets executed whenever the instance is booted:
Go into your Google Cloud Console. Go to your VM instance and click Edit button.
Scroll down to your "Custom metadata", and add a "startup-script" as key and the following script as value:
#! /bin/bash
/usr/sbin/ufw disable
Click save and reboot your instance.
Delete that startup-script and click Save, so that it won't get executed in future boots.
You can try google serial ports. From then you can enable the ssh
https://cloud.google.com/compute/docs/instances/interacting-with-serial-console
ufw allow ssh

VirtualBox Snapshot - How to stop them to grow and take more and more space?

I'm using Genymotion with Oracle VirtualBox, however i do have 250 GB SSD and i'm facing an issue with ( Snapshots ) I googled & searched here, I couldn't find any possible way to disable Auto snap-shots, as i don't need it.
Thanks
When you deploy a new Genymotion virtual device, a snapshot called factory-backup is automatically created so that paid users can reset the device to the factory state via the Launcher GUI. If you don't want this snapshot, then just use VirtualBox Manager directly to delete it once the device is deployed. From the Manager interface, with the VM in question selected, click on Snapshots in the upper right corner, then select factory-backup and hit the delete button. As far as I can tell, there is no provision for disabling this initial snapshot creation via the Genymotion Launcher.
In my experiences, Genymotion does not create further snapshots during the lifetime of a virtual device, only upon initial deployment. But if you are experiencing this, then I recommend again using the VirtualBox Manager to set the Snapshot Folder to a non-existent directory once the device is deployed. This can be done via Settings->General->Advanced for each device you wish to disable snapshotting.

move a virtual machine from one vCenter to another vCenter

I have the following problem:
There two separate vCenters (ESXi). They cannot see each other or communicate in any way.
I can create a Clone of a VM in vCenter1 but then I want to move that Clone in vCenter2.
Is there a way that I can copy the Cloned VM (files) on an external HDD and move them in the other vCenter?
I've figure it out the solution to my problem:
Step 1: from within the vSphere client, while connected to vCenter1, select the VM and then from "File" menu select "Export"->"Export OVF Template" (Note: make sure the VM is Powered Off otherwise this feature is not available - it will be gray). This action will allow you to save on your machine/laptop the VM (as an .vmdk, .ovf and a .mf file).
Step 2: Connect to the vCenter2 with your vSphere client and from "File" menu select "Deploy OVF Template..." and then select the location where the VM was saved in the previous step.
That was all!
Thanks!
Yes, you can do this.
Copy all of the cloned VM's files from its directory, and place it on its destination datastore.
In the VI client connected to the destination vCenter, go to the Inventory->Datastores view.
Open the datastore browser for the datastore where you placed the VM's files.
Find the .vmx file that you copied over and right-click it.
Choose 'Register Virtual Machine', and follow whatever prompts ensue. (Depending on your version of vCenter, this may be 'Add to Inventory' or some other variant)
The VM registration process should finish with the cloned VM usable in the new vCenter!
Good luck!
For moving a virtual machine you need not clone the VM, just copy the VM files (after powering the VM off) to external HDD and register the same on destination host.
A much simpler way to do this is to use vCenter Converter Standalone Client and do a P2V but in this case a V2V. It is much faster than copying the entire VM files onto some storage somewhere and copy it onto your new vCenter. It takes a long time to copy or exporting it to an OVF template and then import it. You can set your vCenter Converter Standalone Client to V2V in one step and synchronize and then have it power up the VM on the new Vcenter and shut off on the old vCenter. Simple.
For me using this method I was able to move a VM from one vCenter to another vCenter in about 30 minutes as compared to copying or exporting which took over 2hrs. Your results may vary.
This process below, from another responder, would work even better if you can present that datastore to ESXi servers on the vCenter and then follow step 2. Eliminating having to copy all the VMs then follow rest of the process.
Copy all of the cloned VM's files from its directory, and place it on its destination datastore.
In the VI client connected to the destination vCenter, go to the Inventory->Datastores view.
Open the datastore browser for the datastore where you placed the VM's files.
Find the .vmx file that you copied over and right-click it.
Choose 'Register Virtual Machine', and follow whatever prompts ensue. (Depending on your version of vCenter, this may be 'Add to Inventory' or some other variant)
Copying the VM files onto an external HDD and then bringing it in to the destination will take a lot longer and requires multiple steps. Using vCenter Converter Standalone Client will do everything for you and is much faster. No external HDD required. Not sure where you got the cloning part from. vCenter Converter Standalone Client is simply copying the VM files by importing and exporting from source to destination, shutdown the source VM, then register the VM at destination and power on. All in one step. Takes about 1 min to set that up vCenter Converter Standalone Client.
You don't have to export your VMs at all. You can move the VM and clone to a TAXI host in vCenter 1. Then add the host to vCenter 2, and vMotion away whatever VMs to other hosts previously managed by vCenter 2. When done, you can add the TAXI host back to vCenter 1.
If you'd like to do this using the command line, you can do this if you have ESXi 6.0 (or possibly even ESXi 5.5) running, by using govc, which is a very helpful utility for interacting with both your vCenter and its associated resources.
Depending on your setup, you can
# setup your credentials
export GOVC_USERNAME=YOUR_USERNAME GOVC_PASSWORD=YOUR_PASSWORD
govc export.ovf -u your-vcsa-url.example.com -vm VM_NAME -dc VMS_DATACENTER export-folder
Then, you'll have your VM VM_NAME exported in the folder export-folder. From there, you can then
govc import.ovf -u your-other-vcsa-url.example.com -vm NEW_VM_NAME -dc NEW_DATACENTER export-folder/VM_NAME.ovf
That'll import it into your other vCenter. You might have to specify -ds NEW_DATASTORE too, if you have more than one datastore available, but govc will tell you so if you need to.
The commands above require that govc is installed, which you should, because it's far better than ovftool either way.

VirtualBox snapshots

Heres what happened.
I had a snapshot on which I was working from within a linux VM. A friend requested a clean VM as a clone of mine. So I closed / shutdown my running VM, made a copy of the Disk1.vdi along with the snapshots ({uuid}.vdi). Then I restarted the VM and did merged snapshots, deleted my home directory and made a tar+bz2 for my friend.
Then after I restored my backups, I am not able to mount my snapshot. The VM seems to boot from my version before snapshot. I cant seem to find a way to mount back my snapshot.
Any idea how to make VirtualBox see the snapshot and mount it?
I am not an expert but have coincidentally done some investigation into just this topic. You indicated that you backed up your disks (VDI and snapshots) before making changes but you did not back up the VM itself (the XML file). So you have created an incompatibility by restoring VDI and snapshots to the changed VM (that still thinks there is only a merged disk with no snapshots). Without a backup of the original VM file itself you may be out of luck. (Please see Cloning a VM With Multiple Snapshots for supporting evidence.)
You can get back to work the snapshot, tricky but may try this (with no Virtual Machine running):
Open VirtualBOX GUI
Go to manage disks
Detach your main VDI from your Virtual machine.
Set it as INMUTABLE.
Reattach the main VDI from your Virtual machine
Exit from ALL virtualbox processes
Get the uuid of your snapshot VDI with VBoxManage showhdinfo, the one you want to use not the one created when making immutable the main VDI
Now edit the VBOX file with text editor and look for the path of the snapshot that was created so you know where to go to delete that small file, do not close the text editor
Delete that small snapshot vdi file
Now, on the text editor, replace the uuid of the snapshot and the path to the snapshot vdi to point to your snapshot vdi file
Save the VBOX file and close the text editor.
For future times: Remember to also backup the VBOX files too.
The trick is based on making VirtualBox create a fake snapshot file (a file that you will manually delete) and replace the references added to the vbox file with your snapshot, but take take to also replace the uuid of the snapshot file with the correct one, for that you can get it with showhdinfo.
Be warned, the snapshot uuid on the VBOX file appears on to sections, the register (near the beginning) and the attached section (near the end), you must replace both, you can use search & replace the newly uuid with yours.
Hope it will work for you; i never do snapshots of a virtualbox, I prefer the immutable way (but that is only for just one level).