I created a VM Instance (n1-standard-8) for a project. I was using AI Platform > Workbench (Jupyter Notebook). I was able to read the data from Cloud storage and process it. After 2 months, I tried to start the notebook and clicked on 'OPEN JUPYTERLAB'. It just spins up saying "Setting up proxy to Jupyterlab".
Environment: Kaggle Python
Machine Type: n1-standard-8 (8 vCPUs, 30
GB RAM)
What is the possible issue?
PS: New to Google Cloud
One possible solution is to create a new VPC without adding the DNS rules for the various notebooks endpoints.
Then, use the configured network with a new notebook instance clicking the “OPEN JUPYTERLAB” URL.
You can see more information here.
Another possible thing that could be happening if you check your logs is an error that shows this “ - Required 'compute.instances.get' permission for project'”. This happens because you are using the non-default service account that you specified during the notebook creation. So the solution to this is to use the default service account.
Related
I created a few GCP VM instances yesterday all using the same configuration but running different tasks.
I could SSH into those instances via the GCP console and they were all working fine.
Today I want to check if the tasks are done, but I cannot SSH into any of those instances via the browser anymore...The error message reads:
Connection via Cloud Identity-Aware Proxy Failed
Code: 4010
Reason: destination read failed
You may be able to connect without using the Cloud Identity-Aware Proxy.
So I retried with Cloud Identity-Award Proxy disabled. But then it reads:
Connection Failed
An error occurred while communicating with the SSH server. Check the server and the network configuration.
Running
gcloud compute instances list
displayed all my instances and the status is RUNNING.
But when I ran
gcloud compute instances get-serial-port-output [instance-name]
using the [instance-name] returned from the above command. (This is to check if the boot disk of the instance has run out of free space.)
It returned
(gcloud.compute.instances.get-serial-port-output) Could not fetch serial port output: The resource '...' was not found
Some extra info:
I'm accessing the VM instance from the same internet (my home internet) and everything else is the same
I'm the owner of the project
My account is using a GCP free trial with $300 credit
The instances have machine type c2-standard-4 and are using Linux Deep Learning
The gcloud config looks right to me:
$ gcloud config list
[component_manager]
disable_update_check = True
[compute]
gce_metadata_read_timeout_sec = 5
[core]
account = [my_account]
disable_usage_reporting = True
project = [my_project]
[metrics]
environment = devshell
Update:
I reset one of the instances and now I can successfully SSH into that instance. However the job running on the instance stopped after reset.
I want to keep the jobs running on the other instances. Is there a way to SSH into other instances without reset?
You issue is at the VM side. Task's you're running make the ssh service unable to accept incoming connection and only after the restart you were able to connect.
You should be able to see the instance's serial console output using gcloud compute instances get-serial-port-output [instance-name] but if for some reason you're not You may try instead using GCP console - go to the instance's details and click on Serial port 1 (console) and you will see the output.
You may even interact with your VM (login) via the console. This is particularily usefull if something stopped the ssh service but for that you need a login/password so first you have to access the VM or use the startup script to add a user with your password. But then again - this requires a restart.
In either case it seems that the restarting your VM's is the best option. But you may try to figure out what is causing ssh service to stop after some time by inspecting logs. Or you can create your own (disk space, memory, cpu etc) by using cron with df -Th /mountpoint/path | tail -n1 >> /name_of_the_log_file.log.
You can for example use cron for checking & starting ssh service.
And if something doesn't work as supposed to (according to documentation) - go to the IssueTracker and create a new issue to get more help.
I'm trying to execute a Script on a Google VM through Terraform.
First I tried it via Google Startup Scripts. But since the metadata is visible in the Google Console (startup scripts count as metadata) and that would mean that anybody with read access can see that script which is not acceptable.
So i tried to get the script from a Storage Account. But for that i need to attach a service account to the VM so the VM has the rights to access the Storage Account. Now people that have access to the VM also have access to my script as long as the service account is attached to the VM. In order to "detach" the service account i would have to stop the VM. Also if i don't want to permanently keep the attachment of the service account i would have to attach the service account via a script which requires another stop and start of the VM. This is probably not possible and also really ugly.
I don't understand how the remote-exec ressource works on GCP VMs. Because i have to specify a user and a userpassword to connect to the VM and then execute the script. But the windows password needs to be set manually via the google console, so i can't specify those things at this point in time.
So does anybody know how I can execute a Script where not anybody has access to my script via Terraform?
Greetings :) and Thanks in advance
I ended up just running a gcloud script in which i removed the Metadata from the VM after the Terraform apply was finished. In my Gitlab pipeline i just called the script in the "after_script"-section. Unfortunately the credentials are visible for approximately 3min.
I was trying to create VM instance on Google Could Platform, but it always gives me an error says Failed to load.
================
Answer
Found the solution by accident, I was browsing in the MarketPlace and trying to launch a Ubuntu 16.04 OS but end with this error
Compute Engine API has not been used in project xxxxxxxxxxx before or
it is disabled. Enable it by visiting
https://console.developers.google.com/apis/api/compute.googleapis.com/overview?project=xxxxxxxxxxx
then retry. If you enabled this API recently, wait a few minutes for
the action to propagate to our systems and retry.
So I goto the link mentioned in the above Error message. After I enabled Compute Engine API there I can find the Create Instance button in my VM instances section.
This error message is related to UI issues with cache and cookies. So to fix this you can try any of the following:
1.- Log In to GCP in an Incognito window.
2.- Clear/delete your web browser cache and cookies.
3.- Login to GCP through another web browser.
Please verify Compute Engine API is enabled in your project by following step 3 from this guide.
Additional, there was a known issue yesterday that prevent new projects to enable Compute Engine API and therefore not be able to create VM Instances. This issue has been fixed by now so you shoud be able to create this Instances with no problem.
I've just created a VM instance at GCP with os-login feature enabled. I can connect via ssh using web browser console, SDK, but, I can't connect using the android app called Cloud Console.
Is there any extra step while configuring os-login to use the offical google android app?
Thanks.
I believe everything should work out of the box. There certainly isn't mention of any extra documentation on the Cloud Console Mobile App landing page.
I spun up a new instance (europe-west4-b, n1-standard-1, debian-9-stretch-v20190813) and set the metadata to enable-oslogin = TRUE and I managed to connect right away, albeit from an iOS device.
I would suggest trying different networks (WiFi, 3G/4G), double check any suspicious firewall tags on the instance, maybe even try from a different device. If the connection attempt did reach the server, chances are there's at least one log entry mentioning it in /var/log (/var/log/auth.log in Debian is a good place to start).
Background
In our current Google Cloud Project it seems it's common that people create a VM, start it, do some of their stuff and then just leave it there without turning it off or deleting the VM.
I'm trying to write an automated script that pulls the list of VM instances every day and send an E-mail to the VM owners and see if they forgot to turn off their VMs.
Question
So is there an easy way to find out who created a GCE VM instance using Google Cloud API?
You can view the information in Stackdriver -> Logging -> Logs. The log is a JSON file, the actor field is what you are looking for. You can export the logs for analysis.
See https://cloud.google.com/logging/docs/