I am facing the following error while getting into Google Cloud Instance using the serial port. When I run this command, it starts throwing the error.
gcloud compute connect-to-serial-port instance-1
This is the error:
Sep 20 14:28:35 instance-1 OSConfigAgent[670]: 2022-09-20T14:28:35.5396Z OSConfigAgent Error main.go:196: network error when requesting metadata, make sure your instance has an active network and can reach the metadata server: Get http://169.254.169.254/computeMetadata/v1/?recursive=true&alt=json&wait_for_change=true&last_etag=b6d33d232458e45a&timeout_sec=60: dial tcp 169.254.169.254:80: connect: network is unreachable
Sep 20 14:29:33 instance-1 OSConfigAgent[670]: 2022-09-20T14:29:33.5432Z OSConfigAgent Warning: Error waiting for task (attempt 10 of 10): error fetching Instance IDToken: error getting token from metadata: Get http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/identity?audience=osconfig.googleapis.com&format=full: dial tcp 169.254.169.254:80: connect: network is unreachable
I am also unable to access the instance using external IP, and the SSH does not work either. SSH throws the following error:
These are my network rules.
I don't want to restart my instance as I have a job running in Jenkinwhich will destroy the whole day's runtime.
From your error message "Connection via Cloud Identity-Aware Proxy Failed" these error occur when you try to use SSH when connecting to a VM that doesn't have a public IP address for which you haven't configured Identity-Aware Proxy on port 22.
you can Create a firewall rule on port 22 that allows ingress traffic from Identity-Aware Proxy.
Also from what #John Hanley suggested to check your VM if it have a service account.
Related
I was no longer able to SSH into a Google Cloud Compute Engine VM instance that previously showed no problems.
The error logs show the following
#type: "type.googleapis.com/google.protobuf.Struct" value: {
conditionNotMet: { userVisibleMessage: "Supplied fingerprint does not
match current metadata fingerprint."
Trying SSH through the console showed
Code: 4003 Reason: failed to connect to backend Please ensure that:
your user account has iap.tunnelInstances.accessViaIAP permission
VM has a firewall rule that allows TCP ingress traffic from the IP range XXX.0/20, port: 22
you can make a proper https connection to the IAP for TCP hostname: https://tunnel.cloudproxy.app You may be able to connect without using
the Cloud Identity-Aware Proxy.
The VM instance logs showed the following
Error watching metadata: Get
http://metadata.google.internal/computeMetadata/v1//?recursive=true&alt=json&wait_for_change=true&timeout_sec=60&last_etag=XXX:
net/http: request canceled (Client.Timeout exceeded while awaiting
headers)
After stopping and restarting the instance I was able to ssh again but I would like to understand the reason for the problem in the first place.
The error message you received indicates that the metadata server's response caused the connection to the Google Compute Engine VM instance to time out. This could be because the server was taking too long to respond or there was a problem with the network. You can try to resolve this issue by either increasing the timeout value by using this doc or waiting for the instance to become healthy using the gcloud compute wait command.
The instance was unable to reach the metadata server, as suggested by the timeout error message. This could be a problem with the instance itself or with the network connection. A firewall or network configuration issue could have prevented the instance from connecting to the metadata server, or an issue with the underlying infrastructure could have rendered the instance temporarily unavailable.
To prevent this issue from happening again, you can increase the timeout value or use the gcloud compute wait command to wait for the instance to become healthy.it is recommended that you regularly update the SSH key used to connect to the instance, and check that the instance can reach the metadata server by making an HTTPS request to the IAP for TCP hostname. Additionally, it is important to ensure that your user account has the "iap.tunnelInstances.accessViaIAP" permission, and that the VM has a firewall rule that allows TCP ingress traffic from the IP range XXX.0/20, port: 22.
If you are using windows vm try troubleshooting steps mentioned in this doc.
This is regarding the GCP compute instances. My GCP instances are not able to fetch the GCE metadata from the metadata server & therefore when I am installing Openstack on google cloud (GCP) via packstack over a CentOS image I am getting this error.
VM instances are a part of default network with no Firewall rules. I am able to login the nodes externally also, which proves the network is OK.
ERROR : Error appeared during Puppet run: 10.142.0.16_compute.pp
Error: Facter: GCE metadata request failed: Timeout was reached
in /var/log/messages I am getting this message repeatedly:
Oct 25 20:16:31 controller-8 google_guest_agent[146448]: ERROR main.go:190 Network error when requesting metadata, make sure your instance has an active network and can reach the metadata server.
Oct 25 20:16:31 controller-8 google_guest_agent[146448]: ERROR main.go:193 Error watching metadata: Get http://169.254.169.254/computeMetadata/v1//?recursive=true&alt=json&wait_for_change=true&timeout_sec=60&last_etag=6f06fe6d055dd9f5: dial tcp 169.254.169.254:80: connect: no route to host
Oct 25 20:19:07 controller-8 OSConfigAgent[146888]: 2021-10-25T20:19:07.5468Z OSConfigAgent Error main.go:218: Get http://169.254.169.254/computeMetadata/v1/?recursive=true&alt=json&wait_for_change=true&last_etag=6f06fe6d055dd9f5&timeout_sec=60: dial tcp 169.254.169.254:80: connect: no route to host
Oct 25 20:20:08 controller-8 OSConfigAgent[146888]: 2021-10-25T20:20:08.9868Z OSConfigAgent Error main.go:218: network error when requesting metadata, make sure your instance has an active network and can reach the metadata server: Get http://169.254.169.254/computeMetadata/v1/?recursive=true&alt=json&wait_for_change=true&last_etag=6f06fe6d055dd9f5&timeout_sec=60: dial tcp 169.254.169.254:80: connect: no route to host
Oct 25 20:21:10 controller-8 OSConfigAgent[146888]: 2021-10-25T20:21:10.4268Z OSConfigAgent Error main.go:218: network error when requesting metadata, make sure your instance has an active network and can reach the metadata server: Get http://169.254.169.254/computeMetadata/v1/?recursive=true&alt=json&wait_for_change=true&last_etag=6f06fe6d055dd9f5&timeout_sec=60: dial tcp 169.254.169.254:80: connect: no route to host
Oct 25 20:22:10 controller-8 OSConfigAgent[146888]: 2021-10-25T20:22:10.7148Z OSConfigAgent Error main.go:218: network error when requesting metadata, make sure your instance has an active network and can reach the metadata server: Get http://169.254.169.254/computeMetadata/v1/?recursive=true&alt=json&wait_for_change=true&last_etag=6f06fe6d055dd9f5&timeout_sec=60: dial tcp 169.254.169.254:80: connect: no route to host
IIUC, according this error tips: connect: no route to host, try use the metadata.google.internal domain to replace this IP 169.254.169.254 ?
Check from this documentation Parts of a metadata request , and make sure you have the correctly /etc/resolv.conf file for nameserver before. And you should check the service account setting by John Hanley's suggestion.
Thanks, #John Hanley. Reference from your tips and this question -- "Why can't I access Metadata Server of GCP Instance?" 's earliest answer.
I want to access to SSH from GCP like as always but Error 4003 appears
Screenshoots of Error 4003
this is my VPC Firewall
Screenshoots of VPC Firewall
this is my Instance Details
Screenshoots of instance detail 1
Screenshoots of instance detail 2
Screenshoots of instance detail 3
this is my IAM
Screenshoots of IAM
This error is about you haven't configured Identity-Aware Proxy on port 22.
Failed to connect to backend
The following errors might occur when you connect to your VM from the Google Cloud Console or the gcloud tool:
The Cloud Console:
-- Connection via Cloud Identity-Aware Proxy Failed
-- Code: 4003
-- Reason: failed to connect to backend
The gcloud tool:
ERROR: (gcloud.compute.start-iap-tunnel) Error while connecting [4003: u'failed to connect to backend'].
These errors occur when you try to use SSH to connect to a VM that doesn't have a public IP address and for which you haven't configured Identity-Aware Proxy on port 22.
To resolve this issue Create a firewall rule on port 22 that allows ingress traffic from Identity-Aware Proxy.
click here for more information.
My GPC Linux VM is working fine and one day naturally can't log in, even though it is still running,
I used the nomachine to login and got error "authentication failed, please try again."
when SSH:
Connection via Cloud Identity-Aware Proxy Failed
Code: 4003
Reason: failed to connect to backend
You may be able to connect without using the Cloud Identity-Aware Proxy.
I also try add:
#! /bin/bash
useradd -G sudo USERNAME
echo 'USERNAME:PASSWORD' | chpasswd
and
route add default gw <gateway ip> eth0
I created another virtual machine and its image, but after a day I couldn't connect
Ensure you have a firewall rule to allow Cloud Identity-Aware Proxy (IAP) to connect to port 22 on the instance. Full instructions can be found at [1].
[1] https://cloud.google.com/iap/docs/using-tcp-forwarding?authuser=0#create-firewall-rule
Cause:
When an instance does not have a public IP address, SSH in a Browser needs to forward the SSH connection through IAP. The error "failed to connect to backend" indicates that the IAP proxy service was unable to open a TCP connection to the instance.
This is most often due to the VPC firewall not having a rule which allows the proxy to connect to the instance. It coulld also be due to the OS firewall or other VM network connectivity issue.
I have seen a similar error on stackoverflow, but with a different code (so maybe it is not the same?). Any how I have been thrown this error spontaneously. Sometimes after 20 seconds of starting my instance (and launching SSH in browser), sometimes after 30 minutes, but it completely shuts down my instance.
Connection via Cloud Identity-Aware Proxy Failed
Code: 4010
Reason: destination read failed
You may be able to connect without using the Cloud Identity-Aware Proxy.
If I click the "Cloud Identity-Aware Proxy" button I am getting:
Connection Failed
We are unable to connect to the VM on port 22. Learn more about possible causes of this issue.
Any idea what is happening? I havn't done any changes in my instance settings for a long time.
The issue regarding the Identity-Aware Proxy(IAP) connection to the instance is due to the lack of a firewall rule allow-ingress-from-iap with this IP ranges 35.235.240.0/20 that needs to be configured when using IAP.
To Allow SSH access to all VM instances in your network, do the following:
1- Open the Firewall Rules page (Navigation menu > VPC network > Firewall) and click Create firewall rule
2- Configure the following settings:
Name: allow-ingress-from-iap
Direction of traffic: Ingress
Target: All instances in the network
Source filter: IP ranges
Source IP ranges: 35.235.240.0/20
Protocols and ports: Select TCP and enter 22 to allow SSH
3- Click Create