Iam trying to set up aws cluster. I installed kubectl and configured aws with credentials. When I tried to display pods or any kubectl commands, Iam getting this error
revaa#revaa-Lenovo-E41-25:~$ kubectl get pod
Unable to connect to the server: dial tcp 10.0.12.77:443: i/o timeout
How to resolve this
The ip used is a private, it cannot be accessed outside the aws. Change your cluster's api server endpoint access to public. It worked.
Related
I was no longer able to SSH into a Google Cloud Compute Engine VM instance that previously showed no problems.
The error logs show the following
#type: "type.googleapis.com/google.protobuf.Struct" value: {
conditionNotMet: { userVisibleMessage: "Supplied fingerprint does not
match current metadata fingerprint."
Trying SSH through the console showed
Code: 4003 Reason: failed to connect to backend Please ensure that:
your user account has iap.tunnelInstances.accessViaIAP permission
VM has a firewall rule that allows TCP ingress traffic from the IP range XXX.0/20, port: 22
you can make a proper https connection to the IAP for TCP hostname: https://tunnel.cloudproxy.app You may be able to connect without using
the Cloud Identity-Aware Proxy.
The VM instance logs showed the following
Error watching metadata: Get
http://metadata.google.internal/computeMetadata/v1//?recursive=true&alt=json&wait_for_change=true&timeout_sec=60&last_etag=XXX:
net/http: request canceled (Client.Timeout exceeded while awaiting
headers)
After stopping and restarting the instance I was able to ssh again but I would like to understand the reason for the problem in the first place.
The error message you received indicates that the metadata server's response caused the connection to the Google Compute Engine VM instance to time out. This could be because the server was taking too long to respond or there was a problem with the network. You can try to resolve this issue by either increasing the timeout value by using this doc or waiting for the instance to become healthy using the gcloud compute wait command.
The instance was unable to reach the metadata server, as suggested by the timeout error message. This could be a problem with the instance itself or with the network connection. A firewall or network configuration issue could have prevented the instance from connecting to the metadata server, or an issue with the underlying infrastructure could have rendered the instance temporarily unavailable.
To prevent this issue from happening again, you can increase the timeout value or use the gcloud compute wait command to wait for the instance to become healthy.it is recommended that you regularly update the SSH key used to connect to the instance, and check that the instance can reach the metadata server by making an HTTPS request to the IAP for TCP hostname. Additionally, it is important to ensure that your user account has the "iap.tunnelInstances.accessViaIAP" permission, and that the VM has a firewall rule that allows TCP ingress traffic from the IP range XXX.0/20, port: 22.
If you are using windows vm try troubleshooting steps mentioned in this doc.
I am trying to push logs to Cloudwatch from a RHEL Instance. Originally I was getting the error:
[outputs.cloudwatchlogs] Aws error received when sending logs to LogGroup/LogStream: RequestError: send request failed caused by: Post "https://logs.<region>.amazonaws.com/": dial tcp xx.xx.xx.xx:443: i/o timeout
I tried everything I could think of, I saw some things online that it could be Proxy related. I have a proxy server instance on the AWS account.
I added the following into the common-config.toml for Cloudwatch:
[proxy] http_proxy = "htttp://${PROXY_SERVER}:$PORT" https_proxy = "http://${PROXY_SERVER}:$PORT" no_proxy = "XX.XX.XX.XX"
The error I am getting now is:
[outputs.cloudwatchlogs] Aws error received when sending logs to LogGroup/LogStream: RequestError: send request failed caused by: Post "https://logs.<region>.amazonaws.com/": proxyconnect tcp: dial tcp: lookup http on XX.XX.XX.XX:53: server misbehaving
I am in a private VPC and I can't ping public sites as I get 100% packet loss. I can manually run the AWS Cli command to push an entry into the log stream. For now I am just trying to push /var/log/messages from my instance. Can anyone help with why the CloudWatch logs aren't pushing?
Some thing I have tried that didn't work for the original error:
exporting no_proxy
adding AWS_STS_REGIONAL_ENDPOINTS as an env variable
Ensuring port 443 is open on SG's
Ensuring IAM profile has correct permissions for CW and EC2
I am facing the following error while getting into Google Cloud Instance using the serial port. When I run this command, it starts throwing the error.
gcloud compute connect-to-serial-port instance-1
This is the error:
Sep 20 14:28:35 instance-1 OSConfigAgent[670]: 2022-09-20T14:28:35.5396Z OSConfigAgent Error main.go:196: network error when requesting metadata, make sure your instance has an active network and can reach the metadata server: Get http://169.254.169.254/computeMetadata/v1/?recursive=true&alt=json&wait_for_change=true&last_etag=b6d33d232458e45a&timeout_sec=60: dial tcp 169.254.169.254:80: connect: network is unreachable
Sep 20 14:29:33 instance-1 OSConfigAgent[670]: 2022-09-20T14:29:33.5432Z OSConfigAgent Warning: Error waiting for task (attempt 10 of 10): error fetching Instance IDToken: error getting token from metadata: Get http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/identity?audience=osconfig.googleapis.com&format=full: dial tcp 169.254.169.254:80: connect: network is unreachable
I am also unable to access the instance using external IP, and the SSH does not work either. SSH throws the following error:
These are my network rules.
I don't want to restart my instance as I have a job running in Jenkinwhich will destroy the whole day's runtime.
From your error message "Connection via Cloud Identity-Aware Proxy Failed" these error occur when you try to use SSH when connecting to a VM that doesn't have a public IP address for which you haven't configured Identity-Aware Proxy on port 22.
you can Create a firewall rule on port 22 that allows ingress traffic from Identity-Aware Proxy.
Also from what #John Hanley suggested to check your VM if it have a service account.
My GPC Linux VM is working fine and one day naturally can't log in, even though it is still running,
I used the nomachine to login and got error "authentication failed, please try again."
when SSH:
Connection via Cloud Identity-Aware Proxy Failed
Code: 4003
Reason: failed to connect to backend
You may be able to connect without using the Cloud Identity-Aware Proxy.
I also try add:
#! /bin/bash
useradd -G sudo USERNAME
echo 'USERNAME:PASSWORD' | chpasswd
and
route add default gw <gateway ip> eth0
I created another virtual machine and its image, but after a day I couldn't connect
Ensure you have a firewall rule to allow Cloud Identity-Aware Proxy (IAP) to connect to port 22 on the instance. Full instructions can be found at [1].
[1] https://cloud.google.com/iap/docs/using-tcp-forwarding?authuser=0#create-firewall-rule
Cause:
When an instance does not have a public IP address, SSH in a Browser needs to forward the SSH connection through IAP. The error "failed to connect to backend" indicates that the IAP proxy service was unable to open a TCP connection to the instance.
This is most often due to the VPC firewall not having a rule which allows the proxy to connect to the instance. It coulld also be due to the OS firewall or other VM network connectivity issue.
I am following the Django sample for GAE and have problem to connect to Cloud SQL instance by Proxy from Google Cloud Shell. Possibly related to permission setting since I see the request not authorized,
Other context,
"gcloud beta sql connect auth-instance --user=root" has no problem to connect.
I have a service account for SQL Proxy Client.
I possibly miss something. Could someone please shed some light? Thanks in advance.
Thanks in advance.
Proxy log:
./cloud_sql_proxy -instances=auth-158903:asia-east1:auth-instance=tcp:3306
2017/02/17 14:00:59 Listening on 127.0.0.1:3306 for auth-158903:asia-east1:auth-instance
2017/02/17 14:00:59 Ready for new connections
2017/02/17 14:01:07 New connection for "auth-158903:asia-east1:auth-instance"
2017/02/17 14:03:16 couldn't connect to "auth-158903:asia-east1:auth-instance": dial tcp 107.167.191.26:3307: getsockopt: connection timed out
Client Log:
mysql -u root -p --host 127.0.0.1
Enter password:
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0
I also try with credential file but still no luck,
./cloud_sql_proxy -instances=auth-158903:asia-east1:auth-instance=tcp:3306 -credential_file=Auth-2eede8ae0d0b.jason
2017/02/17 14:21:36 using credential file for authentication; email=sql-proxy-client#auth-158903.iam.gserviceaccount.com
2017/02/17 14:21:36 Listening on 127.0.0.1:3306 for auth-158903:asia-east1:auth-instance
2017/02/17 14:21:36 Ready for new connections
2017/02/17 14:21:46 New connection for "auth-158903:asia-east1:auth-instance"
2017/02/17 14:21:48 couldn't connect to "auth-158903:asia-east1:auth-instance": ensure that the account has access to "auth-158903:asia-east1:auth-instance" (and make sure there's no typo in that name). Error during get instance auth-158903:asia-east1:auth-instance: googleapi: **Error 403: The client is not authorized to make this request., notAuthorized**
I can reproduce this issue exactly if I only give my service account "Cloud SQL Client" IAM role. When I give my service account the "Cloud SQL Viewer" role as well, it can then connect. I suggest you try this and see if it helps.
It looks like a network connectivity issue.
Read this carefully if you use a private IP :
https://cloud.google.com/sql/docs/mysql/private-ip
Note that the Cloud SQL instance is in a Google managed network and the proxy is meant to be used to simplify connections to the DB within the VPC network.
In short: running cloud-sql-proxy from a local machine will not work, because it's not in the VPC network. It should work from a Compute Engine VM that is connected to the same VPC as the DB.
What I usually do as a workaround is use gcloud ssh from a local machine and port forward over a small VM in compute engine, like:
gcloud beta compute ssh --zone "europe-north1-b" "instance-1" --project "my-project" -- -L 3306:cloud_sql_server_ip:3306
Then you can connect to localhost:3306 (make sure nothing else is running or change first port number to one that is free locally)
The Cloud SQL proxy uses port 3307 instead of the more usual MySQL port 3306. This is because it uses TLS in a different way and has different IP ACLs. As a consequence, firewalls that allow MySQL traffic won't allow Cloud SQL proxy by default.
Take a look and see if you have a firewall on your network that blocks port 3307. To use Cloud SQL proxy, authorize this port for outbound connections.