I am following the Django sample for GAE and have problem to connect to Cloud SQL instance by Proxy from Google Cloud Shell. Possibly related to permission setting since I see the request not authorized,
Other context,
"gcloud beta sql connect auth-instance --user=root" has no problem to connect.
I have a service account for SQL Proxy Client.
I possibly miss something. Could someone please shed some light? Thanks in advance.
Thanks in advance.
Proxy log:
./cloud_sql_proxy -instances=auth-158903:asia-east1:auth-instance=tcp:3306
2017/02/17 14:00:59 Listening on 127.0.0.1:3306 for auth-158903:asia-east1:auth-instance
2017/02/17 14:00:59 Ready for new connections
2017/02/17 14:01:07 New connection for "auth-158903:asia-east1:auth-instance"
2017/02/17 14:03:16 couldn't connect to "auth-158903:asia-east1:auth-instance": dial tcp 107.167.191.26:3307: getsockopt: connection timed out
Client Log:
mysql -u root -p --host 127.0.0.1
Enter password:
ERROR 2013 (HY000): Lost connection to MySQL server at 'reading initial communication packet', system error: 0
I also try with credential file but still no luck,
./cloud_sql_proxy -instances=auth-158903:asia-east1:auth-instance=tcp:3306 -credential_file=Auth-2eede8ae0d0b.jason
2017/02/17 14:21:36 using credential file for authentication; email=sql-proxy-client#auth-158903.iam.gserviceaccount.com
2017/02/17 14:21:36 Listening on 127.0.0.1:3306 for auth-158903:asia-east1:auth-instance
2017/02/17 14:21:36 Ready for new connections
2017/02/17 14:21:46 New connection for "auth-158903:asia-east1:auth-instance"
2017/02/17 14:21:48 couldn't connect to "auth-158903:asia-east1:auth-instance": ensure that the account has access to "auth-158903:asia-east1:auth-instance" (and make sure there's no typo in that name). Error during get instance auth-158903:asia-east1:auth-instance: googleapi: **Error 403: The client is not authorized to make this request., notAuthorized**
I can reproduce this issue exactly if I only give my service account "Cloud SQL Client" IAM role. When I give my service account the "Cloud SQL Viewer" role as well, it can then connect. I suggest you try this and see if it helps.
It looks like a network connectivity issue.
Read this carefully if you use a private IP :
https://cloud.google.com/sql/docs/mysql/private-ip
Note that the Cloud SQL instance is in a Google managed network and the proxy is meant to be used to simplify connections to the DB within the VPC network.
In short: running cloud-sql-proxy from a local machine will not work, because it's not in the VPC network. It should work from a Compute Engine VM that is connected to the same VPC as the DB.
What I usually do as a workaround is use gcloud ssh from a local machine and port forward over a small VM in compute engine, like:
gcloud beta compute ssh --zone "europe-north1-b" "instance-1" --project "my-project" -- -L 3306:cloud_sql_server_ip:3306
Then you can connect to localhost:3306 (make sure nothing else is running or change first port number to one that is free locally)
The Cloud SQL proxy uses port 3307 instead of the more usual MySQL port 3306. This is because it uses TLS in a different way and has different IP ACLs. As a consequence, firewalls that allow MySQL traffic won't allow Cloud SQL proxy by default.
Take a look and see if you have a firewall on your network that blocks port 3307. To use Cloud SQL proxy, authorize this port for outbound connections.
Related
I was no longer able to SSH into a Google Cloud Compute Engine VM instance that previously showed no problems.
The error logs show the following
#type: "type.googleapis.com/google.protobuf.Struct" value: {
conditionNotMet: { userVisibleMessage: "Supplied fingerprint does not
match current metadata fingerprint."
Trying SSH through the console showed
Code: 4003 Reason: failed to connect to backend Please ensure that:
your user account has iap.tunnelInstances.accessViaIAP permission
VM has a firewall rule that allows TCP ingress traffic from the IP range XXX.0/20, port: 22
you can make a proper https connection to the IAP for TCP hostname: https://tunnel.cloudproxy.app You may be able to connect without using
the Cloud Identity-Aware Proxy.
The VM instance logs showed the following
Error watching metadata: Get
http://metadata.google.internal/computeMetadata/v1//?recursive=true&alt=json&wait_for_change=true&timeout_sec=60&last_etag=XXX:
net/http: request canceled (Client.Timeout exceeded while awaiting
headers)
After stopping and restarting the instance I was able to ssh again but I would like to understand the reason for the problem in the first place.
The error message you received indicates that the metadata server's response caused the connection to the Google Compute Engine VM instance to time out. This could be because the server was taking too long to respond or there was a problem with the network. You can try to resolve this issue by either increasing the timeout value by using this doc or waiting for the instance to become healthy using the gcloud compute wait command.
The instance was unable to reach the metadata server, as suggested by the timeout error message. This could be a problem with the instance itself or with the network connection. A firewall or network configuration issue could have prevented the instance from connecting to the metadata server, or an issue with the underlying infrastructure could have rendered the instance temporarily unavailable.
To prevent this issue from happening again, you can increase the timeout value or use the gcloud compute wait command to wait for the instance to become healthy.it is recommended that you regularly update the SSH key used to connect to the instance, and check that the instance can reach the metadata server by making an HTTPS request to the IAP for TCP hostname. Additionally, it is important to ensure that your user account has the "iap.tunnelInstances.accessViaIAP" permission, and that the VM has a firewall rule that allows TCP ingress traffic from the IP range XXX.0/20, port: 22.
If you are using windows vm try troubleshooting steps mentioned in this doc.
I am trying to explore a way to connect postgres 13 cloud sql instance with only private IP from my local windows machine ..
I am able to connect through a compute instance tied with the same subnet as a default GCP behavior..
I want it to be secure my cloud instance to be accessible only through a VPN setup.. I have a Open VPN CE installed and whitelisted the Open VPN ip in the GCP firewall rule. Still getting the error message.
psql: error: could not connect to server: Connection timed out
Is the server running on host "{ip_address}" and accepting
TCP/IP connections on port 5432?
There are methods though to connect via private IP having enabled cloud proxy. But is there a way that i can make it happen via other VPNs.?
When you receive this error most of the time it is due to not having
PostgreSQL configured to allow TCP/IP connections or at least no
connections from your particular workstation. Here is a list of
common causes to this problem:
a) postgresql.conf is not set up to allow TCP/IP connections.
You'll want to look at the listen_address configuration parameter.
b) postgresql.conf is not set up to allow connections on a
non-standard port number. To determine this look at the port
configuration option.
c) Authentication rules in PostgreSQL's access configuration file
(pg_hba.conf) are not set up to allow either your user or IP
address to connect to that database. See the
official-documentation for more information on setting up your
pg_hba.conf properly.
d) Ensure that there are no firewalls, such as iptables that are
keeping your local system from even establishing a connection to the
remote host. For common PostgreSQL problems and possible solutions,
check here.
You have to edit the postgresql.conf file and change the line
with 'listen_addresses'. This file you can find in the
/etc/postgresql/13/main directory. To connect PostgreSQL server from other computers, you have change this config line in this way:
listen_addresses = '*'
Then you have to edit the pg_hba.conf file, too. In this file you
have set, from which computers you can connect to this server and
what method of authentication you can use. Usually you will need
similar line:
host all all <IP address> md5
For detailed steps, you can check here.
Finally i had to set the cloud SQL proxy on a f1-micro VM instance having only private IP ..
Whitelisted the port 5432 on the firewall rule.
From table plus i use the private IP of the vm instance to connect to my cloud postgress instance.
Very well If somebody has other alternatives please do let me know.
My GPC Linux VM is working fine and one day naturally can't log in, even though it is still running,
I used the nomachine to login and got error "authentication failed, please try again."
when SSH:
Connection via Cloud Identity-Aware Proxy Failed
Code: 4003
Reason: failed to connect to backend
You may be able to connect without using the Cloud Identity-Aware Proxy.
I also try add:
#! /bin/bash
useradd -G sudo USERNAME
echo 'USERNAME:PASSWORD' | chpasswd
and
route add default gw <gateway ip> eth0
I created another virtual machine and its image, but after a day I couldn't connect
Ensure you have a firewall rule to allow Cloud Identity-Aware Proxy (IAP) to connect to port 22 on the instance. Full instructions can be found at [1].
[1] https://cloud.google.com/iap/docs/using-tcp-forwarding?authuser=0#create-firewall-rule
Cause:
When an instance does not have a public IP address, SSH in a Browser needs to forward the SSH connection through IAP. The error "failed to connect to backend" indicates that the IAP proxy service was unable to open a TCP connection to the instance.
This is most often due to the VPC firewall not having a rule which allows the proxy to connect to the instance. It coulld also be due to the OS firewall or other VM network connectivity issue.
I successfully created a Compute Engine VM instance, and installed MySQL on it using this guide:
https://cloud.google.com/solutions/setup-mysql
Now I want to connect to it from App Engine and from my home too if possible.
Using this guide: https://cloud.google.com/appengine/docs/standard/python/connecting-vpc
I created the connector, giving it the suggested 10.x.x.x/28 IP address.
In my app engine's app.yaml I inserted:
vpc_access_connector:
name: projects/xxxxxxxxxx/locations/europe-west3/conectors/xxxxxx
The VM instance shows an internal IP of 10.x.x.x and an external IP of 34.x.x.x
I am trying to connect from PHP using this line:
new mysqli($servername, $username, $password, $dbname, 3306, null);
but I get different errors.
When connecting to 34.x.x.x from my home:
mysqli::__construct(): (HY000/2002): No connection could be made because the target machine actively refused it.
(I even created a Firewall rule to allow all traffic from my home IP)
When connecting to10.x.x.x from app engine:
2002: Connection timed out
When connecting to 10.x.x.x from app engine:
2002: Connection refused
How can I make a connection?
My first guess is that the service inside the instance is not up and running, check if the service of MySQL is running and listening, you can try this by doing a nmap test vs. the public IP of your MySQL VM instance nmap 34.0.0.0. You should see something like this (if you are using the standard port):
PORT STATE SERVICE
22/tcp open ssh
80/tcp closed http
443/tcp closed https
3306/tcp open mysql
Remember that there are two firewalls you need to take care of on GCP, (VPC firewall & OS firewall). So if you are not able to see this port start the SQL service by connecting into your VM instance and typing sudo systemctl start mysql, now run again nmap 34.0.0.0 and you should see the service.
To connect from App Engine to MySQL on Compute Engine VM instance just follow this other guide .
To connect from your home It will be better for you to connect to the instance via SSH and then to access your database, or you can keep connecting by a VPN to access it with the internal IP address if you need to connect directly to the socket, just take care to avoid hitting the limitations or missing something like IAM roles and permission needed for this connection.
When creating my Postgres Cloud SQL instance I specified that would like to connect to it using private IP and chose my default network.
My VM sits in the same default network.
Now, I follow instructions as described here https://cloud.google.com/sql/docs/postgres/connect-compute-engine
and try executing
psql -h [CLOUD_SQL_PRIVATE_IP_ADDR] -U postgres
from my VM, but get this error:
psql: could not connect to server: Connection timed out Is the server
running on host "CLOUD_SQL_PRIVATE_IP_ADDR" and accepting TCP/IP connections on
port 5432?
Anything I am under-looking?
P.S. My Service Networking API (whatever that is) is enabled.
If you have ssh to a VM in the same network you can connect to Cloud SQL using cloud SQL proxy:
Open the ssh window (VM-instances in Computer engine and click on ssh), then download the proxy file with:
wget https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64 -O cloud_sql_proxy
Execute, in the ssh shell
chmod +x cloud_sql_proxy
Create a service account with role Cloud SQL Client and create an api key. Download the json key in your local computer.
In the ssh vm shell click on the wheel and "upload", and upload the key file
5.
./cloud_sql_proxy -instances=<Instance connection name>=tcp:5432 -credential_file=<name of the json file>
where "Instance connection name" can be found in SQL-Overview -> Connect to this instance
Finally
psql "host=127.0.0.1 port=5432 sslmode=disable user=<your-user-name> dbname=<your-db-name>"
On the other hand, if you want to connect to cloud sql from your local computer and the cloud sql instance does not have a public ip you have to connect through a bastion host configuration.
https://cloud.google.com/solutions/connecting-securely
According to this document connect via private ip, you need to setup following item:
You must have enabled the Service Networking API for your project. If you are using shared VPC , you also need to enable this API for the host project.
Enabling APIs requires the servicemanagement.services.bind IAM permission.
Establishing private services access requires the Network Administrator IAM role.
After private services access is established for your network, you do not need the Network Administrator role to configure an instance to use private IP.