I have a cloud run service which is trying to connect to a cloud SQL (postgres) instance. There is a timeout connecting:
File "/usr/local/lib/python3.10/site-packages/google/cloud/sql/connector/connector.py", line 261, in connect_async
return await asyncio.wait_for(get_connection(), timeout)
File "/usr/local/lib/python3.10/asyncio/tasks.py", line 458, in wait_for
raise exceptions.TimeoutError() from exc"
Checked that:
The instance is added in the cloud run config under "Cloud SQL connections"
This code works fine locally against the same instance with the same service account. To ensure it uses the service account, not my own login, checked:
gcloud auth revoke
Check that application doesn't work
export GOOGLE_APPLICATION_CREDENTIALS=/<serviceaccountcreds>.json
Check that application works.
Where do I dig?
The connection is gotten with the sample code, nothing fancy.
You can connect to Cloud SQL using TCP to private db instance IP by the VPC Connector. Or by the Cloud SQL Proxy (auto SSL/TSL) to the Cloud SQL Proxy Server available by default in the Cloud SQL instance (auth by IAM).
To test from your local dev PC you do:
gcloud components install cloud_sql_proxy
create an instance (you have one already..)
gcloud sql instances create sql-db ...
create a sample db:
gcloud sql databases create my_db --instance sql-db
connect with:
gcloud beta sql connect sql-db --user root
remove the default root user and configure a new one that can connect only passing by the Cloud SQL Proxy (if you want that kind of security)
gcloud sql users delete root --host % --instance sql-db
gcloud sql users create root --host "cloudsqlproxy~%" --instance sql-db
# network "cloudsqlproxy" is shared between
# cloud sql instance and the proxy server
to deploy the service with access to cloudsql:
DB_INSTANCE=$PROJECT\:$REGION\:sql-db
gcloud run deploy <SERVICE_NAME> \
--add-cloudsql-instances $DB_INSTANCE \
--set-env-vars DB="mysql://root#unix(/cloudsql/$DB_INSTANCE)/my_db"
# all other flags...
# DB is alternative to SOCKET_PATH, both for the Cloud SQL client init
the role for the service account to work with cloudsql:
gcloud run services add-iam-policy-binding $SERVICE_NAME \
--member serviceAccount:$SERVICE_ACCOUNT \
--role roles/cloudsql.client
Related
When I try to delete a private service connection using gcloud CLI, I'm getting an error message saying Producer services (Eg: CloudSQL) are still using the connection. There are no GCP resources created in this new GCP project.
dineshsonachalam#macbook iac % gcloud services vpc-peerings delete --network=default --service=servicenetworking.googleapis.com
ERROR: (gcloud.services.vpc-peerings.delete) The operation resulted in a failure - "Failed to delete connection: Producer services (Eg: CloudSQL, Cloud MemStore etc) are still using this connection"
But I was able to delete the private service connection using the Google Console UI. It would be very helpful if someone shared how to delete a private service connection using gcloud CLI.
I replicated the same scenario and I was able to delete the private connection through GCloud by using:
gcloud services vpc-peerings delete \
--service=servicenetworking.googleapis.com \
--network=VPC_NETWORK \
--project=PROJECT_ID
Trying connect SQL instance to Cloud Run Service, using Fully Managed cloud run works fine but when I try to connect service via Anthos (which is required as we need to use websockets on services) I just get ENOENT (No Entry), update IAM for GKE with correct permissions, recreated cluster with all services enabled/
Here's the deploy command I am doing
gcloud run deploy \
--project ${GOOGLE_PROJECT_ID} \
--platform gke \
--cluster dev \
--cluster-location ${GOOGLE_COMPUTE_ZONE} \
--image gcr.io/${GOOGLE_PROJECT_ID}/${PROJECT_NAME} \
--set-cloudsql-instances "${GOOGLE_PROJECT_ID}:europe-west1:dev" \
--set-env-vars "$(tr '\n' ',' < "${ENV_KEY_PRODUCTION}")" \
--set-env-vars "SERVICE=${1}" \
--set-env-vars "DB_HOST=/cloudsql/${GOOGLE_PROJECT_ID}:europe-west1:dev" \
"${1}"
If I use the private IP from SQL and remove --set-cloudsql-instances and set DB_HOST as private IP it works.
But adding --set-cloudsql-instances should make a sidecar for service in GKE cluster and allow it to connect to SQL?
The documentation isn't clear... the parameter '--set-cloudsql-instances' is only available for Cloud Run Managed version. The first sentence of the section is important. And the limitation is not clear in the doc
Only applicable if connecting to Cloud Run (fully managed). Specify --platform=managed to use:
--[no-]allow-unauthenticated
Whether to enable allowing unauthenticated access to the service. This may take a few moments to take effect. Use --allow-unauthenticated to enable and --no-allow-unauthenticated to disable.
--clear-vpc-connector
Remove the VPC connector for this Service.
--revision-suffix=REVISION_SUFFIX
Specify the suffix of the revision name. Revision names always start with the service name automatically. For example, specifying [--revision-suffix=v1] for a service named 'helloworld', would lead to a revision named 'helloworld-v1'.
--vpc-connector=VPC_CONNECTOR
Set a VPC connector for this Service.
These flags modify the Cloud SQL instances this Service connects to. You can specify a name of a Cloud SQL instance if it's in the same project and region as your Cloud Run service; otherwise specify :: for the instance. At most one of these may be specified:
--add-cloudsql-instances=[CLOUDSQL-INSTANCES,…]
Append the given values to the current Cloud SQL instances.
--clear-cloudsql-instances
Empty the current Cloud SQL instances.
--remove-cloudsql-instances=[CLOUDSQL-INSTANCES,…]
Remove the given values from the current Cloud SQL instances.
--set-cloudsql-instances=[CLOUDSQL-INSTANCES,…]
Completely replace the current Cloud SQL instances with the given values.
I am trying to connect google cloud sql instance with cloud function written in python.
global pg_pool, pg_pool_db_name
pg_config = {
'user': CONNECTION_DATA[dbname]['DB_USER'],
'password': CONNECTION_DATA[dbname]['DB_PASSWORD'],
'host': host,
'dbname': dbname
}
pg_pool = ThreadedConnectionPool(1, 1, **pg_config)
This function test call failed and it outputs error message:
Error: function terminated. Recommended action: inspect logs for termination reason. Details:could not connect to server: Connection refused
Is the server running locally and accepting
connections on Unix domain socket "/cloudsql/my_instance_name/.s.PGSQL.5432"?
Anybody experienced on this kind of situation.
I wrote a tutorial about Connect from google cloud functions to Cloud SQL
9.Create a service account for your cloud function. Ensure that the service account for your service has the following IAM roles: Cloud SQL Client, and for connecting from Cloud Functions to Cloud Sql on internal ip we need also the role Compute Network User.
gcloud iam service-accounts create cloud-function-to-sql
gcloud projects add-iam-policy-binding gcf-to-sql --member serviceAccount:cloud-function-to-sql#gcf-to-sql.iam.gserviceaccount.com --role roles/cloudsql.client
gcloud projects add-iam-policy-binding gcf-to-sql --member serviceAccount:cloud-function-to-sql#gcf-to-sql.iam.gserviceaccount.com --role roles/compute.networkUser
Then deploy the function using the service account you just created:
2.Deploy the cloud function:
gcloud beta functions deploy gcf_to_sql --runtime python37 --region europe-west2 --service-account cloud-function-to-sql --trigger-http
EDIT
Please read this documentation:
1.Creating a service account
2.Granting roles to service accounts
3.Deploying Cloud function from Cloud Console
Click More to display advanced options, such as setting a region,
specifying a timeout, or adding environment variables
On Advanced options, choose the service account you just created.
I just created a 2nd gen My SQL instance in GCP. I used the below command in the console to connect to the sql instance but I'm getting permissions error.
gcloud beta sql connect instance-name --user=root
You need the [cloud_sql_proxy] component to use the sql connect
command.
ERROR: (gcloud.beta.sql.connect) You cannot perform this action
because you do not have permission to modify the Google Cloud SDK
installation directory [/google/google-cloud-sdk].
I have tested the command and it is giving me the same error. After that I removed the beta and run the command gcloud sql connect instance-name --user=root.
I got the following message:
Whitelisting your IP for incoming connection for 5 minutes...done.
Connecting to database with SQL user [root].Enter password:
So try removing the beta from the command.
When I try to connect to Google Cloud SQL via the Cloud Shell, I get the following error:
gcloud beta sql connect mysql-1 --user=root
ERROR: (gcloud.beta.sql.connect) You do not have permission to access instance [mysql-1]: The client is not authorized to make this request.
I am not sure what permission is required to grant this access.
It turns out that I typed in my project name incorrectly when configuring the CLI.
gcloud config set project PROJECT_ID