I am trying to connect google cloud sql instance with cloud function written in python.
global pg_pool, pg_pool_db_name
pg_config = {
'user': CONNECTION_DATA[dbname]['DB_USER'],
'password': CONNECTION_DATA[dbname]['DB_PASSWORD'],
'host': host,
'dbname': dbname
}
pg_pool = ThreadedConnectionPool(1, 1, **pg_config)
This function test call failed and it outputs error message:
Error: function terminated. Recommended action: inspect logs for termination reason. Details:could not connect to server: Connection refused
Is the server running locally and accepting
connections on Unix domain socket "/cloudsql/my_instance_name/.s.PGSQL.5432"?
Anybody experienced on this kind of situation.
I wrote a tutorial about Connect from google cloud functions to Cloud SQL
9.Create a service account for your cloud function. Ensure that the service account for your service has the following IAM roles: Cloud SQL Client, and for connecting from Cloud Functions to Cloud Sql on internal ip we need also the role Compute Network User.
gcloud iam service-accounts create cloud-function-to-sql
gcloud projects add-iam-policy-binding gcf-to-sql --member serviceAccount:cloud-function-to-sql#gcf-to-sql.iam.gserviceaccount.com --role roles/cloudsql.client
gcloud projects add-iam-policy-binding gcf-to-sql --member serviceAccount:cloud-function-to-sql#gcf-to-sql.iam.gserviceaccount.com --role roles/compute.networkUser
Then deploy the function using the service account you just created:
2.Deploy the cloud function:
gcloud beta functions deploy gcf_to_sql --runtime python37 --region europe-west2 --service-account cloud-function-to-sql --trigger-http
EDIT
Please read this documentation:
1.Creating a service account
2.Granting roles to service accounts
3.Deploying Cloud function from Cloud Console
Click More to display advanced options, such as setting a region,
specifying a timeout, or adding environment variables
On Advanced options, choose the service account you just created.
Related
I have a cloud run service which is trying to connect to a cloud SQL (postgres) instance. There is a timeout connecting:
File "/usr/local/lib/python3.10/site-packages/google/cloud/sql/connector/connector.py", line 261, in connect_async
return await asyncio.wait_for(get_connection(), timeout)
File "/usr/local/lib/python3.10/asyncio/tasks.py", line 458, in wait_for
raise exceptions.TimeoutError() from exc"
Checked that:
The instance is added in the cloud run config under "Cloud SQL connections"
This code works fine locally against the same instance with the same service account. To ensure it uses the service account, not my own login, checked:
gcloud auth revoke
Check that application doesn't work
export GOOGLE_APPLICATION_CREDENTIALS=/<serviceaccountcreds>.json
Check that application works.
Where do I dig?
The connection is gotten with the sample code, nothing fancy.
You can connect to Cloud SQL using TCP to private db instance IP by the VPC Connector. Or by the Cloud SQL Proxy (auto SSL/TSL) to the Cloud SQL Proxy Server available by default in the Cloud SQL instance (auth by IAM).
To test from your local dev PC you do:
gcloud components install cloud_sql_proxy
create an instance (you have one already..)
gcloud sql instances create sql-db ...
create a sample db:
gcloud sql databases create my_db --instance sql-db
connect with:
gcloud beta sql connect sql-db --user root
remove the default root user and configure a new one that can connect only passing by the Cloud SQL Proxy (if you want that kind of security)
gcloud sql users delete root --host % --instance sql-db
gcloud sql users create root --host "cloudsqlproxy~%" --instance sql-db
# network "cloudsqlproxy" is shared between
# cloud sql instance and the proxy server
to deploy the service with access to cloudsql:
DB_INSTANCE=$PROJECT\:$REGION\:sql-db
gcloud run deploy <SERVICE_NAME> \
--add-cloudsql-instances $DB_INSTANCE \
--set-env-vars DB="mysql://root#unix(/cloudsql/$DB_INSTANCE)/my_db"
# all other flags...
# DB is alternative to SOCKET_PATH, both for the Cloud SQL client init
the role for the service account to work with cloudsql:
gcloud run services add-iam-policy-binding $SERVICE_NAME \
--member serviceAccount:$SERVICE_ACCOUNT \
--role roles/cloudsql.client
I was using gcloud with a service account to try to figure out why my API Gateway endpoint didn't work when I ran into another problem. First I ran this export GOOGLE_APPLICATION_CREDENTIALS=/path/to/credential/fils/PROJECTNAME-hash.json. Then I ran gcloud services list --available and I got this in my terminal:
ERROR: (gcloud.services.list) User [<SERVICE ACCOUNT NAME>#<MY PROJECT NAME>.iam.gserviceaccount.com] does not have permission to access projects instance [<MY PROJECT NAME>] (or it may not exist): Permission denied to list services for consumer container [projects/<MY PROJECT ID>]
Help Token: <WHAT LOOKS LIKE AN API KEY>
- '#type': type.googleapis.com/google.rpc.PreconditionFailure
violations:
- subject: ?error_code=110002&service=cloudresourcemanager.googleapis.com&permission=serviceusage.services.list&resource=projects/<MY PROJECT NAME>
type: googleapis.com
- '#type': type.googleapis.com/google.rpc.ErrorInfo
domain: serviceusage.googleapis.com
metadata:
permission: serviceusage.services.list
resource: projects/<MY PROJECT NAME>
service: cloudresourcemanager.googleapis.com
reason: AUTH_PERMISSION_DENIED
I believe I have the correct permissions enabled in my service account:
So why am I getting this error and how do I get gcloud services list --available to work with the selected service account?
This problem seemed to stem from the fact that I needed to set the service account to have the role of serviceusage.serviceUsageViewer. In order to do that I need to run the add-iam-policy-binding command but this command needs to be run with an account that has account owner/editor permissions.
Step 1 was to switch the account in gcloud to the master gmail account with which I signed up for GCP services.
I set my gcloud "account" to my master Gmail account with gcloud config set account <MASTER GMAIL ACCOUNT>. Then I ran:
gcloud projects add-iam-policy-binding <PROJECT ID> \
--member "serviceAccount:<SERVICE ACCOUNT>#<PROJECT ID>.iam.gserviceaccount.com" \
--role "roles/serviceusage.serviceUsageViewer"
That command succeeded. I set the gcloud account back to the service account with gcloud config set account <SERVICE ACCOUNT EMAIL> and then ran gcloud services list --available. This command worked this time.
I am trying to create a cluster with GKE. I have a project I have been using already.
When I run
gcloud container clusters create cluster1
I get the following:
ERROR: (gcloud.container.clusters.create) ResponseError: code=403, message=Google Compute Engine: Required 'compute.networks.get' permission for 'projects//global/networks/default'.
The same thing happens when I use the web UI. Both my service account and my user have owner roles.
I have tried the following to get the cluster create command to work:
I tried adding a policy binding for the project for my existing service account:
gcloud projects add-iam-policy-binding <my-project> \
--member serviceAccount:<my-user>#<my-project>.iam.gserviceaccount.com \
--role roles/compute.admin
I read enabling the container api service was required
gcloud services enable container.googleapis.com
Started over. I deleted the service account, created a new one and activated the creds with:
gcloud auth activate-service-account <my-user>#<my-project>.iam.gserviceaccount.com --key-file ${GOOGLE_APPLICATION_CREDENTIALS}
I also tried authenticating with my account user:
gcloud auth login
None of these work and I can't create a cluster
I think I will answer my own question here. From service account docs
When you create a new Cloud project using GCP Console and if Compute Engine API is enabled for your project, a Compute Engine Service account is created for you by default. It is identifiable using the email:
PROJECT_NUMBER-compute#developer.gserviceaccount.com
I had delete the default created service accounts somehow and possible the associated roles. I think this is why I couldn't create a cluster under my project anymore. Rather than try to figure out how to recreate, I decided it was best to just start a new project. Afterwords, the cluster create API and console work just fine.
Debug:
gcloud container subnets list-usable --project service-project --network-project shared-vpc-project
If you get warning in output:
WARNING: Failed to get metadata from network project. GCE_PERMISSION_DENIED:
Google Compute Engine: Required 'compute.projects.get' permission for
'projects/shared-vpc-project'
It means your google managed gke service account in host project doesn't exist.
To solve go to host project apis and enable Kubernetes Engine API. If it's enabled, disable it and enable again back.
I think you should set the compute engine service account permission:
gcloud projects add-iam-policy-binding <my-project> \
--member [PROJECT_NUMBER]-compute#developer.gserviceaccount.com \
--role roles/compute.admin
I'm working on a series of Cloud Functions in one Google Cloud project and, for some reason, I suddenly get this error:
Deployment failure:
Missing necessary permission resourcemanager.projects.getIamPolicy for service-1092904037961#gcf-admin-robot.iam.gserviceaccount.com on resource projects/la-cloud-functions. Please grant service-1092904037961#gcf-admin-robot.iam.gserviceaccount.com the Cloud Functions Service Agent role. You can do that by running 'gcloud iam service-accounts add-iam-policy-binding projects/la-cloud-functions --member=service-1092904037961#gcf-admin-robot.iam.gserviceaccount.com --role=Cloud Functions Service Agent'
Besides the badly formatted error response (you can't have --role=Cloud Functions Service Agent - it should be --role=roles/cloudfunctions.serviceAgent), when I try to run the amended command:
gcloud iam service-accounts add-iam-policy-binding projects/la-cloud-functions --member=service-1092904037961#gcf-admin-robot.iam.gserviceaccount.com --role=roles/cloudfunctions.serviceAgent
I get this error:
The requested URL <code>/v1/projects/la-cloud-functions/serviceAccounts/projects/la-cloud-functions:getIamPolicy?alt=json</code> was not found on this server.
Finally, trying to assign the Cloud Functions Server Agent role through the console gave me another surprise - the role is missing from the list, where it should be under Service Management:
I have tried to reset the service account by re-enabling the Cloud Functions API with this command:
gcloud services enable cloudfunctions.googleapis.com
But again, no success.
Anyone have any ideas on how to fix this problem and make the Cloud Functions Service Agent role available again?
TIA - Joe
Try the following steps to solve this:
Disable Cloud Functions API:
gcloud services disable cloudfunctions.googleapis.com --project la-cloud-functions
Wait about a minute for the disable to complete.
Delete the cloud functions member account using the CLI or using the GCP Console under IAM.
gcloud projects remove-iam-policy-binding la-cloud-functions --member="serviceAccount:service-1092904037961#gcf-admin-robot.iam.gserviceaccount.com" --role="roles/cloudfunctions.serviceAgent"
Wait about a minute. Then verify that this member has been removed in the GCP Console under IAM.
Enable Cloud Functions API:
gcloud services enable cloudfunctions.googleapis.com --project la-cloud-functions
Go back to the GCP Console. You should find a new Google Cloud Functions Service Agent member.
Note:
You are using the wrong command to add cloudfunctions.serviceAgent. Here is the correct command:
gcloud projects add-iam-policy-binding la-cloud-functions --member="serviceAccount:service-1092904037961#gcf-admin-robot.iam.gserviceaccount.com" --role="roles/cloudfunctions.serviceAgent"
When I try to connect to Google Cloud SQL via the Cloud Shell, I get the following error:
gcloud beta sql connect mysql-1 --user=root
ERROR: (gcloud.beta.sql.connect) You do not have permission to access instance [mysql-1]: The client is not authorized to make this request.
I am not sure what permission is required to grant this access.
It turns out that I typed in my project name incorrectly when configuring the CLI.
gcloud config set project PROJECT_ID