I have the following command to deploy a google cloud function.
I use the same command to deploy a google cloud function of gen1 ( except the flag --gen2 ) and everything works fine. In the project that has the database, i have set the role of Cloud Sql Client with the service account that is set in the command.
But when i deploy a google cloud function of gen2, i can't connect to the database of the other project. Given that gen2 cloud functions use Cloud Run i have also checked the service-account that Cloud Run uses, and it matches the service account that has the Cloud Sql Client in the project that has the database.
I get this error
SQLSTATE[HY000] [2002] No such file or directory
gcloud functions deploy $CLOUD_FUNCTION_NAME-staging-gen2
--gen2
--source=./src
--runtime=$CLOUD_FUNCTION_PHP_VERSION
--trigger-http
--entry-point=$CLOUD_FUNCTION_ENTRY_POINT
--allow-unauthenticated
--region=$GCP_ZONE
--project=$PROJECT_ID
--env-vars-file .env.yml
--min-instances=$MINIMUM_NUMBER_OF_INSTANCES
--service-account=*******#appspot.gserviceaccount.com
${VPC_CONNECTOR:+--vpc-connector projects/$PROJECT_ID/locations/$GCP_ZONE/connectors/$VPC_CONNECTOR}
I have also tried to not set the --service-account when i deploy and let it use the default. Then i added the default service account to the project which has the database, in the IAM section with the role of Cloud Sql Client but it still does not work.
I use unix_socket to connect to the database, because the database is hosted in another project.
"mysql:dbname=${dbname};port=${port};unix_socket=/cloudsql/${instanceConnectionName}"
Where instanceConnectionName is project_name:europe-west4:database_name
According to https://cloud.google.com/sql/docs/mysql/connect-functions, you have to add the connection to the cloud run service that is automatically deployed with the gen 2 cloud function. This is the case for other database types as well. Scroll to "If you're using Cloud Functions (2nd gen) and not Cloud Functions (1st gen)"
Related
I would like to connect to a Cloud SQL instance from Cloud Run, using a service account. The connection used to be created within the VPC and we would just provide a connection string with a user and a password to our PostgreSQL client. But now we want the authentication to be managed by Google Cloud IAM, with the service account associated with the Cloud Run service.
On my machine, I can use the enable_iam_login argument to use my own service account. The command to run the Cloud SQL proxy would look like this:
./cloud_sql_proxy -dir=/cloudsql -instances=[PROJECT-ID]:[REGION]:[INSTANCE] \
-enable_iam_login -credential_file=${HOME}/.config/gcloud/application_default_credentials.json
The problem is that I can't seem to find a way to use the IAM authentication method to run the Cloud SQL Proxy from Cloud Run, I can just provide an instance name. Has anyone face this problem before?
Unfortunately, there isn't a way to configure Cloud Run's use of the Cloud SQL proxy to do this for you.
If you are using Java, Python, or Go, there are language specific connectors you can use from Cloud Run. These all have the option to use IAM DB AuthN as part of them.
For anyone using NodeJS on Cloud Run :
you can't enable the -enable_iam_login option in Cloud Run (cf accepted answer), which means you have to use a connector.
AFAIK, there is no NodeJS connector right now.
Which means you have 2 options :
write a connector yourself (good luck) or wait for Google to do it
use built-in database authentication for now.
I am very new to GCP and I would greatly appreciate some help here ...
I have a docker containerized application that runs in AWS/Azure but needs to access gcloud SDK as well as through "Google cloud client libraries".
what is the best way to setup gcloud authentication from an application that runs outside of GCP?
In my Dockerfile, I have this (cut short for brevity)
ENV CLOUDSDK_INSTALL_DIR /usr/local/gcloud/
RUN curl -sSL https://sdk.cloud.google.com | bash
ENV PATH $PATH:$CLOUDSDK_INSTALL_DIR/google-cloud-sdk/bin
RUN gcloud components install app-engine-java kubectl
This container is currently provisioned from an Azure app service & AWS Fargate. When a new container instance is spawned, we would like it to be gcloud enabled with a service account attached already so our application can deploy stuff on GCP using its deployment manager.
I understand gcloud requires us to run gcloud auth login to authenticate to your account. How we can automate the provisioning of our container if this step has to be manual?
Also, from what I understand, for cloud client libraries, we can store the path to service account key json file in an environment variable (GOOGLE_APPLICATION_CREDENTIALS). So this file either has to be stored inside the docker image itself OR has to be mounted from an external storage at the very least?
How safe is it to store this service account key file in an external storage. What are the best practices around this?
There are two main means of authentication in Google Cloud Platform:
User Accounts: Belong to people, represent people involved in your project and they're associated to a Google Account
Service Accounts: Used by an application or an instance.
Learn more about their differences here.
Therefore, you are not required to use the command gcloud auth login to perform gcloud commands.
You should be using gcloud auth activate-service-account instead, along with the --key-file=<path-to-key-file> flag, which will allow you to authenticate without the need of signing into a Google Account with access to your project every time you need to call an API.
This key should be stored securely, preferably encrypted in the platform of your choice. Learn how to do it in GCP here following these steps as an example.
Take a look at these useful links for storing secrets in Microsoft Azure and AWS.
On the other hand, you can deploy services to GCP programmatically either using Cloud Libraries with your programming language of choice, or using Terraform is very intuitive if you prefer to do so over using the Google Cloud SDK through the CLI.
Hope this helped.
I have Cloud SQL and Instance in same project of Google Cloud.
I want to connect to Mysql (from Google Cloud SQL) from Instance using command line.
I am connecting using command:
gcloud sql connect cloud-sql-name --user=username
its giving:
ERROR: (gcloud.sql.connect) There was no instance found or you are not authorized to connect to it.
How do I make connection.
You probably missed the authorization of your instance's IP to be able to connect to the Cloud SQL database. This is done for e.g. through the Cloud Console in the Cloud SQL Instances page.
To see the step-by-step guide for connecting from Compute Engine instance to Cloud SQL (with the mysql client) check this docs page.
Are you trying to connect to your Cloud SQL instance from your Compute Engine VM instance?
If this is the case, I can suggest to first grant your service account (the Compute Engine default service account or a new one) the corresponding IAM role/permission for Cloud SQL [1].
Then, authenticate/activate [2] your service account within your VM instance command line (it is recommended to generate a JSON key File instead of a P12 key File for your service account):
~$ gcloud auth activate-service-account [ACCOUNT] --key-file=KEY_FILE
To generate a key File for your service account, you can go to your Products and Services menu > IAM & admin > Service accounts. Click on the three-dotted button at the right of the corresponding service account and select "Create key".
Your key File should be stored/uploaded to your VM instance, so that it can use it to activate the service account.
It is possible that you would also need to grant your VM instance access for Cloud SQL within its Cloud API access scopes panel. Go to your Products and Services menu > Compute Engine > VM instances. Select your VM instance and edit it.
Be aware that you will need to stop your VM instance before editing Cloud API access scopes. Go to "Access scopes" > "Set access for each API". Enable and Save.
You will also have to enable the Cloud SQL Admin API. Go to your Products and Services menu > API and services. Search for SQL Admin and Enable it (wait a few minutes)
Start the VM instance and try your Cloud SQL tests (re-activate your service account if necessary).
Your Cloud SQL instance for MySQL creates the "root" user. Just make sure to use this as the username, or any other you have created (if any).
Take into account that since you would be connecting from a Compute Engine VM instance, it is possible that yo will be asked for a MySQL Client.
[1] https://cloud.google.com/sql/docs/mysql/project-access-control
[2] https://cloud.google.com/sdk/gcloud/reference/auth/activate-service-account
I'm attempting to connect to a Google Cloud SQL instance via a MySQL client which requires me to connect via the Google Cloud proxy. I have followed all instructions on the Google Cloud Docs, yet after running the command to start the proxy (using correct INSTANCE_CONNECTION_NAME and PATH_TO_KEY_FILE), I receive the following error.
Command :
./cloud_sql_proxy -dir=/cloudsql -instances=INSTANCE_CONNECTION_NAME \ - credential_file=PATH_TO_KEY_FILE &
Error :
couldn't connect to "INSTANCE_CONNECTION_NAME": ensure that the account has access to "INSTANCE_CONNECTION_NAME" (and make sure there's no typo in that name). Error during createEphemeral for INSTANCE_CONNECTION_NAME: googleapi: Error 403: Access Not Configured. Cloud SQL Administration API has not been used in project 563584335869 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/sqladmin.googleapis.com/overview?project=563584335869 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry., accessNotConfigured
I have confirmed that the Google Cloud SQL API is enabled on the console for the project, but the weird thing is my project name is not 563584335869, and I have already set the correct project name via the GC CLI to no avail. So when visiting https://console.developers.google.com/apis/api/sqladmin.googleapis.com/overview?project=563584335869, I see:
The API "sqladmin.googleapis.com" doesn't exist or you don't have permission to access it
I also created a service account under the role Cloud SQL Client, and downloaded the private key which PATH_TO_KEY_FILE points to.
Can you confirm that you have enabled the "Cloud SQL API" (sqladmin.googleapis.com) and not "Cloud SQL" API? (sql-component.googleapis.com)?
Can you also give some more information as to your environment? Are you running this from your local machine or are you working in Cloud Shell?
I'm trying to connect a GCE instance to a Cloud SQL instance (2nd gen) with Cloud SQL Proxy. When i ssh to the GCE instance and
sudo ./cloud_sql_proxy -dir=/cloudsql
I get
2016/07/13 18:59:28 Using gcloud's active project: xxxx-0000
2016/07/13 18:59:29 Error listing instances in xxxx-0000:
googleapi: Error 403: Access Not Configured.
Cloud SQL Administration API has not been used in project 000000000000
before or it is disabled. Enable it by visiting
https://console.developers.google.com/apis/api/sqladmin/overview?project=000000000000
then retry.
If you enabled this API recently, wait a few minutes for the action to
propagate to our systems and retry., accessNotConfigured
2016/07/13 18:59:29 no Cloud SQL Instances found in these projects: [xxxx-0000]
I have followed the documentations at
https://cloud.google.com/sql/docs/compute-engine-access#gce-connect-proxy and
https://cloud.google.com/sql/docs/sql-proxy
Both instances are within the same project and Cloud SQL API is enabled on the GCE instance (plus the default ones.) By my understanding, the default GCE service account should be able to authenticate the proxy.
Using cloud_sql_proxy with -instances seems to work but errors with accessNotConfigured too while trying to connect.
As per the error message, you need make sure that the "Cloud SQL Admin" API is enabled by visiting the provided link.
There's also a GUI way to connect your Compute Engine instance if you're using a 2nd gen GCloud SQL instance. You first get the external IP of your compute engine and include it in the Authorizations tab of your GCloud SQL dashboard. Here's a quick video
Enabling Cloud SQL Admin API will solve your problem: