I am very new to GCP and I would greatly appreciate some help here ...
I have a docker containerized application that runs in AWS/Azure but needs to access gcloud SDK as well as through "Google cloud client libraries".
what is the best way to setup gcloud authentication from an application that runs outside of GCP?
In my Dockerfile, I have this (cut short for brevity)
ENV CLOUDSDK_INSTALL_DIR /usr/local/gcloud/
RUN curl -sSL https://sdk.cloud.google.com | bash
ENV PATH $PATH:$CLOUDSDK_INSTALL_DIR/google-cloud-sdk/bin
RUN gcloud components install app-engine-java kubectl
This container is currently provisioned from an Azure app service & AWS Fargate. When a new container instance is spawned, we would like it to be gcloud enabled with a service account attached already so our application can deploy stuff on GCP using its deployment manager.
I understand gcloud requires us to run gcloud auth login to authenticate to your account. How we can automate the provisioning of our container if this step has to be manual?
Also, from what I understand, for cloud client libraries, we can store the path to service account key json file in an environment variable (GOOGLE_APPLICATION_CREDENTIALS). So this file either has to be stored inside the docker image itself OR has to be mounted from an external storage at the very least?
How safe is it to store this service account key file in an external storage. What are the best practices around this?
There are two main means of authentication in Google Cloud Platform:
User Accounts: Belong to people, represent people involved in your project and they're associated to a Google Account
Service Accounts: Used by an application or an instance.
Learn more about their differences here.
Therefore, you are not required to use the command gcloud auth login to perform gcloud commands.
You should be using gcloud auth activate-service-account instead, along with the --key-file=<path-to-key-file> flag, which will allow you to authenticate without the need of signing into a Google Account with access to your project every time you need to call an API.
This key should be stored securely, preferably encrypted in the platform of your choice. Learn how to do it in GCP here following these steps as an example.
Take a look at these useful links for storing secrets in Microsoft Azure and AWS.
On the other hand, you can deploy services to GCP programmatically either using Cloud Libraries with your programming language of choice, or using Terraform is very intuitive if you prefer to do so over using the Google Cloud SDK through the CLI.
Hope this helped.
Related
I'm trying to connect an automated build. In the app I connect to a google api and in Cloud Run I access secrets both using different service accounts. It's just a test app so nothing major but I keep getting hung up with cloud build using the service account that I use to access the api to run the app. Anybody know of a way to dictate which service account is used to run the app while still using the key to access the api. My thinking is it has to be done in the build process with something in my cloudbuild.yaml or maybe can't be done?
The cloud build process is carried out by a special service account that handles everything. You can delegate the auth to other accounts but there really isn't any need. Running the container in cloud run and choosing a service account to access secrets is easy (from Cloud Run choose edit and deploy => security tab) and that service account has no bearing on how you access other api's with creds.json you add at runtime.
We have GCP account credentials(username/password). We have installed gcloud CLI on the Amazon Linux EC2 machine. We would like to create a script that would auto-login to the GCP account and do the below things sequentially using gcloud CLI.
Login to the GCP account.
Create Project and specify a meaningful project-id.
Create a service account with a meaningful ID.
Assign the owner role to the service account.
Create and download a new JSON key.
Please help us to achieve this
You should use a Service Account not a User (username|password) for automation. The Service Account should be suitably permissioned so that it can create Projects and Service Accounts.
I was unable to find a source for this (but it used to be that?) Google monitors User Accounts for apparent use of automation (e.g. for bots) and these accounts may be disabled.
I know this question is probably a bit vague. I was trying to run one of the examples of Google NLP Library in Google Shell.
I have 0 experience with using API, JSON, Nodejs... I don't understand what they are and how to use them.
Please help
Here is the snapshot of the error:
The error message means that you are using user credentials instead of service account credentials.
When you connect to Google Cloud Shell, you are using your Google Accounts User Credentials. Those credentials are the ones that you used to log in to the Google Cloud Console. When you run an application in Google Cloud Shell, your application is using those credentials unless you explicitly specify different credentials.
The solution is to create a service account in the Google Cloud Console. Then in your program use the service account for credentials for your application.
Google Cloud Service Accounts
When you do not specify the application credentials, the Google Client libraries use a method to locate credentials called ADC (Application Default Credentials). I wrote an article that might help you understand ADC:
Google Cloud Application Default Credentials
The simplest method for you is to create the environment variable GOOGLE_APPLICATION_CREDENTIALS to point to the service account full path location before running your application. Change the path below to point to where the service account is stored on Cloud Shell. You will need to first create the service acount, download it and then upload to Cloud Shell.
export GOOGLE_APPLICATION_CREDENTIALS="$HOME/service-account.json"
Managing files with Cloud Shell
This link will provide more information on how to write applications that use service accounts.
Setting Up Authentication for Server to Server Production Applications
I have just installed gcloud on a docker container.
When I try to run a command locally, I get the following error (which makes sense)
root#3c4b9a147de7:/# gcloud projects list
ERROR: (gcloud.projects.list) You do not currently have an active account selected.
Please run:
$ gcloud auth login
to obtain new credentials, or if you have already logged in with a
different account:
$ gcloud config set account ACCOUNT
to select an already authenticated account to use.
If I set appropriately the service account (to the service which the container will run from) will the above command work?
If I set appropriately the service account (to the service which the
container will run from) will the above command work?
I am not sure what you mean by "to the service which the container will run from". The service might have credentials assigned to it, but anything inside the container will not know this.
To use a service account with the Google Cloud SDK CLI, you need to configure the CLI to use the service account. The following command does this. Replace SA_EMAIL with your service account's email address. The email address can be found inside the service account JSON key file.
gcloud auth activate-service-account SA_EMAIL --key-file=service_account.json
After you run this command save the container so that the credentials will still be there the next time you launch this container.
gcloud auth login is not recommended in container because it references your identity. Prefer a service account. Store it in your container and define the environment variable GOOGLE_APPLICATION_CREDENTIAL to point to this file.
You can also run a gcloud auth activate-service-account with your service account file in param during your container build.
HOWEVER, it's not recommended to use this tool in container. With Cloud Run, it's even possible that this call will be blocked (I never try, but take care of the sandbox environment execution). Prefer the libs or the direct call to API.
At the moment, yes it's failing because the root account that you are using is running locally as you say so it doesn't have any permissions regarding your project.
When something is run through Cloud Run, then it's using by default the Compute Engine service account. If that account has the permissions needed, then the command will succeed. Take a look about authentication in Cloud Build here Something similar would apply for GKE regarding the service account as you can find here.
If the docker container has gcloud sdk installed (google cloud SDK Shell) , run the google Cloud SDK to authenticate using $ gcloud auth login after this you can run gcloud command line. check as well the path in Enviroments variable for the gcloud directory
I have such requirement to manipulate different services on different projects. Hence I need to activate multiple service accounts at the same time to do those jobs. Those service accounts' roles are well controlled so there's no way to make a "admin"-like account to run on all projects. So I'm wondering if there's a way to activate one service account only in current process or some isolated environment without impact on global gcloud info settings. Any suggestion is appreciated.
You can use glcoud config configurations and setup multiple account configurations to select from.
Then use the --configuration=configuration_name to select which one to use: gcloud compute instances list --configuration=NAME
Run gcloud init and select Create a new configuration.
You can change the default configuration with gcloud config configurations activate NAME.
I wrote an article that documents gcloud configurations:
Google Cloud – Understanding Gcloud Configurations
You can also activate service account credentials which will then be part of gcloud config configurations.
Creating and Authorizing Service Account Credentials with the CLI
gcloud auth activate-service-account test#development-123456.iam.gserviceaccount.com --key-file=test_google_account.json