I am trying to authenticate a service account by gcloud auth login command using workload identity federation based on whats mentioned in this official tutorial. Even though the tutorial says both service account keys and workload identity federation works for my use case, WIF is the preferred route forward using credential configuration file. But I am quite confused trying to generate the file for my use case as doing so requires me to create a workload identity provider which are categorized to be among the following types:
AZURE,AWS,OIDC,SAML. I just want to use WIF to authenticate gcloud SDK from my terminal so I am not sure which category should I use.
Is this a possible use case or should I resort to use service account keys ?
But I am quite confused trying to generate the file for my use case as doing so requires me to create a workload identity provider which are categorized to be among the following types:
AZURE,AWS,OIDC,SAML. I just want to use WIF to authenticate gcloud SDK from my terminal so I am not sure which category should I use.
Is this a possible use case or should I resort to use service account keys ?
Workload Identity Federation(WIF), is used in multi-cloud environments and hybrid cloud environments where one needs access to one cloud platform from another cloud platform or from a data center as the services are catered across multiple platforms and needs coordination for running your application.
There are multiple ways to connect other cloud providers with GCP, you can use WIF for connecting with Amazon Web Services (AWS), you could use OpenID Connect (OIDC) or SAML 2.0 to connect with any other cloud providers, such as Microsoft Azure. Refer to the source for more information. (Source: GCP docs)
Since you are trying to connect to gcloud SDK from your terminal you can simply use your credential file or gcloud auth or gcloud init commands for setting up the gcloud cloud SDK and have necessary roles and permissions enabled for the service or user account which you are using for authentication. This is the simplest way to access your gcp environment. JFYI, in Authorize the gcloud CLI documentation(the doc which you were referring to) they are using the credential file which is different from WIF, so if you want to authenticate without using SA(service account) you can simply follow credential file based authentication.
Related
I have a java application that is deployed on GKE cluster. Let's call it the "orchestrator"
The application should be able to deploy other applications on same GCP project where the "orchestrator" app is running (can be same GKE or different GKE cluster), using helm cli commands.
We were able to do that using Google Service Account authentication, where the JSON key is provided to the "orchestrator" and we could use it to generate tokens.
My question is.. since both the "orchestrator" and the others apps are running on same GCP project (sometimes on same GKE cluster), is there a way to use some default credentials auto discovered by GCP, instead of generating and providing a Service Account JSON key to the "orchestrator" app?
That way, the customer won't need to expose this Key to our system and the authentication will be happened behind the scenes, without our app intervention.
Is there something a GCP admin can do which make this use case work seamlessly?
I will elaborate on my comment.
When you are using a Service Account, you have to use keys to authenticate - Each service account is associated with a public/private RSA key pair. As you are working on GKE cluster, did you consider using Workload identity, like mentioned in Best practices for using and managing SA?
According to Best practices for using and managing service accounts all non-human accounts should be represented by Service Account:
Service accounts represent non-human users. They're intended for scenarios where a workload, such as a custom application, needs to access resources or perform actions without end-user involvement.
So in general, whenever you want to provide some permissions to applications, you should use Service Account.
In Types of keys for service accounts you can find information, that all Service Accounts needs RSA pair key:
Each service account is associated with a public/private RSA key pair. The Service Account Credentials API uses this internal key pair to create short-lived service account credentials, and to sign blobs and JSON Web Tokens (JWTs). This key pair is known as the Google-managed key pair.
In addition, you can create multiple public/private RSA key pairs, known as user-managed key pairs, and use the private key to authenticate with Google APIs. This private key is known as a service account key.
You could also think about Workload Identity, but I am not sure if this would fulfill your needs as there are still many unknowns about your environment.
Just as additional information, there was something called Basic Authentication which could be an option for you, but due to security reasons it's not supported since GKE 1.19. This was mentioned in another stack case: We have discouraged Basic authentication in Google Kubernetes Engine (GKE).
To sum up:
Best Practice to provide permissions for non-human accounts is to use Service Account. Each service account requires a pair of RSA Keys and you can create multiple keys.
Good Practice is also to use Workload Identity if you have this option, but due to lack of details it is hard to determine if this would work in your scenario.
Additional links:
Authenticating to the Kubernetes API server
Use the Default Service Account to access the API server
One way to achieve that is to use use default credentials approach mentioned here :
Finding credentials automatically. Instead of exposing the SA key to our App, the GCP admin can attach the same SA to the GKE cluster resource (see attached screenshot), and the default credentials mechanism will use that SA credentials to get access the APIs and resources (depends on the SA roles and permissions).
New to GCP and use IAC for our Terraform. I've managed to build most of the initial organisation config in Terraform no problem with the exception of Cloud Identity and Organisation Policies. I'm using gcloud provided login credentials. Whenever I try to build for those two services I get this:
Error creating Group: googleapi: Error 403: Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the cloudidentity.googleapis.com. We recommend configuring the billing/quota_project setting in gcloud or using a service account through the auth/impersonate_service_account setting. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/. If you are getting this error with curl or similar tools, you may need to specify 'X-Goog-User-Project' HTTP header for quota and billing purposes. For more information regarding 'X-Goog-User-Project' header, please check https://cloud.google.com/apis/docs/system-parameters.
So in this case i'm using the Google Cloud SDK, so the error makes sense. However, the two options it presents don't work:
Setting a quota project makes no difference
I can't create a service account at the organisational level (and when I create one within a project it can't configure these organisational level constructs)
So how do I go about Terraforming these services?
Thanks.
Google docu says that workload identity could be used to authorise GKE pods to consume services provided by Google APIs (and it works fine). It also says that there would be one automatically created identity pool called PROJECT_ID.svc.id.goog.
Docu about workload identity federation says: "You can use a workload identity pool to organize and manage external identities."
After I configured workload identity as described here (and it works fine) I am trying to retrieve workload identity pools existing in my project and I expect to see PROJECT_ID.svc.id.goog. I am using this command to list pools: gcloud beta iam workload-identity-pools list --location="global" --show-deleted and as an output I get:
local#local:~/$ gcloud beta iam workload-identity-pools list --location="global"
Listed 0 items.
So are GKE workload identity pool and workload identity pools, from workload identity federation, simply two absolutely separate entities? Or it is just beta API which is not completely working atm? Or it is something third?
Finding correct name to product is sometime difficult. 2 very similar name for 2 different products. -> That's your mistake.
Workload Identity is a GKE Addon. Before going deeper, you have to know that, on Google Cloud Platform, you don't need to use service account key file because the service account are automatically loaded on every services (Compute Engine, App Engine, Cloud Run, Cloud Function, Cloud Build,...) and accessible through the metadata server . The Google Cloud client libraries automatically detect the environment and use the metadata server if present.
The problem with GKE is that you can run container on several different Compute Engine instances (the nodes) and your different service (K8S services) can have different level of authorization. If you rely on the Compute Engine service account (default behavior without the Workload identity addon), all the pods on the same instances use the same service account (and thus have the same permissions).
To solve that, the Workload Identity addon creates a proxy that intercept the metadata server calls and reply with the correct bind service account for this pods/service on GKE
Workload identity pool is totally different. The principle is to configure third party identity providers (such as AWS, Okta, or even custom) and to define the conditions to accept the third party token (email, claims,...).
When the token is accepted, you can perform a call to impersonate a service account, and thus generate a new token (a Google compliant one this time), that you will be able to use in subsequent calls.
The principle here is to avoid to use service account key file, and to rely on third party identity provider to interact with GCP. For example, on AWS you need to call BigQuery, you can create a token with Workload identity pool and your AWS identity and then call BigQuery without the need to exchanges secrets between the platforms.
Note: the best way to keep a secret secure is not to have secret!
My guess is that historically Google started with GKE Workload Identity as a GKE-specific feature a came to a generic approach titled Workload Identity Federation. Both approaches allow you to create access bindings for external identities but use slightly different syntax. Again, I guess they started with GKE and then came to more generic and flexible scheme.
Another platforms, like AWS or Azure do the same k8s magic with Workload Identity Federation feature.
I wanted to use a service account to manage VM instances on GCE remotely. It did not work. Therefore this question. One difference I found between a service account and a user account, after many hours of trial-error, is that there seems no way to use a service account to ssh in to a VM instance.
What are the other differences?
Links I found related to my question:
https://groups.google.com/forum/#!topic/gce-discussion/Z6OMpVhvowQ
Logging into google compute engine with a service account
The accepted answer is correct but lacks a deeper understanding of what credentials are in Google Cloud.
There are a number of types of credentials. User, Service Account, Group, Domain, etc. The difference is what the credentials represent and what is the authority for those credentials. Internally they are the same in regards to structure, content, etc.
All of the credential types are OAuth 2 credentials. A User Account credential is one that is issued by Google Accounts, G Suite or Identity Server. These credentials cannot be created by you or your software without Google. These credentials are issued by Google. Service Accounts, on the other hand, can create their own credentials as the service account contains the private key used to sign the credentials. The private key is used to create a Signed JWT that is then exchanged for OAuth Access Token, Refresh Token, and Identity Token. A Service Account is just a type of OAuth credential.
Google does not want User Credentials used to access Google Cloud resources except through the Google Cloud Console or Google Cloud SDK Tools and CLI. This is a security issue and design. Otherwise, the use and behavior are nearly identical except that some privileges cannot be assigned to some credential types.
By design, service accounts in Google are meant for non human users. They are a type of account used by resources in your google project (i.e. compute engine service account, app engine service account and etc).
Service accounts are designed as a type of account that is used by your Google cloud resources to communicate with other Google cloud services i.e. GCE to App Engine to Cloud Function or App Engine to google APIS like: vision API, speech to text...), or App Engine to Cloud SQL, etc.
You may refer to google documentation here for more details:
I am just starting to use GCP and I have some questions about the service accounts.
Say there is a team of like 4 remotely located developers and we all want to use the python API to access GCP to launch instances and run stuff on them. My question is should every user get their own service account and keys or should one service account be shared by all? What is the intended use case here?
Google Cloud Service Accounts provide both identity and authorization to Google Cloud.
They are similar to user accounts. If you would like to do auditing or logging of actions with service accounts, you will want to use separate service accounts per user.
Service accounts are typically used for software applications to authorize their actions with the Google Cloud APIs. Service Accounts are using to issue OAuth 2.0 Access Tokens and optionally OIDC Identity Tokens. These tokens are what provides your application with authorization in Google Cloud.
My question is should every user get their own service account and
keys or should one service account be shared by all?
Yes, you should issue separate service account JSON key files to each developer. In the same way that you would not share usernames and passwords for computer systems, you would not share service accounts.
I have written a bunch of articles on Google Cloud Service Accounts that might help you understand how to configure and use them:
Google Cloud Service Accounts