TL;DR
Can anyone suggest a way that I can run tests as a named user that isn't a service account within a CI pipeline?
Detail:
We provide an application running on GCP, the stakeholders of which are all users internal to our organisation. Those users' identities are mastered in our on-premises ADFS implementation and are synced to GCP so that they can be used for authentication.
We authorize access to various parts of the application using ADFS groups that those identities can be added to and again, we sync those group memberships to GCP. The groups are available as Google Groups (https://groups.google.com). At runtime we check whether the authenticated user is a member of a given group. This all works fine.
I would like to run tests in our CI pipeline that ensure that an authenticated user can do the things they're allowed to do and can't do the things that they're not allowed to do. Here are some of the tests I would like to run:
* Ensure gsutil ls allowed-bucket returns an expected list of blobs
* Ensure gsutil ls non-allowed-bucket cannot access the named bucket
* Ensure a cloud function can be called via gcloud functions call function-name
Thus I need to programatically (i.e. without human intervention) authenticate as a user that is a member of a given Google group. I came up with two possibilities:
Using a service account and authenticating using gcloud auth activate-service-account --key-file /path/to/keyfile.json but I cannot find a way to add a service account to the appropriate Google Group - so that option won't work
Preparing a non-privileged user in our on-premises ADFS that we then add to the appropriate Google Group. Unfortunately I don't think there's a way to authenticate as that user without human intervention because gcloud auth login uses a web-based authorization workflow. If it requires human intervention then I can't use it in a CI pipeline.
Related
I am the owner of a google project, and also one of the developers. At times I would like to give my local machine access to run code for various configuration, testing, and maintenance operations via google client libraries on my production environment, e.g.:
from google.cloud import storage
storage.Client()
...
There are two ways I've been doing this:
gcloud auth application-default login and via GOOGLE_APPLICATION_CREDENTIALS by creating a service account and then downloading its json private key and setting the environment variable.
Both make me nervous about accidentally running code that could damage my production environment.
I'm not sure how to give myself least privileges when I'm also the project Owner, and how to carefully turn on/off privileges. Perhaps I shouldn't be doing anything on Production from my local machine, and only running code in cloud instances that are more controlled?
What do people typically do to manage both development and production google projects? I'm leaning towards creating a service account that I manually add/remove from my production IAM as needed, but I've read that the use of service accounts and local private key json files is also risky.
Are there ways to use OAuth that use my personal credentials but restrict scope/access for a specific session?
I assume you don't have access to Google Workspace.
YOu could create a(nother) Google user (commercial) account solely for project ownership?
NOTE You needn't get another Gmail address. The signup flow includes the option to use an existing email address. In this case, your#not-gmail.com gets wrapped with a Google account (and distinct password|2FA).
Service Account Keys carry risk because they're bearer credentials and so you need to be judicious with their management. It's good practice to only create keys when you must and to delete|cycle them promptly. However, a Service Account Key is generally (!) more secure than using gcloud auth application-default because generally (!) Service Accounts are granted fewer permissions than user (e.g. Owner) accounts. See Workload Identity Federation for another approach.
It used to be that Service Accounts were not fully interchangeable with User accounts. Service Accounts used to not be permitted to be Project Owners (this is no longer true). I think there are still cases where Service Accounts are not equivalent (when GCP requires sending e.g. terms of service emails).
I have a service account with Owner permissions on the project (Just for testing this out)
Still I am not able to create API Keys using that service account via gcloud. It says "Permission Denied"
I am using the following commands.
1.
gcloud auth activate-service-account <Service-account>#<project-id>.iam.gserviceaccount.com --key-file=<key-file>.json
2.
gcloud auth list //Gives the service account name
3.
gcloud alpha services api-keys create --display-name=dummy
The above command works if I authenticate as a normal user with Owner permission but with service account it doesn't seems to work. Am I missing something ? Please help.
The APIKEY Api has a strange history. Relesed in Beta about 1 years ago, and now go back to Alpha. There is no public documentation (in reality it has been removed) and if you know this API, you have found it on SO or on old tutorial.
Anyway, just to say that it's not a reliable API and if you want to automate stuff on it (with call with a service account) it's not a good idea. In addition, sometime, APIs don't allow service account call but require user credentials. It was the case previously with the quota APIs, but it has been updated recently (this summer 2020).
Eventually, Google Cloud don't recommend to use APIKEY for security reason (we can discuss this more if you want). And thus, I don't think it is in its (security and best practice) strategy to promote an API that allows APIKEY automation.
I have a CLI tool that interacts with Google KMS. In order for it to work, I fetch the user credentials as a JSON file which is stored on disk. Now a new requirement came along. I need to make a web app out of this CLI tool. The web app will be protected via Google Cloud IAP. Question is, how do I run the CLI tool on behalf of the authenticated user?
You don't. Better use a service-account and assign the required role. That service account still could have domain-wide delegation of rights (able to impersonate just any user, which is known).
Running CLI tools from a web-application probably also could/should be avoided. Iit might be better to convert his CLI tool into a Cloud Function and then call it via HTTP trigger, from within the web-application (so that access to the service account is limited as far as possible).
This might also be something to reconsider, security-wise:
I fetch the user credentials as a JSON file which is stored on disk.
Even if it might have been required, with a service-account it wouldn't.
Using Google Cloud, there exists a BigQuery View table that queries two projects.
However, on the project where the view is located, we wish to run a query against it from Airflow/Composer. Currently it fails with a 403.
AFAIK it will use the default composer service account - however it doesn't have access to the 2nd project used in the sql of the view.
How do I give composer's service account access to the second project?
Think about a service account like a user account: you have a user email that you authorize on different project and component. Exactly the same thing with the service account email.
The service account belongs to a project. An user account belongs to a domain name/organisation. No real difference at the end.
So, you can use a service account email like any user accounts:
Grant authorization in any project
Add it in Google Groups
Even grant it viewer or editor role on GSuite document (Sheet, Docs, Slides,...) to allow it to access and to read/update these document!! Like any users!
EDIT
With Airflow, you can defined connexions and a default connexion. You can use this connexion in your DAG and thus use the service account that you want.
I think you have to add the service account into project IAM.
I have a service running in GCE with default service account A. This service uses gcloud to talk to various GCP services. Currently, it uses service account B to talk to some of the GCP services (using private key). However, we want to get rid of using private key and use account impersonation. To do that, I have added account A to the service account B's role and given token creator role.
I wrote a test program in go and was able to verify the impersonation works. However, our service is in PHP, and uses gcloud SDK. I couldn't find a way to configure gcloud to impersonate a service account or provide custom token.
One option is that I rewrite all the gcloud code to use google SDK, but that is lots of work, and I'd rather avoid that. My question is, how do I invoke gcloud using service account B in this scenario?. Is there a way to pass access token to gcloud or specify impersonation user?
gcloud has a --impersonate-service-account flag for this.
Run gcloud auth login and login using your gcp email address.
Run export GOOGLE_OAUTH_ACCESS_TOKEN=$(gcloud auth print-access-token --impersonate-service-account=<sa-name>.iam.gserviceaccount.com) command to generate temporary credentials from the service account and store as the current OAUTH token. (this OAUTH token only lasts for 1 hour so make sure the operation using this token can complete within that time frame.