When submitting a Cloud Build run via gcloud builds submit ... I'm getting a forbidden error saying I don't have access to the bucket(s). There are 2 places where buckets are normally involved in submitting a Cloud Build, the staging and logs bucket. I specified the buckets for each as buckets (the same one, just different folders) that I do have access too so the command looks like this:
gcloud builds submit
--gcs-log-dir $my_bucket/logs
--gcs-source-staging-dir $my_bucket/source
The error I get is:
ERROR: (gcloud.builds.submit) 403: The user is forbidden from accessing the bucket [$my_bucket]: Please check your organization's policy.
I re-ran with --log-http and --verbosity debug and the expanded error shows the real reason:
DEBUG: https://storageapis.google.com "GET /storage/v1/b/$my_bucket?alt=json"
...
{
"error": {
"code": 403,
"message": "$user does not have serviceusage.services.use access to the Google Cloud Project."
}
}
I did some digging and see that's this error shows up when supplying a quota/billing project with the request (in addition to not having service consumer role). I confirmed this when inspecting the request's HTTP headers which included X-Goog-User-Project: $my_project.
What's weird is that I have access to objects in this bucket and can run gsutil/HTTP commands just fine which are using the same API endpoints with the difference being that gsutil doesn't include that user project in the request.
Is there a way to submit a build that doesn't include the project so that I don't need serviceusage.services.use permission? I tried unsetting the project in my gcloud config but it prompted me that I needed to either set it or pass it with --project flag.
edit: the bucket isn't "requester pays" enabled either which is why gsutil and client libraries work fine
The only reason why you are having this error is you have to enable your billing in order to build your bucket.
I have enabled it when I was trying the tutorial by clicking the "Create a Cloud Storage Bucket" under Getting started at the left side of your Dashboard. Just follow the instructions and you will see the "Enable Billing". Once you have enabled the Billing, you don't need to finish the Tutorial. Go back to your work and run the
$ gcloud build submit
and it's done!
I'm not sure you can run a cloud build without specifying a project. As far as I know, gcloud commands run within a project so it's needed.
If you want to use a different service account you can use service account impersonation adding --impersonate-service-account flag.
For this gcloud invocation, all API requests will be made as the given service account instead of the currently selected account.
According to the GCP documentation:
To run gcloud builds commands, users with only cloudbuild.builds.viewer or cloudbuild.builds.editor roles also require the serviceusage.services.use permission. To give this permission to the user, grant them the serviceusage.serviceUsageConsumer role.
Edit your user on IAM & Admin choosing your user and type "Service Usage Consumer".
However, review your policies and roles because I beliave that this option is for clean users created from the scratch without any other permissions than Object Storage roles.
Related
I use a workflow to create a batch job using a docker image hosted in a docker registry.
All of this happens within the same google cloud project.
My batch job fails with this error :
"docker: Error response from daemon: Head "https://us-west1-docker.pkg.dev/v2/entity/docker-registry/image-name/manifests/latest": denied: Permission "artifactregistry.repositories.downloadArtifacts" denied on resource "projects/project-id/locations/us-west1/repositories/docker-registry" (or it may not exist).
See 'docker run --help'.
From google documentation I understand that Compute Engine's service account doesn't have the roles/artifactregistry.admin : Jobs default to using the Compute Engine default service account
I get the same error after giving the role to the service account :
gcloud projects add-iam-policy-binding project-id \
--member=serviceAccount:compute#developer.gserviceaccount.com \
--role=roles/artifactregistry.admin
While digging service accounts I found another service another service account and also gave it the role : service-xxxx#gcp-sa-cloudbatch.iam.gserviceaccount.com.
It does not solve the problem.
How can I see which service account is used ?
Can I see logs about denied permissions ?
The error occurs when you are trying to push an image on a repository in which a specific hostname associated with its repository location is not yet authenticated and specified in the credential helper.You may refer to this Setting up authentication for Docker .You may check and confirm the service account to make sure you are still impersonating the correct one ,run below as mentioned in document
gcloud auth list
This command will show the active account, along with the other
accounts that are authorized to access your Google Cloud project. The
active account will be marked with an asterisk (*).
Try to run the authentication using a command specifying the location of your repository.You may try to run the configure-docker command against the auth group and see.
gcloud auth configure-docker <location>-docker.pkg.dev
And then try pulling the Docker image again.
Refer Authenticating to a repository for more information and you can see these logs permission denied logs in Cloud logging for more details.
I was trying to integrate gcp cloud build webhook with my gitlab repository. I was using this as reference. I successfully created the trigger(webhook trigger). Also added the necessary permission to the service-accounts and enabled the API as well.
But when I'm trying to use the command like below to trigger the cloud build getting error PERMISSION_DENIED
Used command:
curl -X POST -H "application/json" "https://cloudbuild.googleapis.com/v1/projects/${PROJECT_NAME}/triggers/${TRIGGER_NAME}:webhook?key=${API_KEY}&secret=${SECRET_VALUE}" -d "{}"
NOTE: I've used real value for TRIGGER_NAME, API_KEY and SECRET_VALUE
Getting error like:
{
"error": {
"code": 403,
"message": "The caller does not have permission",
"status": "PERMISSION_DENIED"
}
}
Settings of cloud build:
below is the screenshot of cloud build settings
How could I invoke this webhook trigger successfully?
Hi make sure your service account or role has the following permissions:
IAM Permission
cloudbuild.builds.create
Or Using Predefined Role
Cloud Build Editor
See here for more info: https://cloud.google.com/build/docs/iam-roles-permissions
I also encountered the same symptom as you do when following the tutorial doc. I resolved this issue by providing a different secret value (e.g. 123) when creating my trigger instead of my ssh-private-key, stored in the secret manager.
The specified secret key would be used as part of the Webhook URL parameter. Since ssh-private-key is way too complicated to have the correct match, using an simpler key could have the match much easier. (I do think the document could be more clear)
First, make sure billing is enabled. Also, double check the keys.
Second, make sure you assign the Secret Manager Secret Accessor role to your Cloud Build service account, ${PROJECT_NUMBER}#gcp-sa-cloudbuild.iam.gserviceaccount.com.
Third, try making the conditions for the trigger something basic and trigger it to see if the build works.
When I am authenticating using service-account.json I am not able to access any projects, i.e. the command 'gcloud projects list' results in
Listed 0 items.
Setting gcloud project using the command 'gcloud config set project my-project' results in:
WARNING: You do not appear to have access to project [my-project] or it does not exist.
But this project is present in my service-account.json
I am trying to get my GKE cluster credentials to create a kubeconfig entry.
'gcloud container clusters get-credentials my-cluster --zone=us-central1-a'
ERROR: (gcloud.container.clusters.get-credentials) ResponseError: code=403, message=Required "container.clusters.get" permission(s) for "projects/my-project/zones/us-central1-a/clusters/my -cluster".
When I log in to GCP using gcloud init and logging in with user account all these steps are giving successful results. But how should I authorise user account without opening browser. I cannot use '--console-only' flag as it still requires to copy-paste url and opening browser. I want to automate the full process of authenticating to user account by only using scripts
NOTE: The my-cluster GKE cluster was created under my-project and was created by me.
I suspect (!?) your service account has insufficient permissions (granted through roles).
To list projects (gcloud projects list), the service account must have (at least) resourcemanager.projects.get on (each of the projects) in the list. See:
https://cloud.google.com/resource-manager/docs/access-control-proj#permissions_and_roles
To retrieve cluster permissions, it must have container.clusters.get (see full list).
A good role that includes both is roles/container.clusterAdmin)
Service Accounts are challenging. They are both resources (particles) and identities (waves) and behave differently depending on context.
As resources, Service Accounts are created in (and owned) by a Project and may be granted permissions in any project (not just the owner).
As identities, Service Accounts may be granted roles that aggregate permissions to one of more methods. It is this step that I think you may be missing.
Google's IAM documentation is decent.
I setup a Cloud Build Trigger in my GCP project in order to deploy a Cloud Function from a Cloud Source Repository via a .yaml file. Everything seems to have been setup correctly and permissions granted according to the official documentation, but when I test the trigger by running it manually, I get the following error:
ERROR: (gcloud.functions.deploy) ResponseError: status=[403], code=[Forbidden], message=[Missing necessary permission iam.serviceAccounts.actAs for on resource [MY_SERVICE_ACCOUNT]. Please grant the roles/iam.serviceAccountUser role. You can do that by running 'gcloud iam service-accounts add-iam-policy-binding [MY_SERVICE_ACCOUNT] --member= --role=roles/iam.serviceAccountUser']
Now first of all, running the suggested command doesn't even work because the suggested syntax is bad (missing a value for "member="). But more importantly, I already added that role to the service account the error message is complaining about. I tried removing it, adding it back, both from the UI and the CLI, and still this error always shows.
Why?
I figured it out after a lot of trial and error. The documentation seems to be incorrect (missing some additional necessary permissions). I used this answer to get me there.
In short, you also need to add the cloudfunctions.developer and iam.serviceAccountUser roles to the [PROJECT_NUMBER]#cloudbuild.gserviceaccount.com account, and (I believe) that the aforementioned cloudbuild service account also needs to be added as a member of the service account that has permissions to deploy your Cloud Function (again shown in the linked SO answer).
The documentation really should be reflecting this.
Good luck!
I want to compare Google Cloud Run to both Google App Engine and Google Cloud Functions. The Cloud Run Quickstart: Build and Deploy seems like a good starting point.
My Application Default Credentials are too broad to use during development. I'd like to use a service account, but I struggle to configure one that can complete the quickstart without error.
The question:
What is the least privileged set of predefined roles I can assign to a service account that must execute these commands without errors:
gcloud builds submit --tag gcr.io/{PROJECT-ID}/helloworld
gcloud beta run deploy --image gcr.io/{PROJECT-ID}/helloworld
The first command fails with a (seemingly spurious) error when run via a service account with two roles: Cloud Build Service Account and Cloud Run Admin. I haven't run the second command.
Edit: the error is not spurious. The command builds the image and copies it to the project's container registry, then fails to print the build log to the console (insufficient permissions).
Edit: I ran the second command. It fails with Permission 'iam.serviceaccounts.actAs' denied on {service-account}. I could resolve this by assigning the Service Account User role. But that allows the deploy command to act as the project's runtime service account, which has the Editor role by default. Creating a service account with (effectively) both Viewer and Editor roles isn't much better than using my Application Default Credentials.
So I should change the runtime service account permissions. The Cloud Run Service Identity docs have this to say about least privileged access configuration:
This changes the permissions for all services in a project, as well
as Compute Engine and Google Kubernetes Engine instances. Therefore,
the minimum set of permissions must contain the permissions required
for Cloud Run, Compute Engine, and Google Kubernetes Engine in a
project.
Unfortunately, the docs don't say what those permissions are or which set of predefined roles covers them.
What I've done so far:
Use the dev console to create a new GCP project
Use the dev console to create a new service account with the Cloud Run Admin role
Use the dev console to create (and download) a key for the service account
Create (and activate) a gcloud configuration for the project
$ gcloud config list
[core]
account = {service-account-name}#{project-id}.iam.gserviceaccount.com
disable_usage_reporting = True
project = {project-id}
[run]
region = us-central1
Activate the service account using the downloaded key
Use the dev console to enable the Cloud Run API
Use the dev console to enable Container Registry→Settings→Container Analysis API
Create a sample application and Dockerfile as instructed by the quickstart documentation
Run gcloud builds submit --tag gcr.io/[PROJECT-ID]/helloworld
...fails due to missing cloud build permissions
Add the Cloud Build Editor role to service account and resubmit build
...fails due to missing storage permissions. I didn't pay careful attention to what was missing.
Add the Storage Object Admin role to service account and resubmit build
...fails due to missing storage bucket permissions
Replace service account's Storage Object Admin role with the Storage Admin role and resubmit build
...fails with
Error: (gcloud.builds.submit) HTTPError 403:
<?xml version='1.0' encoding='UTF-8'?>
<Error>
<Code>AccessDenied</Code>
<Message>Access denied.</Message>
<Details>
{service-account-name} does not have storage.objects.get access to
{number}.cloudbuild-logs.googleusercontent.com/log-{uuid}.txt.</Details>
</Error>
Examine the set of available roles and the project's automatically created service accounts. Realize that the Cloud Build Service Account role has many more permissions that the Cloud Build Editor. This surprised me; the legacy Editor role has "Edit access to all resources".
Remove the Cloud Build Editor and Storage Admin roles from service account
Add the Cloud Build Service Account role to service account and resubmit build
...fails with the same HTTP 403 error (missing get access for a log file)
Check Cloud Build→History in the dev console; find successful builds!
Check Container Registry→Images in the dev console; find images!
At this point I think I could finish Google Cloud Run Quickstart: Build and Deploy. But I don't want to proceed with (seemingly spurious) error messages in my build process.
Cloud Run PM here:
We can break this down into the two sets of permissions needed:
# build a container image
gcloud builds submit --tag gcr.io/{PROJECT_ID}/helloworld
You'll need:
Cloud Build Editor and Cloud Build Viewer (as per #wlhee)
# deploy a container image
gcloud beta run deploy --image gcr.io/{PROJECT_ID}/helloworld
You need to do two things:
Grant your service account the Cloud Run Deployer role (if you want to change the IAM policy, say to deploy the service publicly, you'll need Cloud Run Admin).
Follow the Additional Deployment Instructions to grant that service account the ability to deploy your service account
#1
gcloud projects add-iam-policy-binding PROJECT_ID \
--member="serviceAccount:{service-account-name}#{project-id}.iam.gserviceaccount.com" \
--role="roles/run.developer"
#2
gcloud iam service-accounts add-iam-policy-binding \
PROJECT_NUMBER-compute#developer.gserviceaccount.com \
--member="serviceAccount:{service-account-name}#{project-id}.iam.gserviceaccount.com" \
--role="roles/iam.serviceAccountUser"
EDIT: As noted, the latter grants your service account the ability to actAs the runtime service account. What role this service account has is dependent on what it needs to access: if the only thing Run/GKE/GCE accesses is GCS, then give it something like Storage Object Viewer instead of Editor. We are also working on per-service identities, so you can create a service account and "override" the default with something that has least-privilege.
According to https://cloud.google.com/cloud-build/docs/securing-builds/set-service-account-permissions
"Cloud Build Service Account" - Cloud Build executes your builds using a service account, a special Google account that executes builds on your behalf.
In order to call
gcloud builds submit --tag gcr.io/path
Edit:
Please "Cloud Build Editor" and "Viewer" your service account that starts the build, it's due to the current Cloud Build authorization model.
Sorry for the inconvenience.