Unable to create Composer environment [GCP] - google-cloud-platform

I am trying to create basic Composer environment:
image version: 1.17.8/2.1.4
using service account with composer.worker permission
my own user has project.owner permission
public ip
All my attempts failed with following error:
Http error status code: 400
Http error message: BAD REQUEST
Errors in: [Web server]; Error messages:
The caller does not have permission
Required 'deploymentmanager.typeProviders.create' permission for 'projects/<my-project>/global/typeProviders/europe-west2-<name-id>-addons-gke-typer'
deploymentmanager.typeProviders.create is covered by Deployment Manager Type Editor, so I added this permission to both my account and service account, but the error remains the same.
Cloud Composer Service Agent account is present in the project without any modifications to its permissions.
Is there anything else I can check or something that I missed during the set up?

For an account (whether User Account or Service Account) to be able to create a Composer Environment, the account must have a composer.environments.create permission.
And according to Google Cloud's documentation on Cloud Composer Access Control,
The Composer Worker role provides the permissions necessary to run a Cloud Composer environment
VM and intended for service accounts.
The Composer Worker role is not intended for creation of environments thus, it does not have the composer.environments.create permission.
If you want your service account to be able to create a Composer environment, you will need to assign the role Composer Administrator and this has the composer.environments.create permission needed.
You may refer to Access Control for Cloud Composer for the complete list of permission for Composer Worker, Composer Administrator and other Composer related roles.

Related

Batch cannot pull docker image from Artifact Registry

I use a workflow to create a batch job using a docker image hosted in a docker registry.
All of this happens within the same google cloud project.
My batch job fails with this error :
"docker: Error response from daemon: Head "https://us-west1-docker.pkg.dev/v2/entity/docker-registry/image-name/manifests/latest": denied: Permission "artifactregistry.repositories.downloadArtifacts" denied on resource "projects/project-id/locations/us-west1/repositories/docker-registry" (or it may not exist).
See 'docker run --help'.
From google documentation I understand that Compute Engine's service account doesn't have the roles/artifactregistry.admin : Jobs default to using the Compute Engine default service account
I get the same error after giving the role to the service account :
gcloud projects add-iam-policy-binding project-id \
--member=serviceAccount:compute#developer.gserviceaccount.com \
--role=roles/artifactregistry.admin
While digging service accounts I found another service another service account and also gave it the role : service-xxxx#gcp-sa-cloudbatch.iam.gserviceaccount.com.
It does not solve the problem.
How can I see which service account is used ?
Can I see logs about denied permissions ?
The error occurs when you are trying to push an image on a repository in which a specific hostname associated with its repository location is not yet authenticated and specified in the credential helper.You may refer to this Setting up authentication for Docker .You may check and confirm the service account to make sure you are still impersonating the correct one ,run below as mentioned in document
gcloud auth list
This command will show the active account, along with the other
accounts that are authorized to access your Google Cloud project. The
active account will be marked with an asterisk (*).
Try to run the authentication using a command specifying the location of your repository.You may try to run the configure-docker command against the auth group and see.
gcloud auth configure-docker <location>-docker.pkg.dev
And then try pulling the Docker image again.
Refer Authenticating to a repository for more information and you can see these logs permission denied logs in Cloud logging for more details.

How to stop running my terraform commands with a service account

gcloud auth list
Credentialed Accounts ACTIVE ACCOUNT
* xxxx#xxxx.com
To set the active account, run:
$ gcloud config set account `ACCOUNT
The credentials used here are correct and make sense, but when I run it HOPING it to be that user account, the error is saying that this service account does not have permissions needed in this environment. Which makes total sense! but! I can't figure out how to stop terraform from running as this service account and not as my own active user account.
Error: Error loading state: Failed to open state file at
gs://dxxxxdefault.tfstate: googleapi: got HTTP response code 403 with
body: AccessDeniedAccess
denied.terraform-win#xxxx.iam.gserviceaccount.com
does not have storage.objects.get access to the Google Cloud Storage
object.
The error message said that your service account terraform-win#xxxx.iam.gserviceaccount.com cannot access terraform state on Google Cloud Storage, you just need to add storage.objects.get permission to the service account to fix the error.
If you don't want Terraform to use this service account, you can remove impersonation settings in Terraform and environment variable GOOGLE_APPLICATION_CREDENTIALS.

Permissions required scale up environment composer 2?

I already have an environment running, but want to scale up the machine sizes. But keep getting a very similar error to this:
UPDATE operation on this environment failed 3 minutes ago with the
following error message:
Composer Backend timed out. Currently running tasks are [stage:
CP_COMPOSER_AGENT_RUNNING
description: "No agent response published."
response_timestamp {
seconds: 1618203503
nanos: 291000000
}
].
The gcp docs say that the service account doesn't have required permissions. I check required permissions also in the docs. My personal account and service already have the role Environment and Storage Object Administrator. And the service account already has the role Cloud Composer v2 API Service Agent Extension. I don't understand which permission am I missing.
Service account that the composer uses has roles:
Cloud Composer v2 API Service Agent Extension
Editor
Environment and Storage Object Administrator
Service Account User
And my personal account has roles:
Editor
Environment and Storage Object Administrator
Service Account User
Storage Admin
Viewer

google cloud run Failed to trigger build: Permission 'cloudbuild.builds.create' denied

I am trying to setup a trigger to deploy a repository (synced from Bitbucket), to Cloud Run when master branch is pushed.
The trigger is giving me the following warning:
Failed to trigger build: Permission 'cloudbuild.builds.create' denied on resource
After looking through GCP documentation I tried adding the Cloud Build Editor role to the service account that runs the trigger (https://cloud.google.com/build/docs/iam-roles-permissions), but I am still getting the same error.
Is there something aI am missing
Build triggers use the Cloud Build service account to create a build. The error above indicates that the Cloud Build service account is missing the cloudbuild.builds.create IAM permission, which is required for the service account to run a build trigger.
According to GCP documentation:
You can resolve this error by granting the Cloud Build Service Account IAM role to your default Cloud Build service account (ex. [PROJECT_NUMBER]#cloudbuild.gserviceaccount.com).
This also applies if you specified your own service account. It must have the Cloud Build Service Account role.
For instructions on granting this role, see Configuring access for Cloud Build service account.

ERROR: (gcloud.composer.environments.update) Failed to impersonate when terraform runs impersonating as a second account

I am getting the following error (Please see below) when I run my terraform apply.
I am running Terraform 12.x.
GCP Cloud Build runs in a different project other than project-abcd (where these accounts are)
My terraform code tries execute a gcloud command in a GCP cloud build container. It does so by impersonating as composer-bq-sa#prj-abcd.iam.gserviceaccount.com
The service account that terraform runs as is:
terraform_service_account = "org-terraform#abcd.iam.gserviceaccount.com"
(before impersonating)
This IAM account (org-terraform#abcd.iam.gserviceaccount.com) (NOT service account) has the following role bindings (TOTAL 9):
(There is no Service Account with that email)
Composer Administrator
Compute Network Admin
Service Account Token Creator
Owner
Access Context Manager Admin
Security Admin
Service Account Admin
Logs Configuration Writer
Security Center Notification Configurations Editor
The service account (composer-bq-sa#prj-abcd.iam.gserviceaccount.com) has as one of its members: org-terraform#abcd.iam.gserviceaccount.com
When I look at the screen titled "Members with access to this service account" and look at org-terraform#abcd.iam.gserviceaccount.com , I see that it has the following role-bindings (ONLY 4):
Service Account Token Creator
Owner
Security Admin
Service Account Admin
Why am I getting the error below even though IAM account has apparently the right roles and it is one of the members of the service account it is impersonating as?
ERROR
module.gcloud_composer_bucket_env_var.null_resource.run_command[0] (local-exec): WARNING: This command
is using service account impersonation. All API calls will be executed as [**composer-bq-sa#prj-abcd.iam.gserviceaccount.com**].
module.gcloud_composer_bucket_env_var.null_resource.run_command[0] (local-exec): ERROR:
(gcloud.composer.environments.update) Failed to impersonate [**composer-bq-sa#prj-abcd.iam.gserviceaccount.com**]. Make sure the account that's trying to impersonate it has access to the service account itself and the "roles/iam.serviceAccountTokenCreator" role.
Recapping:
In order to grant user permission to impersonate a Service Account follow instructions listed in this document.
Depending on the use case, you may grant user following roles:
roles/iam.serviceAccountUser
roles/iam.serviceAccountTokenCreator
roles/iam.workloadIdentityUser