Terraform on GCP fails to create pubsub topic stating permission denied - google-cloud-platform

Created a service user to manage terraform under the project and gave it roles/owner. Created the key for this terraform user.
Terraform code is simple:
resource "google_pubsub_topic" "my_topic" {
name = "my_topic"
project = "${var.project_id}"
}
just creating a pub-sub topic.
terraform plan works but terraform apply gives me:
google_pubsub_topic.my_topic: googleapi: Error 403: User not authorized to perform this action., forbidden
I have even tried giving the service account roles/pubsub.admin not sure I understand what's going on because my service account has the owner role associated with it yet it can't create a pubsub topic.
Can anybody help me figure this out please?
Thanks in advance.
Edit (to expand on my comment about what worked):
I had created the service account using gcloud.
gcloud iam service-accounts create terraform \
--display-name "Terraform admin account"
gcloud projects add-iam-policy-binding myproject-182220 \
--member serviceAccount:terraform#myproject-182220.iam.gserviceaccount.com \
--role roles/owner
gcloud iam service-accounts keys create terraform-admin.json \
--iam-account terraform#myproject-182220.iam.gserviceaccount.com
To continue my debugging, I created a new service account using the console / GUI --> API & Services --> Credentials --> Create Credentials --> Service Account Key --> New Service Account (With the owner role). With this new service key json file, I was able to run my terraform code without a problem.
Now my confusion is, why did this work but not when I used gcloud to create a service account and give it the same role?

You have to enable the pub/sub api via google cloud or it might be possible using terraform.

The sample usage configuration for the Google provider looks more like:
// Configure the Google Cloud provider
provider "google" {
credentials = "${file("account.json")}"
project = "my-gce-project"
region = "us-central1"
}
// Create a new instance
resource "google_compute_instance" "default" {
# ...
}
This can be read in more detail on the "Google Cloud Provider" documentation page from Terraform.

Related

Can't use GCP IAM API with a service account

I am trying to create a new service account with terraform code. This code works well when I run it on behalf of my user credentials (the project Owner). But I see an error when I run the code on behalf of terraform service account
module "service_accounts" {
source = "terraform-google-modules/service-accounts/google"
version = "~> 4.1"
project_id = var.project
prefix = var.env
names = ["dataproc-sa"]
project_roles = [
"${var.project}=>roles/cloudkms.cryptoKeyEncrypterDecrypter",
"${var.project}=>roles/storage.objectViewer",
"${var.project}=>roles/dataproc.worker",
]
}
The terraform-google-modules/service-accounts/google module documentation says
Service account or user credentials with the following roles must be used to provision the resources of this module:
* Service Account Admin: roles/iam.serviceAccountAdmin
* (optional) Service Account Key Admin: roles/iam.serviceAccountKeyAdmin when generate_keys is set to true
* (optional) roles needed to grant optional IAM roles at the project or organizational level
I've bound roles/iam.serviceAccountAdmin to the gitlab-terraform#xxxxxx.iam.gserviceaccount.com service account, despite this I see the next error
Error: Error creating service account: googleapi: Error 403:
Identity and Access Management (IAM) API has not been used in project xxxxxxxxx
before or it is disabled. Enable it by visiting
https://console.developers.google.com/apis/api/iam.googleapis.com/overview?project=xxxxxxxxx
then retry. If you enabled this API recently, wait a few minutes for
the action to propagate to our systems and retry.
It looks like the terraform service account doesn't have the appropriate permissions to create a new service account
What should I do to allow the service account create another one? What kind of permissions are missed? The code works well if I run it with the project Owner role (but I don't want to do this)
EDIT: Forgot to mention that I tried to bind roles/iam.admin to the terraform service account but it gave me the next error
ERROR: Policy modification failed. For a binding with condition,
run "gcloud alpha iam policies lint-condition" to identify
issues in condition.
ERROR: (gcloud.projects.add-iam-policy-binding)
INVALID_ARGUMENT: Role roles/iam.admin is not supported for this resource.
The problem is the IAM service is not enabled. To enable that service you need the IAM role Service Usage Admin aka roles/serviceusage.serviceUsageAdmin. The Owner and Editor roles also have the permission serviceusage.services.enable.
You can enable the service using the CLI:
gcloud services enable iam.googleapis.com
Or add this to your Terraform HCL to enable the IAM service.
resource "google_project_service" "iam_service" {
project = var.project
service = "iam.googleapis.com"
}
You should also add a depends_on to resources that depend on the service being enabled:
depends_on = [
google_project_service.iam_service
]

Error Deploying Cloud Function from gitlab

I am trying to deploy a cloud function via gitlab using a new service account (Not using default service account). It has the cloud functions developer role but it is still failing with below error:
The error below includes a user as cloud-functions-mixer. I haven't configured anything like that in my repo and not sure why it is coming up.
First of all, running the suggested command doesn't even work because the suggested syntax is bad . I have tried running the below command but it’s not right
Error: googleapi: Error 403: Missing necessary permission iam.serviceAccounts.actAs for cloud-functions-mixer on the service account project-test-tf-02#appspot.gserviceaccount.com.
Grant the role 'roles/iam.serviceAccountUser' to cloud-functions-mixer on the service account project-test-tf-02#appspot.gserviceaccount.com.
You can do that by running 'gcloud iam service-accounts add-iam-policy-binding project-test-tf-02#appspot.gserviceaccount.com --member=cloud-functions-mixer --role=roles/iam.serviceAccountUser'.
Google's instructions about the cloud-functions-mixer are wrong. What you actually need to do is replace the string cloud-functions-mixer with the name of the service account that is building or deploying your function.
The following user-defined service accounts will be used in an example:
my-cloud-function#my-project.iam.gserviceaccount.com is the service account that your function runs as.
build-service-account#my-project.iam.gserviceaccount.com is the service account that builds/deploys your Cloud Function
The command to run is:
gcloud iam service-accounts add-iam-policy-binding
my-cloud-function#my-project.iam.gserviceaccount.com
--member=serviceAccount:build-service-account#my-project.iam.gserviceaccount.com
--role=roles/iam.serviceAccountUser
Docs
Or, in Terraform, you would need a resource like this:
resource "google_service_account_iam_member" "opentok_webhook_mixer" {
service_account_id = google_service_account.my_cloud_function.id
role = "roles/iam.serviceAccountUser"
member = "serviceAccount:${google_service_account.build_service_account.email}"
}
You'll have to update the names of the service account resources.
This approach also works for Google Cloud Build.

How can I give a service account access to a particular secret?

I want to grant a service account access to a secret in Google Secrets Manager.
I can access the secret like this:
gcloud beta secrets versions access 1 --secret="thesecret" --project="myproject"
But when my service account tries the same command, gcloud emits this error:
ERROR: (gcloud.beta.secrets.versions.access) PERMISSION_DENIED: Request had insufficient authentication scopes.
The main question is: What else do I need to do to ensure that the service account can access the secret?
I have granted that service account "roles/secretmanager.secretAccessor" in Terraform like this:
resource google_project_iam_binding the-binding {
project = myproject
role = "roles/secretmanager.secretAccessor"
members = [
"serviceAccount:theserviceaccount#someotherproject.iam.gserviceaccount.com",
]
}
And I can verify that it has that role both in the gcp console and like this:
gcloud projects get-iam-policy myproject \
--flatten="bindings[].members" \
--format='table(bindings.role)' \
--filter="bindings.members:theserviceaccount#someotherproject.iam.gserviceaccount.com"
ROLE
roles/secretmanager.secretAccessor
But there's this concept from the docs:
If a member only needs to access a single secret's value, don't grant that member the ability to access all secrets. For example, you can grant a service account the Secret Accessor role (roles/secretmanager.secretAccessor) on a single secret.
So it's like an iam-policy-binding can have an affinity to a particular secret, but I'm not sure which gcloud commands or terraform resources I can use to create such an affinity.
The first problem is that I was mistaken about which service account my environment was configured to use. So I had granted access to the service account, but I wasn't using it after all (apparently they're initialized inconsistently in my case). I fixed that by running this command before trying to access the secret:
gcloud config set account theserviceaccount#someotherproject.iam.gserviceaccount.com
Also, I didn't realize that there were more than one toplevel gcloud command that let you modify iam policy bindings. I had been exploring gcloud iam ... when what I needed was:
gcloud beta secrets add-iam-policy-binding projects/myproject/secrets/mysecret --member serviceAccount:theserviceaccount#someotherproject.iam.gserviceaccount.com --role roles/secretmanager.secretAccessor

GCP : Unable to create a k8s cluster with a custom service account

I created a specific service account in GCP for provisioning clusters in my project :
gcloud iam service-accounts create [sa_name]
gcloud projects add-iam-policy-binding \
[project_id] \
--role=roles/container.admin \
--member=serviceAccount:[sa_name]#[project_id].iam.gserviceaccount.com
gcloud iam service-accounts keys create [keyfile_name] \
--iam-account=[sa_name]#[project_id].iam.gserviceaccount.com
gcloud auth activate-service-account --key-file=[keyfile_name]
When I run the command gcloud container clusters create [cluster_name]
I always get :
ERROR: (gcloud.container.clusters.create) ResponseError: code=403,
message=Required "container.clusters.create" permission(s) for
"projects/context-platform-staging". See
https://cloud.google.com/kubernetes-engine/docs/troubleshooting#gke_service_account_deleted
for more info.
As you can see, I use roles/container.admin but I even tried to apply the roles/editor and roles/owner to this service account, same behavior.
The only way this command works so far is to use my main google owner account (not a service account).
What am I missing here ?
From the error message, I understood that the service account does not have the permission "container.clusters.create".
Please add the "Container Engine Cluster Admin" and also "Container Engine Admin" roles on the service account that the cluster is being created with:.
To create a cluster, you need both "container.clusters.create" permission on the project. You also need to assign the role “roles/iam.serviceAccountUser” to the user who will use the service account. In this way, the user can access GKE's service account.
For more information and in-depth tutorial, please refer to this article in the GCP documentation.

gcloud: The user does not have access to service account "default"

I attempting to use an activated service account scoped to create and delete gcloud container clusters (k8s clusters), using the following commands:
gcloud config configurations create my-svc-account \
--no-activate \
--project myProject
gcloud auth activate-service-account my-svc-account#my-project.iam.gserviceaccount.com \
--key-file=/path/to/keyfile.json \
--configuration my-svc-account
gcloud container clusters create a-new-cluster \
--configuration my-svc-account \
--project= my-project
--zone "my-zone"
I always receive the error:
...ERROR: (gcloud.container.clusters.create) ResponseError: code=400, message=The user does not have access to service account "default".
How do I grant my-svc-account access to the default service account for GKE?
After talking to Google Support, the issue was that the service account did not have a "Service Account User" permissions activated. Adding "Service Account User" resolves this error.
Add the following role to the service account who makes the operation:
Service Account User
Also see:
https://cloud.google.com/kubernetes-engine/docs/how-to/iam#service_account_user
https://cloud.google.com/iam/docs/service-accounts#the_service_account_user_role
https://cloud.google.com/iam/docs/understanding-roles
For those that ended up here trying to do an Import of Firebase Firestore documents with a command such as:
gcloud beta firestore import --collection-ids='collectionA','collectionB' gs://YOUR_BUCKET
I got around the issue by doing the following:
From the Google Cloud Console Storage Bucket Browser, add the service account completing the operation to the list of members with a role of Storage Admin.
Re-attempt the operation.
For security, I revoked the role after the operation completed, but that's optional.
iam.serviceAccounts.actAs is the exact permission you need from Service Account User
I was getting the The user does not have access to service account... error even though I added the Service Account User role as others have suggested. What I was missing was the organization policy that prevented service account impersonation across projects. This is explained in the docs: https://cloud.google.com/iam/docs/impersonating-service-accounts#enabling-cross-project
Added Service Account User role to service account and it worked for me.