I am trying to create a new service account with terraform code. This code works well when I run it on behalf of my user credentials (the project Owner). But I see an error when I run the code on behalf of terraform service account
module "service_accounts" {
source = "terraform-google-modules/service-accounts/google"
version = "~> 4.1"
project_id = var.project
prefix = var.env
names = ["dataproc-sa"]
project_roles = [
"${var.project}=>roles/cloudkms.cryptoKeyEncrypterDecrypter",
"${var.project}=>roles/storage.objectViewer",
"${var.project}=>roles/dataproc.worker",
]
}
The terraform-google-modules/service-accounts/google module documentation says
Service account or user credentials with the following roles must be used to provision the resources of this module:
* Service Account Admin: roles/iam.serviceAccountAdmin
* (optional) Service Account Key Admin: roles/iam.serviceAccountKeyAdmin when generate_keys is set to true
* (optional) roles needed to grant optional IAM roles at the project or organizational level
I've bound roles/iam.serviceAccountAdmin to the gitlab-terraform#xxxxxx.iam.gserviceaccount.com service account, despite this I see the next error
Error: Error creating service account: googleapi: Error 403:
Identity and Access Management (IAM) API has not been used in project xxxxxxxxx
before or it is disabled. Enable it by visiting
https://console.developers.google.com/apis/api/iam.googleapis.com/overview?project=xxxxxxxxx
then retry. If you enabled this API recently, wait a few minutes for
the action to propagate to our systems and retry.
It looks like the terraform service account doesn't have the appropriate permissions to create a new service account
What should I do to allow the service account create another one? What kind of permissions are missed? The code works well if I run it with the project Owner role (but I don't want to do this)
EDIT: Forgot to mention that I tried to bind roles/iam.admin to the terraform service account but it gave me the next error
ERROR: Policy modification failed. For a binding with condition,
run "gcloud alpha iam policies lint-condition" to identify
issues in condition.
ERROR: (gcloud.projects.add-iam-policy-binding)
INVALID_ARGUMENT: Role roles/iam.admin is not supported for this resource.
The problem is the IAM service is not enabled. To enable that service you need the IAM role Service Usage Admin aka roles/serviceusage.serviceUsageAdmin. The Owner and Editor roles also have the permission serviceusage.services.enable.
You can enable the service using the CLI:
gcloud services enable iam.googleapis.com
Or add this to your Terraform HCL to enable the IAM service.
resource "google_project_service" "iam_service" {
project = var.project
service = "iam.googleapis.com"
}
You should also add a depends_on to resources that depend on the service being enabled:
depends_on = [
google_project_service.iam_service
]
Related
I'm new to GCP and I'm trying to enable a number of API's via Terraform.
variable "gcp_service_list" {
description ="Projectof apis"
type = list(string)
default = [
"cloudresourcemanager.googleapis.com",
"serviceusage.googleapis.com"
]
}
resource "google_project_service" "gcp" {
for_each = toset(var.gcp_service_list)
project = "project-id"
service = each.key
}
but I keep running into the error
Error when reading or editing Project Service Foo/compute.googleapis.com: googleapi: Error 403: The caller does not have permission, forbidden
What permissions do I need to grant my service account in order for it to be able to do this please?
In order to enable service APIs in GCP, your user or service account which is being used to run Terraform needs to have the following role:
roles/serviceusage.serviceUsageAdmin
So, you will either have to grant the user or SA the role above from the console or if you have a Terraform resource to bind roles to users/SA that can be used as well.
From Terraform authentication perspective, if you are using a user account make sure you are properly authenticating from the terminal to GCP using the following command:
gcloud auth application-default login
If you are using a service account, you will need to specify the environment variable GOOGLE_APPLICATION_CREDENTIALS passing the json key file.
For Terraform authentication reference: https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference
On GCP, I'm trying to enable "Identity and Access Management (IAM) API" with this Terraform code below:
provider "google" {
credentials = file("myCredentials.json")
project = "myproject-173831"
region = "asia-northeast1"
}
resource "google_project_service" "project" {
service = "iam.googleapis.com"
}
But I got this error:
Error when reading or editing Project Service : Request List Project Services myproject-173831 returned error: Failed to list enabled
services for project myproject-173831: googleapi: Error 403:
Permission denied to list services for consumer container
[projects/335478934851]
Then, I couldn't enable it
So now, I'm trying to add a role to solve this error above but there are too many roles to choose:
What role do I need to choose?
Choose the role "Quota Administrator":
Then, apply your Terraform code again:
provider "google" {
credentials = file("myCredentials.json")
project = "myproject-173831"
region = "asia-northeast1"
}
resource "google_project_service" "project" {
service = "iam.googleapis.com"
}
Finally, you can enable "Identity and Access Management (IAM) API":
I agree with answer 1 but a few additions:
Comment recommends roles/servicemanagement.quotaViewer, this is insufficient for the terraform resource google_project_service.
Required roles I have made it work with are:
Quota Administrator (roles/servicemanagement.quotaAdmin)
Service Usage Admin (roles/serviceusage.serviceUsageAdmin)
Sometimes it takes a few minutes to enable services and it is worth retrying after a failed run to see if the services are now enabled.
I would also recommend batch enabling services in their own resource/module using the pattern shown here.
I have no ideas what to set here. The whole policy, binding and member stuff is very confusing IMHO. Are any of these roles? Anyway...
Trying to access the secret manager from a cloud function. The cloud function is setup using Terraform:
module "mds_reporting_cloud_function" {
source = "terraform-google-modules/scheduled-function/google"
version = "2.0.0"
project_id = var.function_gcp_project
job_name = var.function_name
job_description = var.function_description
job_schedule = var.function_cron_schedule
function_entry_point = "main"
function_source_directory = "${path.module}/../../../../src"
function_name = var.function_name
region = var.function_gcp_region
bucket_name = var.function_name
function_description = var.function_description
function_environment_variables = var.function_environment_variables
function_runtime = "python38"
topic_name = var.function_name
}
resource "google_cloudfunctions_function_iam_binding" "binding" {
project = var.function_gcp_project
region = var.function_gcp_region
cloud_function = var.function_name
role = "roles/secretmanager.secretAccessor"
members = [
"serviceAccount:${var.function_gcp_project}#appspot.gserviceaccount.com"
]
}
My understanding is, that if no service account for the cloud function is specified it will use the default App Engine service account.
The binding should 'bind' the role to the existing IAM policy of the App Engine service account.
However, it throws this error:
Error:
Error applying IAM policy for cloudfunctions cloudfunction "projects/alpine-proton-280612/locations/europe-west3/functions/mds-reporting-cloud-function":
Error setting IAM policy for cloudfunctions cloudfunction "projects/alpine-proton-280612/locations/europe-west3/functions/mds-reporting-cloud-function":
googleapi: Error 400: Role roles/secretmanager.secretAccessor is not supported for this resource.
Not sure what do.
The best solution is to grant, only on the secret, the permission for the Cloud Functions service account to access the secret. For that use the Secret Manager IAM terraform resource
resource "google_secret_manager_secret_iam_binding" "binding" {
project = var.function_gcp_project
secret_id = google_secret_manager_secret.your-secret.secret_id
# If your secret is not created by terraform, use this format for the id projects/{{project}}/secrets/{{secret_id}}
role = "roles/secretmanager.secretAccessor"
members = [
"serviceAccount:${var.function_gcp_project}#appspot.gserviceaccount.com"
]
}
Important note:
You can also grant this role at the project level, but it's less secure because the function will have access to all the secrets of the project
You use the App Engine (and cloud function) default service account. It's also not so secure. Indeed, any App Engine service, and Cloud Functions with custom service account will use by default this service account, and thus will be able to access the secrets. Prefer a custom service account for your Cloud Functions
The second comment of John is very important. Terraform have several level of write-and-replace of IAM roles. Keep in mind (work for all IAM_XXX terraform module)
Policy replace all the accounts of all the possible roles of the whole resource (on a project, it could be dramatic!)
Binding replace all the accounts of a specific role
Member only add an account on a specific role. Delete nothing.
I am trying to apply the role binding below to grant the Storage Admin Role to a GCP roleset in Vault.
resource "//cloudresourcemanager.googleapis.com/projects/{project_id_number}" {
roles = [
"roles/storage.admin"
]
}
I want to grant access to the project level, not a specific bucket so that the GCP roleset can access and read/write to the Google Container Registry.
When I try to create this roleset in Vault, I get this error:
Error writing data to gcp/roleset/my-roleset: Error making API request.
URL: PUT http://127.0.0.1:8200/v1/gcp/roleset/my-roleset
Code: 400. Errors:
* unable to set policy: googleapi: Error 403: The caller does not have permission
My Vault cluster is running in a GKE cluster which has OAuth Scopes for all Cloud APIs, I am the project owner, and the service account Vault is using has the following permissions:
Cloud KMS CryptoKey Encrypter/Decrypter
Service Account Actor
Service Account Admin
Service Account Key Admin
Service Account Token Creator
Logs Writer
Storage Admin
Storage Object Admin
I have tried giving the service account both Editor and Owner roles, and I still get the same error.
Firstly, am I using the correct resource to create a roleset for the Storage Admin Role at the project level?
Secondly, if so, what could be causing this permission error?
I had previously recreated the cluster and skipped this step:
vault write gcp/config credentials=#credentials.json
Adding the key file fixed this.
There is also a chance that following the steps to create a custom role here and adding that custom role played a part.
Created a service user to manage terraform under the project and gave it roles/owner. Created the key for this terraform user.
Terraform code is simple:
resource "google_pubsub_topic" "my_topic" {
name = "my_topic"
project = "${var.project_id}"
}
just creating a pub-sub topic.
terraform plan works but terraform apply gives me:
google_pubsub_topic.my_topic: googleapi: Error 403: User not authorized to perform this action., forbidden
I have even tried giving the service account roles/pubsub.admin not sure I understand what's going on because my service account has the owner role associated with it yet it can't create a pubsub topic.
Can anybody help me figure this out please?
Thanks in advance.
Edit (to expand on my comment about what worked):
I had created the service account using gcloud.
gcloud iam service-accounts create terraform \
--display-name "Terraform admin account"
gcloud projects add-iam-policy-binding myproject-182220 \
--member serviceAccount:terraform#myproject-182220.iam.gserviceaccount.com \
--role roles/owner
gcloud iam service-accounts keys create terraform-admin.json \
--iam-account terraform#myproject-182220.iam.gserviceaccount.com
To continue my debugging, I created a new service account using the console / GUI --> API & Services --> Credentials --> Create Credentials --> Service Account Key --> New Service Account (With the owner role). With this new service key json file, I was able to run my terraform code without a problem.
Now my confusion is, why did this work but not when I used gcloud to create a service account and give it the same role?
You have to enable the pub/sub api via google cloud or it might be possible using terraform.
The sample usage configuration for the Google provider looks more like:
// Configure the Google Cloud provider
provider "google" {
credentials = "${file("account.json")}"
project = "my-gce-project"
region = "us-central1"
}
// Create a new instance
resource "google_compute_instance" "default" {
# ...
}
This can be read in more detail on the "Google Cloud Provider" documentation page from Terraform.