I am not at all versed in organization policy administration, apologies if my question has an obvious answer.
I tried defining a google_access_context_manager_access_policy via terraform,
resource "google_access_context_manager_access_policy" "org-policy" {
parent = data.google_organization.org.name
title = "Parent policy for ACL restrictions"
}
but I get this error :
╷
│ Error: Error creating AccessPolicy: googleapi: Error 409: Policy already exists with parent organizations/<my-org-id>
│
│ with google_access_context_manager_access_policy.org-policy,
│ on policies.tf line 1, in resource "google_access_context_manager_access_policy" "org-policy":
│ 1: resource "google_access_context_manager_access_policy" "org-policy" {
│
╵
When I perform this command : gcloud access-context-manager policies list --organization <my-org-id> I see that there is a default org policy (that has never been explicitely set)
NAME ORGANIZATION SCOPES TITLE ETAG
<redacted-id> <my-org-id> default policy <what-s-even-this>
Can I create another organization policy upon which I could base all further dependent policies? e.g. google_access_context_manager_access_level
If I can't, how do I redefine the default org policy?
P.S : I'm not sure what tags (community) to invoke for this question. Please advise.
Edit :
Terraform & hashicorp/google versions !
Terraform v1.2.9
terraform {
required_providers {
google = {
source = "hashicorp/google"
version = ">=3.85.0"
}
}
}
Think I've found out: you can only have one policy on a particular level
As your organisation already have one by default you can only create new policies on folder/project.
Assuming you have the latest terraform-google-modules version
resource "google_access_context_manager_access_policy" "org-policy" {
parent = data.google_organization.org.name
title = "Parent policy for ACL restrictions"
scopes = ["folders/00000000000000"]
}
Related
I current have code that I have been using for quiet sometime that calls a custom S3 module. Today I tried to run the same code and I started getting an error regarding the provider.
╷ │ Error: Failed to query available provider packages │ │ Could not
retrieve the list of available versions for provider hashicorp/s3:
provider registry registry.terraform.io does not have a provider named
│ registry.terraform.io/hashicorp/s3 │ │ All modules should specify
their required_providers so that external consumers will get the
correct providers when using a module. To see which modules │ are
currently depending on hashicorp/s3, run the following command: │
terraform providers
Doing some digging seems that terraform is looking for a module registry.terraform.io/hashicorp/s3, which doesn't exist.
So far, I have tried the following things:
Validated that the S3 Resource code meets the standards of the upgrade Hashicorp did to 4.x this year. Plus I have been using it for a couple of months with no issues.
Delete .terraform directory and rerun terraform init (No success same error)
Delete .terraform directory and .terraform.hcl lock and run terraform init -upgrade (No Success)
I have tried to update my provider's file to try to force an upgrade (no Success)
I tried to change the provider to >= current version to pull the latest version with no success
Reading further, it refers to a caching problem of the terraform modules. I tried to run terraform providers lock and received this error.
Error: Could not retrieve providers for locking │ │ Terraform failed
to fetch the requested providers for darwin_amd64 in order to
calculate their checksums: some providers could not be installed: │ -
registry.terraform.io/hashicorp/s3: provider registry
registry.terraform.io does not have a provider named
registry.terraform.io/hashicorp/s3.
Kind of at my wits with what could be wrong. below is a copy of my version.tf which I changed from providers.tf based on another post I was following:
version.tf
# Configure the AWS Provider
provider "aws" {
region = "us-east-1"
use_fips_endpoint = true
}
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 4.9.0"
}
local = {
source = "hashicorp/local"
version = "~> 2.2.1"
}
}
required_version = ">= 1.2.0" #required terraform version
}
S3 Module
I did not include locals, outputs, or variables unless someone thinks we need to see them. As I said before, the module was running correctly until today. Hopefully, this is all you need for the provider's issue. Let me know if other files are needed.
resource "aws_s3_bucket" "buckets" {
count = length(var.bucket_names)
bucket = lower(replace(replace("${var.bucket_names[count.index]}-s3", " ", "-"), "_", "-"))
force_destroy = var.bucket_destroy
tags = local.all_tags
}
# Set Public Access Block for each bucket
resource "aws_s3_bucket_public_access_block" "bucket_public_access_block" {
count = length(var.bucket_names)
bucket = aws_s3_bucket.buckets[count.index].id
block_public_acls = var.bucket_block_public_acls
ignore_public_acls = var.bucket_ignore_public_acls
block_public_policy = var.bucket_block_public_policy
restrict_public_buckets = var.bucket_restrict_public_buckets
}
resource "aws_s3_bucket_acl" "bucket_acl" {
count = length(var.bucket_names)
bucket = aws_s3_bucket.buckets[count.index].id
acl = var.bucket_acl
}
resource "aws_s3_bucket_versioning" "bucket_versioning" {
count = length(var.bucket_names)
bucket = aws_s3_bucket.buckets[count.index].id
versioning_configuration {
status = "Enabled"
}
}
resource "aws_s3_bucket_lifecycle_configuration" "bucket_lifecycle_rule" {
count = length(var.bucket_names)
bucket = aws_s3_bucket.buckets[count.index].id
rule {
id = "${var.bucket_names[count.index]}-lifecycle-${count.index}"
status = "Enabled"
expiration {
days = var.bucket_backup_expiration_days
}
transition {
days = var.bucket_backup_days
storage_class = "GLACIER"
}
}
}
# AWS KMS Key Server Encryption
resource "aws_s3_bucket_server_side_encryption_configuration" "bucket_encryption" {
count = length(var.bucket_names)
bucket = aws_s3_bucket.buckets[count.index].id
rule {
apply_server_side_encryption_by_default {
kms_master_key_id = aws_kms_key.bucket_key[count.index].arn
sse_algorithm = var.bucket_sse
}
}
}
Looking for any other ideas I can use to fix this issue. thank you!!
Although you haven't included it in your question, I'm guessing that somewhere else in this Terraform module you have a block like this:
resource "s3_bucket" "example" {
}
For backward compatibility with modules written for older versions of Terraform, terraform init has some heuristics to guess what provider was intended whenever it encounters a resource that doesn't belong to one of the providers in the module's required_providers block. By default, a resource "belongs to" a provider by matching the prefix of its resource type name -- s3 in this case -- to the local names chosen in the required_providers block.
Given a resource block like the above, terraform init would notice that required_providers doesn't have an entry s3 = { ... } and so will guess that this is an older module trying to use a hypothetical legacy official provider called "s3" (which would now be called hashicorp/s3, because official providers always belong to the hashicorp/ namespace).
The correct name for this resource type is aws_s3_bucket, and so it's important to include the aws_ prefix when you declare a resource of this type:
resource "aws_s3_bucket" "example" {
}
This resource is now by default associated with the provider local name "aws", which does match one of the entries in your required_providers block and so terraform init will see that you intend to use hashicorp/aws to handle this resource.
My colleague and I finally found the problem. Turns out that we had a data call to the S3 bucket. Nothing was wrong with the module but the place I was calling the module had a local.tf action where I was calling s3 in a legacy format see the change below:
WAS
data "s3_bucket" "MyResource" {}
TO
data "aws_s3_bucket" "MyResource" {}
Appreciate the responses from everyone. Resource was the root of the problem but forgot that data is also a resource to check.
I have a Terraform module that I would like to modify. Currently my module creates a service account. I would like to modify it so that someone could pass in an existing service account OR if one is not passed in, then the module creates a service account is it would have originally.
Originally my service account looked like this:
resource "google_service_account" "scheduler" {
account_id = "${var.prefix}-scheduler"
project = var.project
}
I've added the following variable to my variables.tf file:
variable "service_account_email" {
default = null
description = "Existing service account for running ... jobs. If null a new service account will be created."
}
What I originally thought to do was to add some locals
locals {
service_account_count = var.service_account_email == null ? 1 : 0
service_account_email = var.service_account_email == null ? google_service_account.scheduler.email : var.service_account_email
}
Then I could change my service account to look like
resource "google_service_account" "scheduler" {
count = local.service_account_count
account_id = "${var.prefix}-scheduler"
project = var.project
}
And then wherever I would have referenced google_service_account.scheduler.email I can instead reference local.service_account_email .. It doesn't look like I'm able to do this, however, for a few reasons.
I get the following error if I try to use the locals block that mentioned above:
│ Because google_service_account.scheduler has "count" set, its attributes must be accessed on specific instances.
│
│ F`or example, to correlate with indices of a referring resource, use:
│ google_service_account.scheduler[count.index]
╵
If I change it so that I'm using google_service_account.scheduler[count.index].email instead, I get the following error:
│ Because google_service_account.scheduler has "count" set, its attributes must be accessed on specific instances.
│
│ For example, to correlate with indices of a referring resource, use:
│ google_service_account.scheduler[count.index]
╵
Now I'm sort of stuck, because I can't force any resources that would originally have referenced google_service_account.scheduler.email to instead reference the var.service_account_email variable that is being passed in for cases where we would prefer to use an existing service account.
Since you are using count, you have to use [0] to access your resource:
service_account_email = var.service_account_email == null ? google_service_account.scheduler[0].email : var.service_account_email
I am still in the process of learning terraform.
I am trying to deploy a cloudSQL database and provide a default service account to access it.
the following piece of code does not work :
# create default service account
resource "google_service_account" "default_service_account" {
account_id = "${var.database_name}-${random_id.db_name_suffix.hex}"
display_name = "Cloud SQL default Service Account for ${var.database_name}-${random_id.db_name_suffix.hex}"
}
# grant role sqlUser for default service account
resource "google_project_iam_member" "iam_binding_default_service_account" {
project = var.project_id
role = "roles/cloudsql.instanceUser"
member = "serviceAccount:${default_service_account.account_id}.${module.project.project_id}.iam.gserviceaccount.com"
depends_on = [
google_service_account.default_service_account,
]
}
terraform plan complains with :
Error: Reference to undeclared resource
on database.tf line 78, in resource "google_project_iam_member" "iam_binding_default_service_account":
78: member = "serviceAccount:${default_service_account.account_id}.${module.project.project_id}.iam.gserviceaccount.com"
A managed resource "default_service_account" "account_id" has not been
declared in the root module.
I do not understand why the depends_on piece of code does not seem to work and why terraform does not create the default_service_account before trying to populate the iam_binding_default_service_account ?
It should be (forgot google_service_account):
member = "serviceAccount:${google_service_account.default_service_account.account_id}.${module.project.project_id}.iam.gserviceaccount.com}"
Trying so assign a created role to a GCP service account which then is used as a workload identity for a k8s deployment.
Terraform:
resource google_project_iam_custom_role sign_blob_role {
permissions = ["iam.serviceAccounts.signBlob"]
role_id = "signBlob"
title = "Sign Blob"
}
resource google_service_account_iam_member document_signer_workload {
service_account_id = module.document_signer_service_accounts.service_accounts_map.doc-sign.name
role = "roles/iam.workloadIdentityUser"
member = local.document_sign_sa
}
module document_signer_service_accounts {
source = "terraform-google-modules/service-accounts/google"
version = "~> 3.0"
project_id = var.gcp_project_name
prefix = "doc-sign-sa"
names = ["doc-sign"]
project_roles = [
"${var.gcp_project_name}=>roles/viewer",
"${var.gcp_project_name}=>roles/storage.objectViewer",
"${var.gcp_project_name}=>roles/iam.workloadIdentityUser",
"${var.gcp_project_name}=>${google_project_iam_custom_role.sign_blob_role.name}"
]
display_name = substr("GCP SA bound to K8S SA ${local.document_sign_sa}. Used to sign document.", 0, 100)
}
Error:
Error: Request "Create IAM Members roles/signBlob serviceAccount:staging-doc-sign#********************.iam.gserviceaccount.com for \"project \\\"********************\\\"\"" returned error: Error applying IAM policy for project "********************": Error setting IAM policy for project "********************": googleapi: Error 400: Role roles/signBlob is not supported for this resource., badRequest
on .terraform/modules/document_signer_service_accounts/main.tf line 46, in resource "google_project_iam_member" "project-roles":
46: resource "google_project_iam_member" "project-roles" {
When I do the same action on the UI though, it allows me to assign the role.
What am I doing wrong here?
It seems that it could be a problem in the way you are calling the custom role.
"${var.gcp_project_name}=>${google_project_iam_custom_role.sign_blob_role.name}"
The custom role already belongs to the project, so it is not necessary to specify ${var.gcp_project_name}
So, the code should be something like:
project_roles = [
"${var.gcp_project_name}=>roles/viewer",
"${var.gcp_project_name}=>roles/storage.objectViewer",
"${var.gcp_project_name}=>roles/iam.workloadIdentityUser",
"${google_project_iam_custom_role.sign_blob_role.name}"
]
Edit 1
According to this documentation
This is the basic usage of the module service-accounts
module "service_accounts" {
source = "terraform-google-modules/service-accounts/google"
version = "~> 2.0"
project_id = "<PROJECT ID>"
prefix = "test-sa"
names = ["first", "second"]
project_roles = [
"project-foo=>roles/viewer",
"project-spam=>roles/storage.objectViewer",
]
}
I think there should be something wrong with the reference to the attribute from your resource.
Nevertheless I have found a github repository that contains some good examples on how to add a custom role to a Service Account:
# https://www.terraform.io/docs/providers/google/r/google_project_iam.html#google_project_iam_binding
resource "google_project_iam_binding" "new-roles" {
role = "projects/${var.project_id}/roles/${google_project_iam_custom_role.new-custom-role.role_id}"
members = ["serviceAccount:${google_service_account.new.email}"]
}
I think you might find it useful to complete this task.
I am trying to create a very simple structure on GCP using Terraform: a compute instance + storage bucket. I did some research across GCP documentation, Terraform documentation, SO questions as well and still can't understand what's the trick here. There is one suggestion to use google_project_iam_binding, but reading thruogh some articles it seems to be dangerous (read: insecure solution). There's also a general answer with only GCP descriptions, nit using tf terms here, which is still a bit confusing. And also concluding the similar question here, I confirm that the domain name ownership was verified via Google Console.
So, I ended up with the following:
data "google_iam_policy" "admin" {
binding {
role = "roles/iam.serviceAccountUser"
members = [
"user:myemail#domain.name",
"serviceAccount:${google_service_account.serviceaccount.email}",
]
}
}
resource "google_service_account" "serviceaccount" {
account_id = "sa-1"
}
resource "google_service_account_iam_policy" "admin-acc-iam" {
service_account_id = google_service_account.serviceaccount.name
policy_data = data.google_iam_policy.admin.policy_data
}
resource "google_storage_bucket_iam_policy" "policy" {
bucket = google_storage_bucket.storage_bucket.name
policy_data = data.google_iam_policy.admin.policy_data
}
resource "google_compute_network" "vpc_network" {
name = "vpc-network"
auto_create_subnetworks = "true"
}
resource "google_compute_instance" "instance_1" {
name = "instance-1"
machine_type = "f1-micro"
boot_disk {
initialize_params {
image = "cos-cloud/cos-stable"
}
}
network_interface {
network = google_compute_network.vpc_network.self_link
access_config {
}
}
}
resource "google_storage_bucket" "storage_bucket" {
name = "bucket-1"
location = "US"
force_destroy = true
website {
main_page_suffix = "index.html"
not_found_page = "404.html"
}
cors {
origin = ["http://the.domain.name"]
method = ["GET", "HEAD", "PUT", "POST", "DELETE"]
response_header = ["*"]
max_age_seconds = 3600
}
}
but if I terraform apply, logs show me an error like that
Error: Error setting IAM policy for service account 'trololo': googleapi: Error 403: Permission iam.serviceAccounts.setIamPolicy is required to perform this operation on service account trololo., forbidden
2020/09/28 19:19:34 [TRACE] statemgr.Filesystem: removing lock metadata file .terraform.tfstate.lock.info
on main.tf line 35, in resource "google_service_account_iam_policy" "admin-acc-iam":
35: resource "google_service_account_iam_policy" "admin-acc-iam" {
2020/09/28 19:19:34 [TRACE] statemgr.Filesystem: unlocking terraform.tfstate using fcntl flock
Error: googleapi: Error 403: The bucket you tried to create is a domain name owned by another user., forbidden
on main.tf line 82, in resource "google_storage_bucket" "storage_bucket":
and some useless debug info. What's wrong? What account is missing what permissions and how to assign them securely?
I found the problem. As always, in 90% of cases, the issue is sitting in front of the computer.
Here are the steps that helped me to understand and to resolve the problem:
I read few more articles and especially this and this answer were very helpful to understand relations between users, service accounts, permissions
I understood that doing terraform destroy is also very important since there is no rollback of unsuccessful deploy of a new infrastructure changes (like with DB migrations for example) - thus you have to clean up either with destroy or manually
completely removed the "user:${var.admin_email}" user account IAM policy since it useless; everything has to be managed by the newly created service account
left the main service account with most permissions untouched (the one which was created manually and downloaded the access key) since Terraform used it's credentials
and changed the IAM policy for the new service account as roles/iam.serviceAccountAdmin instead of a User - thanks #Wojtek_B for the hint
After this everything works smooth!