I want to create an object through which I will iterate using for_each to create some:
google_service_account
google_project_iam_member
resources
So my object is more or less like this
service_accounts = {
"gha_storage_admin_sa" = {
create = true
project = var.project_id
account_id = "id1"
display_name = "GHA service account 1"
role = "roles/storage.admin"
},
"gha_cluster_scaling_sa" = {
create = false
project = var.project_id
account_id = "id2"
display_name = "GHA service account 2"
role = google_organization_iam_custom_role.my_custom_role.id
},
}
resource "google_service_account" "service_account" {
for_each = {
for k, v in local.service_accounts: k => v
if v.create
}
project = each.value.project
account_id = each.value.account_id
display_name = each.value.display_name
}
resource "google_project_iam_member" "member" {
for_each = local.service_accounts
project = var.project_id
role = each.value.role
member = "serviceAccount:${google_service_account.service_account[each.key].email}"
}
This works fine if the above is a local variable.
I want however to expose it as a regular variable.
My question is whether the referenced resource (google_organization_iam_custom_role.my_custom_role.id) in the second element can be somehow exposed as a variable.
My question is whether the referenced resource (google_organization_iam_custom_role.my_custom_role.id) in the second element can be somehow exposed as a variable.
Sadly its not possible. TF does not support dynamic variables. All variables' values must be know at plan time. You have to pass the actual value of google_organization_iam_custom_role.my_custom_role.id as input variable to your code.
Related
I'm using following local variables to pass map of account_id and account_type to my child module in order to do to create patch manager resources based on account_type from my child module.
locals {
org_sub_accounts_map = zipmap(module.accounts.account_id, module.accounts.tags_all.*.AccountType)
org_sub_accounts = [for k, v in local.org_sub_accounts_map : {
id = k
type = v
}
]
}
module "ssm_patch_manager" {
source = "../../../../modules/aws/cloudformation/stacksets"
accounts = local.org_sub_accounts
account_exception_list = var.account_exception_list
regions = var.region_list
stackset_name = "SSM-PatchManager"
template = "ssm_patch_manager"
parameters = var.patch_manager_default_params
parameter_overrides = var.patch_manager_params_overrides
stackset_admin_role_arn = module.stackset_admin_role.role_arn
depends_on = [module.accounts]
}
local.org_sub_accounts is something like this:
org_sub_accounts = [
{
"id" = "111111111111"
"type" = "Dev"
},
{
"id" = "222222222222"
"type" = "Prod"
},
{
"id" = "33333333333"
"type" = "Dev"
}
]
This works fine with all the existing AWS accounts as terraform aware of the accounts IDs. Now the problem is, when I'm creating a new AWS account from module.accounts, and running the terraform plan, I get below error:
Error: Invalid for_each argument
on ../../../../modules/aws/cloudformation/stacksets/main.tf line 25, in resource "aws_cloudformation_stack_set_instance" "stack":
25: for_each = {
26: for stack_instance in local.instance_data : "${stack_instance.account}.${stack_instance.region}" => stack_instance if contains(var.account_exception_list, stack_instance.account) == false
27: }
├────────────────
│ local.instance_data will be known only after apply
│ var.account_exception_list is list of string with 1 element
The "for_each" value depends on resource attributes that cannot be determined
until apply, so Terraform cannot predict how many instances will be created.
To work around this, use the -target argument to first apply only the
resources that the for_each depends on.
I understand this is clearly because terraform doesn't know the account_id when evaluating the locals variables. Can anyone suggest something here to resolve this issue?
Please note, this is already implemented solution. But we came to know after sometime when we try to create a new account. Therefore, any suggestion without major structure changes in the code would be really helpful.
Update
instance_data is a resource block in child module.
locals {
instance_data = flatten([
for account in var.accounts : [
for region in var.regions : {
account = account.id
type = try(length(account.type), 0) > 0 ? account.type : "default" # To support legacy var.account input, which account 'type' key is not passed.
region = region
}
]
])
resource "aws_cloudformation_stack_set_instance" "stack" {
for_each = {
for stack_instance in local.instance_data : "${stack_instance.account}.${stack_instance.region}" => stack_instance if contains(var.account_exception_list, stack_instance.account) == false
}
account_id = each.value.account
region = each.value.region
parameter_overrides = lookup(var.parameter_overrides, each.value.type, null) # To handle different parameters based on 'AccountType' Tag in sub accounts
stack_set_name = aws_cloudformation_stack_set.stackset.name
}
Finally came up with below solution. Just posting this here if anyone needs to reference in future.
data "aws_organizations_resource_tags" "account" {
count = length(module.organization.non_master_accounts.*.id)
resource_id = module.organization.non_master_accounts.*.id[count.index]
}
# “for_each” value in child module depends on resource attributes (AWS account_id) that cannot be determined until terraform apply, so Terraform cannot predict how many instances will be created.
# Because of this, we use `aws_organizations_resource_tags` data source to create stacksets, instead of module outputs.
locals {
org_sub_accounts = [for account in data.aws_organizations_resource_tags.account : {
id = account.id
type = try(length(account.tags.AccountType), 0) > 0 ? account.tags.AccountType : "default" # In case AccountType tag missing in existing/invited accounts.
}
]
}
# Output in Organization module
output "non_master_accounts" {
value = aws_organizations_organization.root.non_master_accounts
}
I am using the following condition for creating the follwoing resource in Terraform
resource "google_bigquery_dataset_access" "access" {
count = contains(var.bq_access, "viewer") ? length(var.bigquery_datasets) : 0
project = "test-project"
dataset_id = var.bigquery_datasets[count.index].dataset_id
role = "roles/bigquery.dataViewer"
user_by_email = lower(var.user_email[?])
}
I am trying to create multiple instances of this resource based on length(var.bigquery_datasets), the problem is the var.user_email is a list and I want this resource to be created for all emails and for all elements of bigquery_datasets. How can I combine the following code and the code above in one resource block?
resource "google_bigquery_dataset_access" "access" {
count = contains(var.bq_access, "viewer") ? length(var.user_email) : 0
project = "test-project"
dataset_id = var.bigquery_datasets[?].dataset_id
role = "roles/bigquery.dataViewer"
user_by_email = lower(var.user_email[count.index])
}
The length of var.bigquery_datasets and var.user_email is not the same. Any help would be appreciated. Thankyou!
UPDATE :
variable "user_email" {
type = list(any)
description = "email addresses list"
}
eg definition user_email = ["email1#xyz.com", "email2#xyz.com", "email3#xyz.com"]
variable "bigquery_datasets" {
type = list(any)
description = "datasets list"
}
eg definition bigquery_datasets = ["datasetid1", "datasetid2", "datasetid3", "datasetid4", "datasetid5"]
I think that setproduct would solve your issue:
locals {
emails_datasets = setproduct(var.user_email, var.bigquery_datasets)
}
resource "google_bigquery_dataset_access" "access" {
count = contains(var.bq_access, "viewer") ? length(local.emails_datasets) : 0
project = "test-project"
dataset_id = local.emails_datasets[count.index][1]
role = "roles/bigquery.dataViewer"
user_by_email = lower(local.emails_datasets[count.index][0])
}
I am creating AWS ECR repositories via terraform
resource "aws_ecr_repository" "repo1" {
name = "repo1"
image_tag_mutability = "MUTABLE"
image_scanning_configuration {
scan_on_push = true
}
}
resource "aws_ecr_repository" "repo2" {
name = "repo2"
image_tag_mutability = "MUTABLE"
image_scanning_configuration {
scan_on_push = true
}
}
Now I want to attach a policy to all ECR repositories.
Question is, is there a dynamic way to create a list of all the resources (of type ECR) created using the terraform script? If yes then we can have a for_each on that list and attach a policy.
Or is there any better way to do it?
P.S. I know I can attach policy by writing the following for each. I want to avoid duplication and avoid a case where policy is not attached if the block is missed by someone
resource "aws_ecr_lifecycle_policy" "insights_repository_policy" {
repository = aws_ecr_repository.insights_repository.name
policy = local.ecr_cleanup_policy
}
Edit: Question 2
There are some accounts I want to give access to. If I use list of repositories to create and then I want to assign policies for each account then it would make nested for loops. Is there a cleaner solution for that?
local {
accounts = {test=account_id_123, prod=account_id_456}
}
resource "aws_ecr_repository_policy" "access-permission" {
for_each = local.accounts
policy = <<POLICY
...
POLICY
repository = aws_ecr_repository.repo_template.name
}
Not in your form. It would be better if you used for_each or count. For example:
variable "repos" {
default = ["repo1", "repo2"]
}
resource "aws_ecr_repository" "repo" {
for_each = to_set(var.repos)
name = each.key
image_tag_mutability = "MUTABLE"
image_scanning_configuration {
scan_on_push = true
}
}
then you can do:
resource "aws_ecr_lifecycle_policy" "insights_repository_policy" {
for_each = aws_ecr_repository.repo
repository = each.value.name
policy = local.ecr_cleanup_policy
}
main.tf
module "iam_assumable_role" {
for_each = var.service_accounts
source = "../../../../../../modules/iam-assumable-role-with-oidc/"
create_role = true
role_name = each.value.name
provider_url = replace(module.eks.cluster_oidc_issuer_url, "https://", "")
// role_policy_arns = [for i in each.value.policies : "aws_iam_policy.${i}.arn"]
oidc_fully_qualified_subjects = each.value.wildcard == "" ? ["system:serviceaccount:${each.value.namespace}:${each.value.name}"] : []
oidc_subjects_with_wildcards = each.value.wildcard != "" ? ["system:serviceaccount:${each.value.namespace}:${each.value.wildcard}"] : []
tags = var.tags
}
resource "aws_iam_policy" "dev-policy1" {
name_prefix = "dev-policy"
description = "some description"
policy = data.aws_iam_policy_document.dev-policy1.json
}
variable "service_accounts" {
type = map(object({
name = string
namespace = string
wildcard = string
policies = list(any)
}))
}
tfvars
service_accounts = {
"dev-sa" = {
"name" = "dev-sa",
"namespace" = "dev",
"wildcard" = "*",
"policies" = ["dev-policy1", "dev-policy2"]
},
"qa-sa" = {
"name" = "qa-sa",
"namespace" = "qa",
"wildcard" = "*",
"policies" = ["qa-policy1", "qa-policy2"]
}
}
My code is iterating over service_accounts variable and creates appropriate resources. The problem is that in the commented line I cannot get the list of aws_iam_policy.arn s for the provided policy names (policy names are provided through service_account variable). My current code returns the aws_iam_policy.PolicyName.arn as string and not the actual value. Note that dev-policy1 resource s just one of the all policy resources. All policy documents exist as well. module itself is working correctly when I provide policy list directly and not through variable.
Is it possible to achieve the desired in terraform at all?
You have to use for_each, to create your policies, as you can't dynamically references individual resources the way you are trying to do:
# get all policy names. Your names are unique, so its fine to use list
locals {
policy_names = flatten(values(var.service_accounts)[*]["policies"])
}
# create policy for each name in `policy_names`
resource "aws_iam_policy" "policy" {
for_each = local.policy_names
name_prefix = "dev-policy"
description = "some description"
# similar must be done below
# policy = data.aws_iam_policy_document.dev-policy1.json
}
Then you refer to them as:
role_policy_arns = [for i in each.value.policies: aws_iam_policy[${i}].arn]
The Terraform resource, aws_db_proxy, has a list of auth block(s) as an argument. Below is an example from the terraform documentation.
Each auth block represents a user, and each user needs a secret in Secrets Manager. Our platform has four different environments (dev,qa,cert,prod), and we do not use secrets in our lower environments to save on costs. Ideally, I would create two lists of auth blocks, one for lower environments and one for upper environments. Then, in the resource I could pick the appropriate one based on environment.
Is there a way to pass a list of auth blocks to the aws_db_proxy resource?
The other solution I was thinking of was to use two separate aws_db_proxy configurations and use the appropriate one for each environment using the count meta-argument. However, I think this could get a little messy.
resource "aws_db_proxy" "example" {
name = "example"
debug_logging = false
engine_family = "MYSQL"
idle_client_timeout = 1800
require_tls = true
role_arn = aws_iam_role.example.arn
vpc_security_group_ids = [aws_security_group.example.id]
vpc_subnet_ids = [aws_subnet.example.id]
auth {
auth_scheme = "SECRETS"
description = "user1"
iam_auth = "DISABLED"
secret_arn = aws_secretsmanager_secret.example1.arn
}
auth {
auth_scheme = "SECRETS"
description = "example2"
iam_auth = "DISABLED"
secret_arn = aws_secretsmanager_secret.example2.arn
}
auth {
auth_scheme = "SECRETS"
description = "example3"
iam_auth = "DISABLED"
secret_arn = aws_secretsmanager_secret.example3.arn
}
tags = {
Name = "example"
Key = "value"
}
}
You could use dynamic blocks to create auth blocks dynamically.
An example usage would depend on exactly how are you defing your aws_secretsmanager_secret for each user, but you could also make it dynamic.
Below is sample code. I haven't run it as its aim is to demonstrate the concept of the use of dynamic blocks and how you could make your aws_secretsmanager_secret:
# list of users
variable "proxy_users" {
default = ["user1", "example2", "example3"]
}
# secret for each user
resource "aws_secretsmanager_secret" "mysecret" {
for_each = toset(var.proxy_users)
name = "example${each.key}"
# rest of attributes
}
resource "aws_db_proxy" "example" {
name = "example"
debug_logging = false
engine_family = "MYSQL"
idle_client_timeout = 1800
require_tls = true
role_arn = aws_iam_role.example.arn
vpc_security_group_ids = [aws_security_group.example.id]
vpc_subnet_ids = [aws_subnet.example.id]
# create auth for each user
dynamic "auth" {
for_each = var.proxy_users
content {
auth_scheme = "SECRETS"
description = auth.key
iam_auth = "DISABLED"
secret_arn = aws_secretsmanager_secret.mysecret[auth.key].arn
}
}
tags = {
Name = "example"
Key = "value"
}
}
Thank you #Marcin
I had the same issue but I needed to insert existing secrets arn. You really helped
I did the following if anybody needs it
locals {
secrets_list = [
"db-credentials/${var.env-name}/user1",
"db-credentials/${var.env-name}/user2",
"db-credentials/${var.env-name}/user3"
]
}
data "aws_secretsmanager_secret" "rds_secrets" {
for_each = toset(local.secrets_list)
name = each.key
}
resource "aws_db_proxy" "rds_db_proxy" {
name = "${var.env-name}-rds-proxy"
engine_family = "MYSQL"
idle_client_timeout = 900
require_tls = true
.
.
.
.
dynamic "auth" {
for_each = local.secrets_list
content {
secret_arn = data.aws_secretsmanager_secret.rds_secrets[auth.value].arn
auth_scheme = "SECRETS"
iam_auth = "REQUIRED"
}
}
}