Tricky conversion of variable using locals in terraform - amazon-web-services

looking for possible solution/approach for the below req.
I have a var as below
nodes= {
"type1" = {
max_size = 4
tags = { "key1" = "value1"
"key2" = "value2" } # structure of tags can be changed as needed
}
and then I have a locals that will ref this tag to use in an asg
locals {
for nodetype, nodeconfig in var.nodes
..
..
node_tags = nodeconfig.tags
}
local.node_tags must take the final form as below
{
"key" = "key1"
"value" = "value1"
"propagate_at_launch" = "true"
},
{
"key" = "key2"
...
...
}
I need to use these in an asg
resource "aws_autoscaling_group" "nodes" {
for_each = { for i in local.nodes ....
...
...
tags = each.value.node_tags
}
Similar use case - https://github.com/terraform-aws-modules/terraform-aws-autoscaling#tags
I tried something below.. looks ugly and doesn't work.
locals {
for nodetype, nodeconfig in var.nodes
..
..
node_tags = {for i in nodeconfig.tags : "{i}" => {
"key" = keys(i),
"value" = values(i),
"propagate_at_launch" = "true"
}
}
}

What about this one:
variable "nodes" {
default = {
"type1" = {
max_size = 4
tags = { "key1" = "value1"
"key2" = "value2"
}
},
"type2" = {
max_size = 4
tags = { "key11" = "value11"
"key22" = "value22"
}
},
}
}
locals {
node_tags = values(merge([for nodetype, nodeconfig in var.nodes:
{for tag_key , tag_value in nodeconfig.tags:
"${nodetype}-${tag_key}-${tag_value}" =>
{key = tag_key
value = tag_value
"propagate_at_launch" = "true"}
}
]...))
}
output "test" {
value = local.node_tags
}
which gives:
test = [
{
"key" = "key1"
"propagate_at_launch" = "true"
"value" = "value1"
},
{
"key" = "key2"
"propagate_at_launch" = "true"
"value" = "value2"
},
{
"key" = "key11"
"propagate_at_launch" = "true"
"value" = "value11"
},
{
"key" = "key22"
"propagate_at_launch" = "true"
"value" = "value22"
},
]

Related

Terraform for_each nested solution

I need to access the elements of a nested map but I can't do it and I get an error Can't access attributes on a primitive-typed value (string).
# GLUE
locals {
glue_catalog_resources = {
uni = {
name = "mst_business_units",
description = "Unidades de negocios"
columns = [
{
name = "codigouni"
type = "int"
comment = "Code"
},{
name = "descuni"
type = "varchar(256)"
comment = "Description"
},{
name = "estado"
type = "varchar(256)"
comment = "Current status"
}
]
}
}
}
resource "aws_glue_catalog_table" "glue_catalogs_redshift" {
for_each = local.glue_catalog_resources
name = each.value.name
database_name = aws_glue_catalog_database.cl_sales.name
description = each.value.description
retention = 0
table_type = "EXTERNAL_TABLE"
parameters = {
EXTERNAL = "TRUE"
"classification" = "parquet"
}
storage_descriptor {
location = ""
input_format = "org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat"
output_format = "org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat"
number_of_buckets = 0
compressed = "false"
stored_as_sub_directories = "false"
ser_de_info {
serialization_library = "org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe"
parameters = {
"serialization.format" = 1
}
}
parameters = {
"typeOfData": "kinesis"
}
columns {
name = [ for column in each.value.columns: [ for row in column: row.name ] ]
type = [ for column in each.value.columns: [ for row in column: row.type ] ]
comment = [ for column in each.value.columns: [ for row in column: row.comment ] ]
}
}
}
I need to include in the columns tag columns name, type and comment reading it from the map above and I can't do it, what would be the correct way to read it
columns {
name = [ for column in each.column.value: [ for row in column: row.name ] [ for row in column: row.name ] [ for row in column: row.name ] [ for row in column: row.name
type = [ for column in each.value.columns: [ for row in column: row.type ]
comment = [ for column in each.value.columns: [ for row in column: row.comment ] ] ]
}
In this case, columns is a list of key value pairs, so you need to adjust the second for a bit:
name = [for column in local.columns : [for k, v in column : v if k == "name"]]
type = [for column in local.columns : [for k, v in column : v if k == "type"]]
comment = [for column in local.columns : [for k, v in column : v if k == "comment"]]
EDIT: The first part of the answer resolves the issue with accessing the list elements. However, in order for this to work as you would want to, I would suggest a couple of changes. In the local variable, you can change columns to be a map of objects:
locals {
glue_catalog_resources = {
uni = {
name = "mst_business_units",
description = "Unidades de negocios"
columns = {
"codigouni" = {
name = "codigouni"
type = "int"
comment = "Code"
},
"descuni" = {
name = "descuni"
type = "varchar(256)"
comment = "Description"
},
"estado " = {
name = "estado"
type = "varchar(256)"
comment = "Current status"
}
}
}
}
}
Then, in the columns block you would do the following:
resource "aws_glue_catalog_table" "glue_catalogs_redshift" {
for_each = local.glue_catalog_resources
name = each.value.name
database_name = aws_glue_catalog_database.cl_sales.name
description = each.value.description
retention = 0
table_type = "EXTERNAL_TABLE"
parameters = {
EXTERNAL = "TRUE"
"classification" = "parquet"
}
storage_descriptor {
location = ""
input_format = "org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat"
output_format = "org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat"
number_of_buckets = 0
compressed = "false"
stored_as_sub_directories = "false"
ser_de_info {
serialization_library = "org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe"
parameters = {
"serialization.format" = 1
}
}
parameters = {
"typeOfData" : "kinesis"
}
dynamic "columns" {
for_each = each.value.columns
content {
name = columns.value.name
type = columns.value.type
comment = columns.value.comment
}
}
}
}
This approach is using dynamic [1] and for_each [2] to iterate over all of the columns and assign the wanted values to arguments.
[1] https://developer.hashicorp.com/terraform/language/expressions/dynamic-blocks
[2] https://developer.hashicorp.com/terraform/language/meta-arguments/for_each

Terraform trouble with list in a map using for_each

I'm trying to take each value from the list in the map and iterate over it with the description in a map for a prefix list but can't work out how.
Variable:
users = {
"user1" = {
description = ""
secret_key_value = {
username = "user1"
home_directory = "/user1/"
}
allowlist = ["200.0.0.1/32"]
},
"user2" = {
description = ""
secret_key_value = {
username = "user2"
home_directory = "/user2/"
}
allowlist = ["200.0.0.5/32", "200.0.0.10/32"]
}
Resource:
resource "aws_ec2_managed_prefix_list" "sftp" {
count = local.prefix_list_enabled ? 1 : 0
name = "User Whitelist"
address_family = "IPv4"
max_entries = 10
dynamic "entry" {
for_each = {
for k, v in var.users : k => v
if v.allowlist != "" || v.description != ""
}
content {
cidr = entry.value.allowlist
description = entry.value.description
}
}
}
With the above, I'm getting "Inappropriate value for attribute "cidr": string required.". I need to break up the list values in the allowlist variable key and iterate through them with the description. Does anyone know how I can achieve this?
You have to flatten your users:
locals {
users_flat = merge([
for k,v in var.users: {
for cidr in v.allowlist:
"${k}-${cidr}" => {
description = v.description
secret_key_value = v.secret_key_value
"cidr" = cidr
}
}
]...)
}
then
dynamic "entry" {
for_each = local.users_flat
content {
cidr = entry.value.cidr
description = entry.value.description
}
}

flatten object made of nested list in terraform

I have the following variable that I try to parse :
variable.tf
variable "rbac_roles" {
type = object(
{
view = list(object({
group_name = string,
group_id = string,
namespaces = list(string)
})),
edit = list(object({
group_name = string,
group_id = string,
namespaces = list(string)
})),
admin = list(object({
group_name = string,
group_id = string,
namespaces = list(string)
}))
}
)
}
variable.tfvars
rbac_roles = {
view = [
{
group_name = "group1",
group_id = "123",
namespaces = ["default", "namespace1"]
},
{
group_name = "group2",
group_id = "456",
namespaces = ["namespace2"]
}
],
edit = [
{
group_name = "group1",
group_id = "123",
namespaces = ["namespace2"]
}
],
admin = [
{
group_name = "group3",
group_id = "789",
namespaces = ["default, namespace1, namespace2"]
},
]
}
I try to create the following resources :
resource "kubernetes_role_binding" "view_cluster_role_binding" {
metadata {
name = ${group}-${namespace}-viewer-binding
namespace = ${namespace}
}
role_ref {
api_group = "rbac.authorization.k8s.io"
kind = "Role"
name = "view"
}
subject {
kind = "Group"
name = ${group}
api_group = "rbac.authorization.k8s.io"
}
}
resource "kubernetes_role_binding" "edit_cluster_role_binding" {
metadata {
name = ${group}-${namespace}-viewer-binding
namespace = ${namespace}
}
role_ref {
api_group = "rbac.authorization.k8s.io"
kind = "Role"
name = "edit"
}
subject {
kind = "Group"
name = ${group}
api_group = "rbac.authorization.k8s.io"
}
}
resource "kubernetes_role_binding" "admin_cluster_role_binding" {
metadata {
name = ${group}-${namespace}-viewer-binding
namespace = ${namespace}
}
role_ref {
api_group = "rbac.authorization.k8s.io"
kind = "Role"
name = "admin"
}
subject {
kind = "Group"
name = ${group}
api_group = "rbac.authorization.k8s.io"
}
}
So far I have tried to flatten() my list and to loop over it with for and foreach but I haven't been successful yet. From what I understand I need to use a locals{} to reach my goal, but I can't get the right syntax. Any help will be appreciated !
First this is wrong ["default, namespace1, namespace2"]. It should be ["default", "namespace1", "namespace2"]. Once you fix that, you can flatten your data structure as follows:
locals {
flat_rbac_roles = merge([
for role, groups in var.rbac_roles:
merge([
for group_idx, group in groups:
{
for namespace_idx, namespace in group["namespaces"]:
"${role}-${group_idx}-${namespace_idx}" => {
role_name = role
group_name = group["group_name"]
group_id = group["group_id"]
namespace = namespace
}
}
]...)
]...)
}
which gives:
{
"admin-0-0" = {
"group_id" = "789"
"group_name" = "group3"
"namespace" = "default"
"role_name" = "admin"
}
"admin-0-1" = {
"group_id" = "789"
"group_name" = "group3"
"namespace" = "namespace1"
"role_name" = "admin"
}
"admin-0-2" = {
"group_id" = "789"
"group_name" = "group3"
"namespace" = "namespace2"
"role_name" = "admin"
}
"edit-0-0" = {
"group_id" = "123"
"group_name" = "group1"
"namespace" = "namespace2"
"role_name" = "edit"
}
"view-0-0" = {
"group_id" = "123"
"group_name" = "group1"
"namespace" = "default"
"role_name" = "view"
}
"view-0-1" = {
"group_id" = "123"
"group_name" = "group1"
"namespace" = "namespace1"
"role_name" = "view"
}
"view-1-0" = {
"group_id" = "456"
"group_name" = "group2"
"namespace" = "namespace2"
"role_name" = "view"
}
}
Using the flatten() approach :
resource "kubernetes_role_binding" "default_roles_binding" {
for_each = {
for binding in flatten([
for role_name, groups in var.rbac_roles : [
for group in groups : [
for ns in group.namespaces : [
{
binding_name = lower("${ns}-${group.group_name}-${role_name}")
role = role_name
group_id = group.group_id
group_name = group.group_name
ns = ns
}
]
]
]]) : binding.binding_name => binding }
metadata {
namespace = each.value.ns
name = each.value.binding_name
annotations = { "group_name" : each.value.group_name }
}
role_ref {
api_group = "rbac.authorization.k8s.io"
kind = "Role"
name = each.value.role
}
subject {
kind = "Group"
name = each.value.group_id
}
depends_on = [
azurerm_kubernetes_cluster.aks
]
}

How to define GCP log based metric with multiple labels using terraform , not able to define label_extractors block

resource "google_logging_metric" "logging_metric" {
for_each = { for inst in var.log_based_metrics : inst.name => inst }
name = each.value.name
filter = each.value.filter
metric_descriptor {
metric_kind = each.value.metric_kind
value_type = each.value.value_type
dynamic "labels" {
for_each = each.value.labels
content {
key = labels.value["label_key"]
value_type = labels.value["label_value_type"]
description = labels.value["label_description"]
}
}
display_name = each.value.display_name
}
label_extractors = {
<< How to define multiple label_keys with label extractors >>
}
}
I have tried for_each but it was referring to first for_each in the resource and we cant use dynamic as it creates multiple label_extractors blocks, which is not intended.
My variables file :
log_based_metrics = [
{
name = "name1",
filter = "something"
metric_kind = "DELTA",
value_type = "INT64",
labels = [
{
label_key = "deployment",
label_value_type = "STRING",
label_description = "deployment",
label_extractor = "REGEXP_EXTRACT(jsonPayload.involvedObject.name, \"(.*)-[^-]*-[^-]*$\")"
}
]
},
{
name = "name2",
filter = "something",
metric_kind = "DELTA",
value_type = "INT64",
labels = [
{
label_key = "deployment",
label_value_type = "STRING",
label_description = "deployment",
label_extractor = "REGEXP_EXTRACT(jsonPayload.involvedObject.name, \"(.*)-[^-]*-[^-]*$\")"
}
]
},
{
name = "name3",
filter = "something",
metric_kind = "DELTA",
value_type = "INT64",
labels = [
{
label_key = "deployment",
label_value_type = "STRING",
label_description = "deployment",
label_extractor = "REGEXP_EXTRACT(jsonPayload.involvedObject.name, \"(.*)-[^-]*-[^-]*$\")"
}
]
}
]
label_extractor should be something like this :(label_extractor ,label_key from each label block)
for example :
labels {
key = "mass"
value_type = "STRING"
description = "amount of matter"
}
labels {
key = "sku"
value_type = "INT64"
description = "Identifying number for item"
}
display_name = "My metric"
}
value_extractor = "EXTRACT(jsonPayload.request)"
label_extractors = {
"mass" = "EXTRACT(jsonPayload.request)"
"sku" = "EXTRACT(jsonPayload.id)"
}
label_extractors is an attribute, not a block. So you have to just create a map that you want. I don't know exact structure of your input data (not shown in the question), but it would be something as follows (rough example):
label_extractors = { for val in each.value.labels: val.label_key => val.label_extractor }

Terraform create weird for_each

I wanted to create a for_each loop that loops only over objects in array that have some specific key : value pair.
My input variables are:
inputs = {
names = ["first", "second"]
lifecycle_rules = [
{
name = first
condition = {
age = "1"
}
action = {
type = "Delete"
}
},{
condition = {
age = "2"
}
action = {
type = "Delete"
}
},
{
name = second
condition = {
age = "3"
}
action = {
type = "Delete"
}
},{
condition = {
age = "4"
}
action = {
type = "Delete"
}
}
]
and in my main.tf (btw for deploying gcp bucket for reference), I wanted to separate the lifecycle per bucket and wanted to apply only the rules that have the buckets name in it.
So if anyone has idea how to modify for_each code below to work, I would highly appreciate. I believe only the for_each needs to be changed to loop over the right elements (let's say only objects in that list that have name = first). from the var.lifecycle_rules set
resource "google_storage_bucket" "buckets" {
count = length(var.names)
name = "${lower(element(var.names, count.index))}"
...
dynamic "lifecycle_rule" {
#for_each = length(lookup(lifecycle_rules[lookup(element(var.names, count.index))])
for_each = lifecycle_rules
content {
action {
type = lifecycle_rule.value.action.type
storage_class = lookup(lifecycle_rule.value.action, "storage_class", null)
}
condition {
#age = lifecycle_rule.value.name == element(var.names, count.index) ? lookup(lifecycle_rule.value.condition, "age", null) : null
age = lookup(lifecycle_rule.value.condition, "age", null) : null
...
I think that this "wierd" look can be obtained in two stages.
Reorganize lifecycle_rules into a map based on names
variable "input" {
default = {
names = ["first", "second"],
lifecycle_rules = [
{
name = "first",
condition = {
age = "1"
},
action = {
type = "Delete"
}
},
{
condition = {
age = "2"
},
action = {
type = "Delete"
}
},
{
name = "second",
condition = {
age = "3"
},
action = {
type = "Delete"
}
},
{
condition = {
age = "4"
},
action = {
type = "Delete"
}
}
]
}
}
locals {
new = {
for name in var.input.names:
name => [for rule in var.input.lifecycle_rules:
contains(keys(rule), "name" ) ?
rule.name == name ? rule: null :
null ]
}
}
which will give local.new in the form of:
{
"first" = [
{
"action" = {
"type" = "Delete"
}
"condition" = {
"age" = "1"
}
"name" = "first"
},
null,
null,
null,
]
"second" = [
null,
null,
{
"action" = {
"type" = "Delete"
}
"condition" = {
"age" = "3"
}
"name" = "second"
},
null,
]
}
Perform the for_each
resource "google_storage_bucket" "buckets" {
for_each = toset(var.input.names)
name = each.key
dynamic "lifecycle_rule" {
# iterate for each name skipping null values
for_each = [for v in local.new[each.key]: v if v != null]
content {
action {
type = lifecycle_rule.value["action"].type
storage_class = lookup(lifecycle_rule.value["action"], "storage_class", null)
}
condition {
age = lookup( tag.value["condition"], "age", null)
}
}
}
}
I could only verify the first step and partial second (using aws_autoscaling_group and its multiple tag components). I don't access to google cloud to fully test the code.