Terraform, create map where list elements are values - list

I am trying to parse this variable in terraform. The main goal is to create map of "group" = "member" type. Did i set type in variable correct? Required output provided below.
variable "iam" "this"{
type = map(map(list(string)))
default = {
"admins" = {
"user" = [
"oleh_mykolaishyn#gmail.com",
"yevhen_plaksa#gmail.com"
]
"service_principal" = [
"dpaf2-dev-sa"
]
}
"dev" = {
"user" = [
"oleh_mykolaishyn#gmail.com",
"yevhen_plaksa#gmail.com"
]
"service_principal" = []
}
"ops" = {
"user" = [
"oleksandr_gorkun#gmail.com",
]
"service_principal" = [
"dpaf2-dev-sa",
"dpaf2-iacda-app-id",
]
}
}
}
Required output:
{
"admins" = "oleh_mykolaishyn#gmail.com"
"admins" = "yevhen_plaksa#gmail.com"
"admins" = "dpaf2-dev-sa"
"dev" = "oleh_mykolaishyn#gmail.com"
"dev" = "yevhen_plaksa#gmail.com"
"ops" = "oleksandr_gorkun#gmail.com"
"ops" = "dpaf2-dev-sa"
"ops" = "dpaf2-iacda-app-id"
}

Start with this and manipulate it to suit your exact needs. There are many ways to structure resources using maps, arrays and nesting.
locals {
service_principal = {
admin = ["dpaf2-dev-sa"]
dev = []
ops = [
"dpaf2-dev-sa",
"dpaf2-iacda-app-id",
]
}
users = {
admin = [
"oleh_mykolaishyn#gmail.com",
"yevhen_plaksa#gmail.com"
]
dev = [
"yevhen_plaksa#gmail.com"
]
ops = [
"oleksandr_gorkun#gmail.com"
]
}
result = flatten([for role in keys(local.users) : [for user in local.users[role] :
{ role = role, user = user, principals = local.service_principal[role] }]])
}
resource "null_resource" "users" {
count = length(local.result)
provisioner "local-exec" {
command = "echo user: $USER has role: $ROLE and principals: $SERVICE_PRINCIPAL >> ouput.txt"
environment = {
USER = local.result[count.index].user
ROLE = local.result[count.index].role
SERVICE_PRINCIPAL = join(", ", local.result[count.index].principals)
}
}
}
If you run:
terraform init
terraform plan
terraform apply --auto-approve
Then check your directory for a file named output.txt which will contain
user:oleh_mykolaishyn#gmail.com has role:admin and principals: dpaf2-dev-sa
user:yevhen_plaksa#gmail.com has role:dev and principals:
user:oleksandr_gorkun#gmail.com has role:ops and principals: dpaf2-dev-sa, dpaf2-iacda-app-id
user:yevhen_plaksa#gmail.com has role:admin and principals: dpaf2-dev-sa
I know the text file isn't your desired output but the loop is just a demonstration of how you can iterate over the local variable.
Again, many ways to sort and slice it. I avoided your desired output because maps cannot have duplicate keys so it'll never work.
If you want to see the result in isolation you can do:
terraform console
local.result
And it will appear in the terminal. To exit the console type exit.

Related

Terraform : can I have 2 local block with same key but with different value

I have following local block, observe the domain key
locals {
organization = "xxxx"
domain = "cs"
env = {
prod = "prod"
stg = "stg"
dev = "dev"
}
}
locals {
s3_artifact_bucket_name = {
prod = join("-", [
local.s3_artifact_bucket_name_prefix,
local.env["prod"]
])
stg = join("-", [
local.s3_artifact_bucket_name_prefix,
local.env["stg"]
])
dev = join("-", [
local.s3_artifact_bucket_name_prefix,
local.env["dev"]
])
}
s3_artifact_bucket_name_prefix = join("-", [
local.organization,
local.domain,
local.s3_bucket_awsresource,
local.s3_artifact_bucket_purpose
])
s3_bucket_awsresource = join("-", [
"bucket",
var.cd_account_id
])
s3_artifact_bucket_purpose = "artifacts-iac"
}
local.domain ( cs) is being used to create some another local name.
now I want to add another local block with same domain but different value ( as there are 3 values for domain ) and create another local name for s3 bucket with common as value inside it.
locals {
# placeholder for access logs bucket name
domain = "common"
s3_bucket_awsresource = join("-", [
"bucket",
var.cd_account_id
])
s3_bucket_purpose = "s3-access-logs"
access_logs_bucket_region = "us-east-1"
}
here in the next step I want to reference local.domain as common and not cs
locals {
s3_artifact_access_logs_bucket_name = join("-", [local.organization, local.domain, s3_bucket_awsresource, local.s3_bucket_purpose, local.access_logs_bucket_region])
}
I am afraid local.domain will point out to which domain key and value? is there a way to use domain as key but with different value.
No, you cannot have a duplicate local in the same workspace.
Consider making a map instead (as you have done with env) or perhaps a list containing the three values, then using another control variable to select which value to use.
Thank to #Ben and #Calin, I was able to solve it,
Posting it for better understanding, using a map for the same key solved my problem
locals {
organization = "xxx"
domain = {
customer_service = "cs"
common = "common"
}
env = {
prod = "prod"
stg = "stg"
dev = "dev"
}
}
locals {
s3_artifact_access_logs_bucket_name = join("-", [
local.organization,
local.domain["common"],
local.s3_bucket_awsresource,
local.s3_access_logs_bucket_suffix
])
s3_access_logs_bucket_suffix = join("-", [
local.s3_acess_logs_bucket_purpose,
local.s3_access_logs_bucket_region
])
s3_acess_logs_bucket_purpose = "s3-access-logs"
s3_access_logs_bucket_region = "us-east-1"
}
It won’t work. Instead, you can define domain as a map, just like env, and create three different keys with three different values.

Apply same data for two resources in loop terraform

I want to create two Amazon SNS topics with the same aws_iam_policy_document, aws_sns_topic_policy & time_sleep configs.
This is my terraform, my_sns_topic.tf:
resource "aws_sns_topic" "topic_a" {
name = "topic-a"
}
resource "aws_sns_topic" "topic_b" {
name = "topic-b"
}
data "aws_iam_policy_document" "topic_notification" {
version = "2008-10-17"
statement {
sid = "__default_statement_ID"
actions = [
"SNS:Publish"
]
# Cut off some lines for simplification.
## NEW LINE ADDED
statement {
sid = "allow_snowflake_subscription"
principals {
type = "AWS"
identifiers = [var.storage_aws_iam_user_arn]
}
actions = ["SNS:Subscribe"]
resources = [aws_sns_topic.topic_a.arn] # Troubles with this line
}
}
resource "aws_sns_topic_policy" "topic_policy_notification" {
arn = aws_sns_topic.topic_a.arn
policy = data.aws_iam_policy_document.topic_policy_notification.json
}
resource "time_sleep" "topic_wait_10s" {
depends_on = [aws_sns_topic.topic_a]
create_duration = "10s"
}
As you can see here, I set up the configuration only for topic-a. I want to loop this over to apply for topic-b as well.
It would be better to use map and for_each, instead of separately creating "a" and "b" topics:
variable "topics" {
default = ["a", "b"]
}
resource "aws_sns_topic" "topic" {
for_each = toset(var.topics)
name = "topic-${each.key}"
}
data "aws_iam_policy_document" "topic_notification" {
version = "2008-10-17"
statement {
sid = "__default_statement_ID"
actions = [
"SNS:Publish"
]
# Cut off some lines for simplification.
}
resource "aws_sns_topic_policy" "topic_policy_notification" {
for_each = toset(var.topics)
arn = aws_sns_topic.topic[each.key].arn
policy = data.aws_iam_policy_document.topic_policy_notification.json
}
resource "time_sleep" "topic_wait_10s" {
for_each = toset(var.topics)
depends_on = [aws_sns_topic.topic[each.key]]
create_duration = "10s"
}

Terraform for_each loop aws_auth eks gets overwritten

I am creating my very own EKS cluster and had some issues adding in roles via my namespace declared in locals dynamically
I am using the terraform-aws-eks-auth Tag v1.0.0 to populate my IAM roles into the groups that i have created in terraform.
I expect that each.key variable would populate/append in the list of maps entries of map_roles but so far it doesn't do so. The end result in aws auth config map shows that it only contains AdminAccessRole, my_eks_admin_role, dev-kiwi-ingress-nginx_admin_role and dev-kiwi-ingress-nginx_readonly_role. The previous namespaces that was supposed to be dynamically created were overwritten.
Should I even use for_each in such case? How can I populate aws-auth dynamically by namespaces used, combined with 2 static entries.
I appreciate any advice or help!
locals {
namespaces = [
"${var.env}-devops",
"${var.env}-devops-ingress-nginx",
"${var.env}-shared",
"${var.env}-shared-ingress-nginx",
"${var.env}-pear",
"${var.env}-pear-ingress-nginx",
"${var.env}-apple",
"${var.env}-apple-ingress-nginx",
"${var.env}-banana",
"${var.env}-banana-ingress-nginx",
"${var.env}-kiwi",
"${var.env}-kiwi-ingress-nginx"
]
}
module "eks_auth" {
source = "./terraform-modules/terraform-aws-eks-auth-1.0.0"
eks = module.eks
# Additional IAM roles to add to the aws-auth configmap.
for_each = toset(local.namespaces)
map_roles = [
{
rolearn = "arn:aws:iam::1234567890:role/AdminAccessRole"
username = "AdminAccessRole"
groups = ["system:masters"]
},
{
rolearn = "arn:aws:iam::1234567890:role/my-eks-admin-role"
username = "my_eks_admin_role"
groups = ["system:masters"]
},
{
rolearn = aws_iam_role.eks_namespace_admin_roles[each.key].arn
username = "${each.key}_admin_role"
groups = ["${each.key}:${each.key}_group"]
},
{
rolearn = aws_iam_role.eks_namespace_readonly_roles[each.key].arn
username = "${each.key}_readonly_role"
groups = ["${each.key}:${each.key}_group"]
}
]
map_users = [
{
userarn = "arn:aws:iam::1234567890:user/terraform-service-account"
username = "terraform-service-account"
groups = ["system:masters"]
}
]
}
Terraform v1.0.11
aws provider version = "~> 4.9.0"
By adding the for_each in the level shown in the question, terraform will attempt to create multiple instances of the eks_auth module
In your case you need to apply the looping in the variable creation.
Also in order to not loop through the static definitions we use concat to merge the lists E.g.
map_roles = concat( [ for index,value in toset(local.namespaces) :
{
rolearn = aws_iam_role.eks_namespace_admin_roles[value].arn
username = "${value}_admin_role"
groups = ["${value}:${value}_group"]
},
{
rolearn = aws_iam_role.eks_namespace_readonly_roles[value].arn
username = "${value}_readonly_role"
groups = ["${value}:${value}_group"]
}
],
[
{
rolearn = "arn:aws:iam::1234567890:role/AdminAccessRole"
username = "AdminAccessRole"
groups = ["system:masters"]
},
{
rolearn = "arn:aws:iam::1234567890:role/my-eks-admin-role"
username = "my_eks_admin_role"
groups = ["system:masters"]
}
])
With the help from Tolis, this is the answer to the issue.
It was still throwing out errors when tried to FOR loop a couple of map entries, so i split them up and concat multiple list instead.
module "eks_auth" {
source = "./terraform-modules/terraform-aws-eks-auth-1.0.0"
eks = module.eks
# Additional IAM roles to add to the aws-auth configmap.
for_each = toset(local.namespaces)
map_roles = concat(
[
for key in toset(local.namespaces) :
{
rolearn = aws_iam_role.eks_namespace_admin_roles[key].arn
username = "${key}_admin_role"
groups = ["${key}:${key}_group"]
}
],
[
for key in toset(local.namespaces) :
{
rolearn = aws_iam_role.eks_namespace_readonly_roles[key].arn
username = "${key}_readonly_role"
groups = ["${key}:${key}_group"]
}
],
[
{
rolearn = "arn:aws:iam::1234567890:role/AdminAccessRole"
username = "AdminAccessRole"
groups = ["system:masters"]
},
{
rolearn = "arn:aws:iam::1234567890:role/my-eks-admin-role"
username = "my_eks_admin_role"
groups = ["system:masters"]
}
])
# Additional IAM users to add to the aws-auth configmap.
map_users = [
]
}

How. to iterate list(map(list)) in terraform

I want to create files(key) in s3. By below code, I can successfully create the files in s3 -
locals {
rules = [
{
user = "user-1",
roles = "test"
}
}
resource "aws_s3_bucket_object" "this" {
for_each = zipmap(local.rules.*.user, local.rules.*)
bucket = "test-09129"
acl = "private"
source = "/dev/null"
key = "${each.value.roles}/${each.key}"
}
file in s3 - test/user-1 created
But roles can be multiple so I create list for roles.
locals {
rules = [
{
user = "user-1",
roles = ["test"]
},
{
user = "user-2",
roles = ["test", "dev", "prd"]
}]
}
Now I want to create files based on above list. How to iterate list(roles) to create files based on user and role?
desire output -
file in s3 -
test/user-1
test/user-2
dev/user-2
prd/user-2
You can flatten your rules into helper_rules (assuming I understand correctly your desired outcome):
locals {
rules = [
{
user = "user-1",
roles = ["test"]
},
{
user = "user-2",
roles = ["test", "dev", "prd"]
}]
helper_rules = merge([
for user, rules in zipmap(local.rules.*.user, local.rules.*.roles):
{ for rule in rules:
"${user}-${rule}" => {user = user, rule = rule}
}
]...)
}
This will result in helper_rules:
{
"user-1-test" = {
"rule" = "test"
"user" = "user-1"
}
"user-2-dev" = {
"rule" = "dev"
"user" = "user-2"
}
"user-2-prd" = {
"rule" = "prd"
"user" = "user-2"
}
"user-2-test" = {
"rule" = "test"
"user" = "user-2"
}
}
With that, your objects could be:
resource "aws_s3_bucket_object" "this" {
for_each = local.helper_rules
bucket = "test-09129"
acl = "private"
source = "/dev/null"
key = "${each.value.role}/${each.value.user}"
}

Terraform: List of AMI specific to ubuntu 20.08 LTS AWS

Problem: I am using terraform to get a list of AMI for a specific OS - ubuntu 20.08
I have checked different examples link
When I use the script this does not give me list of AMI
Script
data "aws_ami" "ubuntu" {
most_recent = true
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-xenial-20.08-amd64-server-*"]
}
filter {
name = "virtualization - type"
values = ["hvm"]
}
owners = ["AWS"]
}
I have referred the below link as well
How are data sources used in Terraform?
Output:
[ec2-user#ip-172-31-84-148 ~]$ terraform plan
provider.aws.region
The region where AWS operations will take place. Examples
are us-east-1, us-west-2, etc.
Enter a value: us-east-1
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.aws_ami.std_ami: Refreshing state...
------------------------------------------------------------------------
No changes. Infrastructure is up-to-date.
This means that Terraform did not detect any differences between your configuration and real physical resources that exist. As a result, no actions need to be performed.
i am not sure where am I going wrong i have checked a lot of links some i have listed below.
Your data should be:
data "aws_ami" "ubuntu" {
most_recent = true
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
owners = ["099720109477"]
}
output "test" {
value = data.aws_ami.ubuntu
}
The owner of Ubuntu is not AWS, and the image is ubuntu-focal-20.04-amd64-server-, not ubuntu-xenial-20.08-amd64-server-.
The above results in (us-east-1):
{
"architecture" = "x86_64"
"arn" = "arn:aws:ec2:us-east-1::image/ami-0dba2cb6798deb6d8"
"block_device_mappings" = [
{
"device_name" = "/dev/sda1"
"ebs" = {
"delete_on_termination" = "true"
"encrypted" = "false"
"iops" = "0"
"snapshot_id" = "snap-0f06f1549ff7327c9"
"volume_size" = "8"
"volume_type" = "gp2"
}
"no_device" = ""
"virtual_name" = ""
},
{
"device_name" = "/dev/sdb"
"ebs" = {}
"no_device" = ""
"virtual_name" = "ephemeral0"
},
{
"device_name" = "/dev/sdc"
"ebs" = {}
"no_device" = ""
"virtual_name" = "ephemeral1"
},
]
"creation_date" = "2020-09-08T00:55:25.000Z"
"description" = "Canonical, Ubuntu, 20.04 LTS, amd64 focal image build on 2020-09-07"
"filter" = [
{
"name" = "name"
"values" = [
"ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*",
]
},
{
"name" = "virtualization-type"
"values" = [
"hvm",
]
},
]
"hypervisor" = "xen"
"id" = "ami-0dba2cb6798deb6d8"
"image_id" = "ami-0dba2cb6798deb6d8"
"image_location" = "099720109477/ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20200907"
"image_type" = "machine"
"most_recent" = true
"name" = "ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20200907"
"owner_id" = "099720109477"
"owners" = [
"099720109477",
]
"product_codes" = []
"public" = true
"root_device_name" = "/dev/sda1"
"root_device_type" = "ebs"
"root_snapshot_id" = "snap-0f06f1549ff7327c9"
"sriov_net_support" = "simple"
"state" = "available"
"state_reason" = {
"code" = "UNSET"
"message" = "UNSET"
}
"tags" = {}
"virtualization_type" = "hvm"
}