Iterating through a list while already using for_each (Terraform 0.12) - amazon-web-services

I'm currently trying to use TF 0.12 to create AWS Organizations accounts. Right now I have a map of accounts with applicable info, here is an example where "Services" is the account name:
accountMap = {
...
Services = {
OU = ["Development", "Production"]
},
...
}
OU refers to the org units the account should be a part of. I'm currently already using for_each to loop through this map of account names, but I'm stuck on how to use the OUs as a suffix, so the org account name would become "Services-Development" and "Services-Production". I have tried similar to the following:
resource "aws_organizations_account" "main" {
for_each = var.ouMap
name = "${each.key}-${var.accountMap["${each.value[*]}"]}"
...
}
However, "name" requires a string and I get an error since I am providing a list of the OUs, but I may want one account to belong to several OUs or just a single OU. So, how can I either convert the list to a string one at a time, while in the same for_each iteration (but for my differing OUs)?
I'm open to other suggestions on best practice to map AWS Org accounts to multiple OUs as I'm still rather new to Terraform.

A local value can be computed using nested for loop in terraform v0.12.
The local value can later be used in resources. This example treats a null resource.
accountMap = {
Services = {
OU = ["Development", "Production"]
}
}
locals {
organization_account = flatten(
[for k, v in var.accountMap: [for v2 in v.OU: "${k}-${v2}"]]
)
}
resource "null_resource" "foo" {
count = length(local.organization_account)
provisioner "local-exec" {
command = "echo ${local.organization_account[count.index]}"
}
}
output "organization_account" {
value = local.organization_account
}

Related

How can terraform loop huge comma separated list counting the first 5 items assigning to dynamic subnet block?

I am trying to create multiple packet mirror resources, using for_each
However in GCP packet mirror policy is restricted to only 5 subnets per policy
Now I am stumped how I can create multiple packet mirror policies referencing lets say from variable mirror_vpc_subnets below
variable "mirror_vpc_subnets" {
description = "Mirror VPC Subnets list to be mirrored."
type = list(string)
default = []
}
Now the objective is to get terraform to loop the huge list in my tfvars below and loop, but cherry picking the 1st 5 then assigning it to the 1st packet mirror resource I want to create called lets say packetmirror1
Then it looks again starting from the next list with appworkstream6-subnet and creates me packetmirror2
Then it looks again starting from the next list with appworkstream11-subnet and creates me packetmirror3
Hope this makes sense...
TFVARS here
mirror_vpc_subnets = [
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream1-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream2-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream3-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream4-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream5-subnet"
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream6-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream7-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream8-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream9-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream10-subnet"
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream11-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream12-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream13-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream14-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream15-subnet"
]
Please advise how this resource can be created in a loop also incrementing the packetmirror name on that creation loop
resource "google_compute_packet_mirroring" "main" {
name = var.packet_mirror_policy_name
project = var.gcp_project_id
region = var.region
network {
url = var.collector_mirror_network_selflink
}
collector_ilb {
url = var.forwarding_rule
}
mirrored_resources {
tags = var.mirrored_tags
dynamic "subnetworks" {
for_each = var.mirror_vpc_subnets
content {
url = subnetworks.value
}
}
dynamic "instances" {
for_each = var.mirror_vpc_instances
content {
url = instances.value
}
}
}

Nesting for_each loops and configuring multiple regions in Python

We have a long(ish) list of ip addresses to trust, for services published by CloudFlare, and rather than asking each team that publishes a service from an account in aws to implement this in their security groups / acls etc. I thought a prefix list would be perfect. I would like to set this up in a central account, that is then shared to all the child accounts across the aws Organisation. Ideally, I would like to avoid declaring more resources than necessary, so I am using for_each loops and a dynamic entry. So far so good.
However, prefix lists are not a global object, meaning they need to be created per region, and Terraform requires that this is set on the Provider level.
Is there a way to have a single resource declaration work with a single local map to dynamically manage all these moving parts?
It seems the fewest steps I can do, is to have a resource declaration per region, and then a local map per provider, which is already making 6 "blocks" for only 3 regions, + 2 lists for ipv4 and ipv6...
Here's the code example:
locals {
# The two ip lists to add to appropriate prefix lists
cloudflare-ips = {
prefixlist_cloudflare_ipv4 = {
ips = [
"10.0.0.0/32",
"173.245.48.0/20",
...
],
type = "IPv4",
},
prefixlist_cloudflare_ipv6 = {
ips = [
"2400:cb00::/32",
"2606:4700::/32",
...
],
type = "IPv6",
}
}
}
# Generate multipel predix lists from the map of ipv4 and ipv6 addresses
resource "aws_ec2_managed_prefix_list" "cloudflare-ipv4" {
for_each = local.cloudflare-ips
name = "cloudflare_${each.value.type}"
# Here is where I have to add a provider for the region, and this cannot be done within any type of loop ?
provider = aws.ap-southeast-1
address_family = each.value.type
max_entries = 50
dynamic "entry" {
for_each = tolist( each.value.ips )
content {
cidr = entry.value
description = "CloudFlare ${entry.key}"
}
}
}
I've tried including a list of strings with region / provider names and calling these directly from inside the loop, but even though it can evaluate a string (e.g. provider = "aws.ap-southeast-1") it does not accept reading from the local map like so:
provider = each.value.region
with error:
│ The provider argument requires a provider type name, optionally followed by a period and then a configuration alias.
I guess the next step would be to make this into a module ? Any other suggestions... ?
this cannot be done within any type of loop
That's correct. You can't have dynamic provider, thus you can't use any loops and variables. Its value must be hardcoded.

How can I use var in resource calling

I'm importing roles which already have been created in AWS console and unfortunately the names are strange. So in order to use those roles I am trying like this
I've two IAM roles as follows
data "aws_iam_role" "reithera-rtcov201" {
name = "exomcloudrosareitherartcov-YRX1M2GJKD6H"
}
data "aws_iam_role" "dompe-rlx0120" {
name = "exomcloudrosadomperlx0120p-1SCGY0RG5JXFF"
}
In this file I have 2 variables as follows:
sponsor = ["reithera", "dompe"]
study = ["rtcov201", "rlx0120"]
I'm trying in the following way, but terraform doesn't allow to use $.
data.aws_iam_role.${var.sponsor}-${var.study}.arn
Do you know any solution for this.
Its not possible. You can dynamically create references to resources.
Instead of two separate data sources you should create one:
variable "iam_roles"
default = ["exomcloudrosareitherartcov-YRX1M2GJKD6H", "exomcloudrosadomperlx0120p-1SCGY0RG5JXFF"]
}
and then
data "aws_iam_role" "role" {
for_each = toset(var.iam_roles)
name = each.key
}
and you can refer to them using role name:
data.aws_iam_role["exomcloudrosareitherartcov-YRX1M2GJKD6H"].arn

How to deploy a Terraform module with many roles?

I'm currently working on an AWS terraform project where I have an array of ROLE IDs (as variables) for different accounts.
variable "slave_account_id" {
default = ["5686435678", "9889865446"]
}
Each of these roles allow a my current AWS account (linked with terraform) to deploy a module on those accounts (assuming for each account the role id)
Thus, I would like to create different providers for each role based on the variable array "slave_account_id".
I tried to do it this way:
provider "aws" {
counter = "${length(var.slave_account_id)}"
alias = "aws-assume-${counter.index}"
region = "eu-west-1"
assume_role {
role_arn = "arn:aws:iam::${var.slave_account_id[counter.index]}:role/slave_role_for_master"
session_name = "${var.slave_session_name[counter.index]}"
external_id = "EXTERNAL_ID"
}
}
This way I would have planned to use this code inside my module:
module "my_super_module" {
counter = "${length(var.slave_account_id)}"
providers = {
aws = "aws.aws-assume-${counter.index}"
}
[...]
}
But this doesn't work (from what I understood I cannot 'concatenate" variable inside the alias of a provider because provider has to be defined before we can interpolate).
Here is the execution result (error du to alias section of the provider):
Error: Invalid provider configuration alias
An alias must be a valid name. A name must start with a letter and may contain
only letters, digits, underscores, and dashes.
Error: Duplicate provider configuration
on main.tf line 5:
5: provider "aws" {
A default (non-aliased) provider configuration for "aws" was already given at
main.tf:1,1-15. If multiple configurations are required, set the "alias"
argument for alternative configurations.
Error: Unsuitable value type
on main.tf line 8, in provider "aws":
8: alias = "aws-assume-${counter.index}"
Unsuitable value: value must be known
Error: Variables not allowed
on main.tf line 8, in provider "aws":
8: alias = "aws-assume-${counter.index}"
Variables may not be used here.
Error: Invalid provider configuration reference
on main.tf line 33, in module "my-lambda":
33: aws = "aws.aws-assume-${counter.index}"
A provider configuration reference must not be given in quotes.
Hence I am a bit lost...
How to deploy a module with a list of role ids (one module for each account) ?
Provider configurations in Terraform are not dynamically-constructable (that is, to decide which to create based on a value) because Terraform needs to associate providers with resources very early in the lifecycle, during graph construction and before expression evaluation is possible.
Instead, we can refactor the problem so that each module takes a fixed number of AWS providers (most often one, but in some cases two if the module's purpose is e.g. to set up peering between two regions or two accounts) and then instantiate the module multiple times in the root:
provider "aws" {
alias = "eu-west-1_5686435678"
region = "eu-west-1"
assume_role {
role_arn = "arn:aws:iam::acct5686435678:role/admin"
session_name = "whatever_session_name"
external_id = "EXTERNAL_ID"
}
}
provider "aws" {
alias = "eu-west-1_9889865446"
region = "eu-west-1"
assume_role {
role_arn = "arn:aws:iam::acct9889865446:role/admin"
session_name = "whatever_session_name"
external_id = "EXTERNAL_ID"
}
}
module "acct5686435678" {
source = "./modules/aws-account"
providers = {
aws = aws.eu-west-1_5686435678
}
}
module "acct9889865446" {
source = "./modules/aws-account"
providers = {
aws = aws.eu-west-1_9889865446
}
}
module "peering_5686435678_9889865446" {
source = "./modules/aws-account-peering"
providers = {
aws.from = aws.eu-west-1_5686435678
aws.to = aws.eu-west-1_9889865446
}
}
Instantiating the same module multiple times is a common technique for situations where the same infrastructure must be created over multiple AWS accounts or over multiple AWS regions.
With that said, if the multiple AWS accounts are representing separate environments rather than separate components within an environment, it's often preferable to use a separate root configuration per environment while still sharing modules, so that updates to each environment are entirely separated, each environment has its own state, etc.

Preventing destroy of resources when refactoring Terraform to use indices

When I was just starting to use Terraform, I more or less naively declared resources individually, like this:
resource "aws_cloudwatch_log_group" "image1_log" {
name = "${var.image1}-log-group"
tags = module.tagging.tags
}
resource "aws_cloudwatch_log_group" "image2_log" {
name = "${var.image2}-log-group"
tags = module.tagging.tags
}
resource "aws_cloudwatch_log_stream" "image1_stream" {
name = "${var.image1}-log-stream"
log_group_name = aws_cloudwatch_log_group.image1_log.name
}
resource "aws_cloudwatch_log_stream" "image2_stream" {
name = "${var.image2}-log-stream"
log_group_name = aws_cloudwatch_log_group.image2_log.name
}
Then, 10-20 different log groups later, I realized this wasn't going to work well as infrastructure grew. I decided to define a variable list:
variable "image_names" {
type = list(string)
default = [
"image1",
"image2"
]
}
Then I replaced the resources using indices:
resource "aws_cloudwatch_log_group" "service-log-groups" {
name = "${element(var.image_names, count.index)}-log-group"
count = length(var.image_names)
tags = module.tagging.tags
}
resource "aws_cloudwatch_log_stream" "service-log-streams" {
name = "${element(var.image_names, count.index)}-log-stream"
log_group_name = aws_cloudwatch_log_group.service-log-groups[count.index].name
count = length(var.image_names)
}
The problem here is that when I run terraform apply, I get 4 resources to add, 4 resources to destroy. I tested this with an old log group, and saw that all my logs were wiped (obviously, since the log was destroyed).
The names and other attributes of the log groups/streams are identical- I'm simply refactoring the infrastructure code to be more maintainable. How can I maintain my existing log groups without deleting them yet still refactor my code to use lists?
You'll need to move the existing resources within the Terraform state.
Try running terraform show to get the strings under which the resources are stored, this will be something like [module.xyz.]aws_cloudwatch_log_group.image1_log ...
You can move it with terraform state mv [module.xyz.]aws_cloudwatch_log_group.image1_log '[module.xyz.]aws_cloudwatch_log_group.service-log-groups[0]'.
You can choose which index to assign to each resource by changing [0] accordingly.
Delete the old resource definition for each moved resource, as Terraform would otherwise try to create a new group/stream.
Try it with the first import and check with terraform plan if the resource was moved correctly...
Also check if you need to choose some index for the image_names list jsut to be sure, but I think that won't be necessary.