Nesting for_each loops and configuring multiple regions in Python - amazon-web-services

We have a long(ish) list of ip addresses to trust, for services published by CloudFlare, and rather than asking each team that publishes a service from an account in aws to implement this in their security groups / acls etc. I thought a prefix list would be perfect. I would like to set this up in a central account, that is then shared to all the child accounts across the aws Organisation. Ideally, I would like to avoid declaring more resources than necessary, so I am using for_each loops and a dynamic entry. So far so good.
However, prefix lists are not a global object, meaning they need to be created per region, and Terraform requires that this is set on the Provider level.
Is there a way to have a single resource declaration work with a single local map to dynamically manage all these moving parts?
It seems the fewest steps I can do, is to have a resource declaration per region, and then a local map per provider, which is already making 6 "blocks" for only 3 regions, + 2 lists for ipv4 and ipv6...
Here's the code example:
locals {
# The two ip lists to add to appropriate prefix lists
cloudflare-ips = {
prefixlist_cloudflare_ipv4 = {
ips = [
"10.0.0.0/32",
"173.245.48.0/20",
...
],
type = "IPv4",
},
prefixlist_cloudflare_ipv6 = {
ips = [
"2400:cb00::/32",
"2606:4700::/32",
...
],
type = "IPv6",
}
}
}
# Generate multipel predix lists from the map of ipv4 and ipv6 addresses
resource "aws_ec2_managed_prefix_list" "cloudflare-ipv4" {
for_each = local.cloudflare-ips
name = "cloudflare_${each.value.type}"
# Here is where I have to add a provider for the region, and this cannot be done within any type of loop ?
provider = aws.ap-southeast-1
address_family = each.value.type
max_entries = 50
dynamic "entry" {
for_each = tolist( each.value.ips )
content {
cidr = entry.value
description = "CloudFlare ${entry.key}"
}
}
}
I've tried including a list of strings with region / provider names and calling these directly from inside the loop, but even though it can evaluate a string (e.g. provider = "aws.ap-southeast-1") it does not accept reading from the local map like so:
provider = each.value.region
with error:
│ The provider argument requires a provider type name, optionally followed by a period and then a configuration alias.
I guess the next step would be to make this into a module ? Any other suggestions... ?

this cannot be done within any type of loop
That's correct. You can't have dynamic provider, thus you can't use any loops and variables. Its value must be hardcoded.

Related

How can terraform loop huge comma separated list counting the first 5 items assigning to dynamic subnet block?

I am trying to create multiple packet mirror resources, using for_each
However in GCP packet mirror policy is restricted to only 5 subnets per policy
Now I am stumped how I can create multiple packet mirror policies referencing lets say from variable mirror_vpc_subnets below
variable "mirror_vpc_subnets" {
description = "Mirror VPC Subnets list to be mirrored."
type = list(string)
default = []
}
Now the objective is to get terraform to loop the huge list in my tfvars below and loop, but cherry picking the 1st 5 then assigning it to the 1st packet mirror resource I want to create called lets say packetmirror1
Then it looks again starting from the next list with appworkstream6-subnet and creates me packetmirror2
Then it looks again starting from the next list with appworkstream11-subnet and creates me packetmirror3
Hope this makes sense...
TFVARS here
mirror_vpc_subnets = [
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream1-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream2-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream3-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream4-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream5-subnet"
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream6-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream7-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream8-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream9-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream10-subnet"
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream11-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream12-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream13-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream14-subnet",
"projects/gcp_project_name/regions/europe-west2/subnetworks/appworkstream15-subnet"
]
Please advise how this resource can be created in a loop also incrementing the packetmirror name on that creation loop
resource "google_compute_packet_mirroring" "main" {
name = var.packet_mirror_policy_name
project = var.gcp_project_id
region = var.region
network {
url = var.collector_mirror_network_selflink
}
collector_ilb {
url = var.forwarding_rule
}
mirrored_resources {
tags = var.mirrored_tags
dynamic "subnetworks" {
for_each = var.mirror_vpc_subnets
content {
url = subnetworks.value
}
}
dynamic "instances" {
for_each = var.mirror_vpc_instances
content {
url = instances.value
}
}
}

Terraform Resource attribute not being removed when passing in empty values

I am working with a GCP Cloud Composer resource and added in a dynamic block to create an attribute for the resource to set allowed_ip_ranges which can be used as an IP filter for accessing the Apache Airflow Web UI.
I was able to get the allowed ranges setup and can update them in place to new values also.
If I attempt to pass in a blank list I am expecting the IP address(es) to be removed as attributes for the resource but Terraform seems to think that no changes are needed.
There is probably something wrong in my code but I am not sure what exactly I would need to do. Does it involve adding in a conditional expression to the for_each loop in the dynamic block?
Child module main.tf
web_server_network_access_control {
dynamic "allowed_ip_range" {
for_each = var.allowed_ip_range
content {
value = allowed_ip_range.value["value"]
description = allowed_ip_range.value["description"]
}
}
}
Child module variables.tf
variable "allowed_ip_range" {
description = "The IP ranges which are allowed to access the Apache Airflow Web Server UI."
type = list(map(string))
default = []
}
Parent module terraform.tfvars
allowed_ip_range = [
{
value = "11.0.0.2/32"
description = "Test dynamic block 1"
},
]
You can set the default value in your variables.tf file:
variable "allowed_ip_range" {
description = "The IP ranges which are allowed to access the Apache Airflow Web Server UI"
type = list(map(string))
default = [
{
value = "0.0.0.0/0"
description = "Allows access from all IPv4 addresses (default value)"
},
{
value = "::0/0"
description = "Allows access from all IPv6 addresses (default value)"
},
]
}
And when you will delete your variable from terraform.tfvars, you will have the default values

share multiple DNS domain with multiple aws accounts by terraform in AWS Resource Access Manager

I'm forwarding DNS requests sent to a list of internal domains (on premise) by using AWS Route53 resolver. By terraform, I want to share the rules I created to other accounts of the company, so I have the following:
# I create as much share endpoint as domain I have, so If I have 30 domains, I'll make 30 endpoint RAM:
resource "aws_ram_resource_share" "endpoint_share" {
count = length(var.forward_domain)
name = "route53-${var.forward_domain[count.index]}-share"
allow_external_principals = false
}
# Here I share every single endpoint with all the AWS ACcount we have
resource "aws_ram_principal_association" "endpoint_ram_principal" {
count = length(var.resource_share_accounts)
principal = var.resource_share_accounts[count.index]
resource_share_arn = {
for item in aws_ram_resource_share.endpoint_share[*]:
item.arn
}
}
The last block, calls the arn output of the first one which is a list.
Now, this last block doesn't work, I don't know how to use multiple counts, when I run this, I get the following error:
Error: Invalid 'for' expression
line 37: Key expression is required when building an object.
Any idea how to make this work?
Terraform version: 0.12.23
Use square brackets in resource_share_arn, like this:
resource_share_arn = [
for item in aws_ram_resource_share.endpoint_share[*]:
item.arn
]

Iterating through a list while already using for_each (Terraform 0.12)

I'm currently trying to use TF 0.12 to create AWS Organizations accounts. Right now I have a map of accounts with applicable info, here is an example where "Services" is the account name:
accountMap = {
...
Services = {
OU = ["Development", "Production"]
},
...
}
OU refers to the org units the account should be a part of. I'm currently already using for_each to loop through this map of account names, but I'm stuck on how to use the OUs as a suffix, so the org account name would become "Services-Development" and "Services-Production". I have tried similar to the following:
resource "aws_organizations_account" "main" {
for_each = var.ouMap
name = "${each.key}-${var.accountMap["${each.value[*]}"]}"
...
}
However, "name" requires a string and I get an error since I am providing a list of the OUs, but I may want one account to belong to several OUs or just a single OU. So, how can I either convert the list to a string one at a time, while in the same for_each iteration (but for my differing OUs)?
I'm open to other suggestions on best practice to map AWS Org accounts to multiple OUs as I'm still rather new to Terraform.
A local value can be computed using nested for loop in terraform v0.12.
The local value can later be used in resources. This example treats a null resource.
accountMap = {
Services = {
OU = ["Development", "Production"]
}
}
locals {
organization_account = flatten(
[for k, v in var.accountMap: [for v2 in v.OU: "${k}-${v2}"]]
)
}
resource "null_resource" "foo" {
count = length(local.organization_account)
provisioner "local-exec" {
command = "echo ${local.organization_account[count.index]}"
}
}
output "organization_account" {
value = local.organization_account
}

Setting "count" based on the length of an attribute on another resource

I have a fairly simple Terraform configuration, which creates a Route53 zone and then creates NS records in Cloudflare to delegate the subdomain to that zone. At present, it assumes there's always exactly four authoritative DNS servers for every Route53 zone, and creates four separate cloudflare_record resources, but I'd like to generalise that, partially because who knows if AWS will start putting a fifth authoritative server out there in the future, but also as a "test case" for more complicated stuff in the future (like AWS AZs, which I know vary in count between regions).
What I've come up with so far is:
resource "cloudflare_record" "public-zone-ns" {
domain = "example.com"
name = "${terraform.env}"
type = "NS"
ttl = "120"
count = "${length(aws_route53_zone.public-zone.name_servers)}"
value = "${lookup(aws_route53_zone.public-zone.name_servers, count.index)}"
}
resource "aws_route53_zone" "public-zone" {
name = "${terraform.env}.example.com"
}
When I run terraform plan over this, though, I get this error:
Error running plan: 1 error(s) occurred:
* cloudflare_record.public-zone-ns: cloudflare_record.public-zone-ns: value of 'count' cannot be computed
I think what that means is that because the aws_route53_zone hasn't actually be created, terraform doesn't know what length(aws_route53_zone.public-zone.name_servers) is, and therefore the interpolation into cloudflare_record.public-zone-ns.count fails and I'm screwed.
However, it seems surprising to me that Terraform would be so inflexible; surely being able to create a variable number of resources like this would be meat-and-potatoes stuff. Hard-coding the length, or creating separate resources, just seems so... limiting.
So, what am I missing? How can I create a number of resources when I don't know in advance how many I need?
Currently count not being able to be calculated is an open issue in terraform https://github.com/hashicorp/terraform/issues/12570
You could move the name servers to a variable array and then get the length of that, all in one terraform script.
I got your point, this surprised me as well. Even I added depends_on to resource cloudflare_record, it is helpless.
What you can do to pass this issue is to split it into two stacks and make sure the route 53 record is created before cloudflare record.
Stack #1
resource "aws_route53_zone" "public-zone" {
name = "${terraform.env}.example.com"
}
output "name_servers" {
value = "${aws_route53_zone.public-zone.name_servers}"
}
Stack #2
data "terraform_remote_state" "route53" {
backend = "s3"
config {
bucket = "terraform-state-prod"
key = "network/terraform.tfstate"
region = "us-east-1"
}
}
resource "cloudflare_record" "public-zone-ns" {
domain = "example.com"
name = "${terraform.env}"
type = "NS"
ttl = "120"
count = "${length(data.terraform_remote_state.route53.name_servers)}"
value = "${element(data.terraform_remote_state.route53.name_servers, count.index)}"
}