Terrafrom dynamic block with dynamic content - amazon-web-services

I am trying to create a terraform module for aws_route_table creation, here is an example of this resource definition:
resource "aws_route_table" "example" {
vpc_id = aws_vpc.example.id
route {
cidr_block = "10.0.1.0/24"
gateway_id = aws_internet_gateway.example.id
}
route {
ipv6_cidr_block = "::/0"
egress_only_gateway_id = aws_egress_only_internet_gateway.example.id
}
tags = {
Name = "example"
}
}
I am trying to make it more dynamic by using dynamic blocks. The problem is that I always have to define the keys in content block
resource "aws_route_table" "example" {
...
dynamic "route" {
for_each = var.route
content {
cidr_block = route.value.cidr_block
gateway_id = route.value.gateway_id
}
}
...
}
So in this case, I will need to write two dynamic blocks, one for the content with cidr_block and gateway_id and one for the content with ipv6_cidr_block and egress_only_gateway_id.
Is there any way to do this without defining keys explicitly. Something like this:
dynamic "route" {
for_each = var.route
content {
var.route.map
}
}

Yes, you can create route dynamically, because block route acts as Attributes as Blocks. So you can do (example)
# define all your routes in a variable (example content)
variable "routes" {
default = [
{
cidr_block = "0.0.0.0/0"
gateway_id = "igw-0377483faa64bf010"
},
{
cidr_block = "172.31.0.0/20"
instance_id = "i-043fc97db72ad1b59"
}
]
}
# need to provide default values (null) for all possibilities
# in route
locals {
routes_helper = [
for route in var.routes: merge({
carrier_gateway_id = null
destination_prefix_list_id = null
egress_only_gateway_id = null
ipv6_cidr_block = null
local_gateway_id = null
nat_gateway_id = null
network_interface_id = null
transit_gateway_id = null
vpc_endpoint_id = null
instance_id = null
gateway_id = null
vpc_peering_connection_id = null
}, route)
]
}
resource "aws_route_table" "example" {
vpc_id = aws_vpc.example.id
# route can be attribute, instead of blocks
route = local.routes_helper
tags = {
Name = "example"
}
}
Docs do not recommend to use that in general, but I think route is a good example where this would be acceptable.

The general answer for a block type that doesn't employ the "Attributes as Blocks" backward-compatibility shim Marcin mentioned is to rely on the fact that for Terraform provider arguments there is no difference in meaning between omitting an argument or setting it explicitly to null.
That means that you can write an expression to decide whether to populate a particular argument or not, by making it return an explicit null in the appropriate cases. For example:
dynamic "route" {
for_each = var.route
content {
cidr_block = try(route.value.cidr_block, null)
gateway_id = try(route.value.gateway_id, null)
ipv6_cidr_block = try(route.value.ipv6_cidr_block, null)
# ...
}
}
A nested block like this is more like a fixed data structure than a collection, so there isn't any syntax to construct it dynamically.
With that said, in this particular case route is, as Marcin noted, actually really just an attribute of an object type, and Terraform Core is allowing to use it with block syntax as a concession to backward compatibility with older Terraform versions. For that reason, there's not really any significant difference between these two approaches in practice, but the "Attributes as Blocks" mechanism is only for certain situations where providers were design to assume Terraform v0.11 validation bugs and so the conditional null approach I described above is the more general answer, which should work for normal nested blocks too.

Related

Dynamic routes in Terraform

My simplified setup is something like this:
2 AZs and each consists of 2 subnets (Subnet1-AZ-A, Subnet2-AZ-A, Subnet1-AZ-B, Subnet2-AZ-B)
I want to create routing table that will fetch CIDR blocks from Subnet1 (from both AZs) and create routes:
CIDR_BLOCK of Subnet-1-AZ-A -> VPC_ENDPOINT in Subnet-2-AZ-A
CIDR_BLOCK of Subnet 1-AZ-B -> VPC_ENDPOINT in Subnet-2-AZ-B
I don't want to hardcode values, because number of AZs (and subnets) may change.
Here's code that I'm using:
Subnet-1A
resource "aws_subnet" "Subnet1" {
count = length(var.az)
vpc_id = aws_vpc.app-vpc.id
cidr_block = cidrsubnet(var.vpc_cidr, 8, 4+count.index)
availability_zone = var.az[count.index]
}
VPC_Endpoints:
resource "aws_vpc_endpoint" "endpoint-spoke" {
count = length(var.az)
service_name = var.glbe_service_name
subnet_ids = [aws_subnet.endpoint-subnet[count.index].id]
vpc_endpoint_type = var.glbe_endpoint_type
vpc_id = aws_vpc.app-vpc.id
}
Route-Table
resource "aws_route_table" "IGWRT" {
vpc_id = aws_vpc.app-vpc.id
dynamic "route" {
for_each = aws_subnet.Subnet1[*].cidr_block
content {
cidr_block = route.value
vpc_endpoint_id = aws_vpc_endpoint.endpoint-spoke[*].id
}
}
}
The code in Route-Table is not working. I know aws_subnet.Subnet1[*].cidr_block will be a list that contains Subnet1 CIDR blocks for example: [10.0.0.0/20, 10.0.1.0/24], but how I can create dynamic routes where:
10.0.0.0/0 will be pointing to VPC_Endpoint in respective AZ
10.0.0.1/0 will be pointing to VPC_Endpoint in respective AZ
Update the dynamic block in the aws_route_table resource:
resource "aws_route_table" "IGWRT" {
vpc_id = aws_vpc.app-vpc.id
dynamic "route" {
for_each = aws_subnet.Subnet1
content {
cidr_block = route.value.cidr_block
vpc_endpoint_id = aws_vpc_endpoint.endpoint-spoke[route.key].id
}
}
}
However, I strongly recommend you to use a for_each instead of the count, if the elements of the list change (the variable az), it will affect all of the resources as their index will be updated.

How Do I Use A Terraform Data Source To Reference A Managed Prefix List?

I'm trying to update a terraform module to add a new security group, which will have an inbound rule populated with two managed prefix lists. The prefix lists are shared to my AWS account from a different account using AWS Resource Access Manager, however I have tried referencing prefix lists created within my own AWS account and am seeing the same error.
Below is the terraform I am using:
resource "aws_security_group" "akamai_sg" {
name = "akamai-pl-sg"
description = "Manage access from Akamai to ${var.environment} alb"
vpc_id = var.vpc_id
tags = merge(var.common_tags, tomap({ "Name" = "akamai-pl-sg" }))
revoke_rules_on_delete = true
}
resource "aws_security_group_rule" "akamai_to_internal_alb" {
for_each = toset(var.domains_inc_akamai)
type = "ingress"
description = "Allow Akamai into ${var.environment}${var.domain_name_suffix}-alb"
from_port = var.alb_listener_port
to_port = var.alb_listener_port
protocol = "tcp"
security_group_id = aws_security_group.akamai_sg.id
prefix_list_ids = [data.aws_prefix_list.akamai-site-shield.id, data.aws_prefix_list.akamai-staging.id]
}
data "aws_prefix_list" "akamai-site-shield" {
filter {
name = "prefix-list-id"
values = ["pl-xxxxxxxxxx"]
}
}
data "aws_prefix_list" "akamai-staging" {
filter {
name = "prefix-list-id"
values = ["pl-xxxxxxxxxx"]
}
}
The terraform error I am revieving reads:
"Error: no matching prefix list found; the prefix list ID or name may be invalid or not exist in the current region"
Is anyone able to help, or see where I am going wrong?
Thanks in advance.
Would not be the following possible?
data "aws_vpc_endpoint" "s3" {
vpc_id = aws_vpc.foo.id
service_name = "com.amazonaws.us-west-2.s3"
}
data "aws_prefix_list" "s3" {
prefix_list_id = aws_vpc_endpoint.s3.prefix_list_id
}
It seems the solution is to use:
data "aws_ec2_managed_prefix_list" "example" {
filter {
name = "prefix-list-name"
values = ["my-prefix-list"]
}
}

For_each and count in same terraform AWS resource

I'm trying to create "aws_route" in terraform iterating over a list of route_tables with vpc_peering of another service. The other service vpc destination_cidr_block is a list.
variable "route_tables" {
type = set(string)
description = "Set of route table entries eg : rt-1, rt-2 , rt-3"
}
variable "ext_service_destination_cidr_blocks"{
type = list(string)
description = "list of destination cidr blocks of external service, eg:[\"10.10.1.1/20\", \"10.2.10.1/10\"]"
}
resource "aws_route" "ext_service_route" {
// iterating over route tables [ rt-1, rt-2 , rt-3 ]
for_each = var.route_tables
route_table_id = each.key
// Iterating over cidr list
count = var.ext_service_destination_cidr_blocks
destination_cidr_block = var.ext_service_destination_cidr_blocks[count.index]
vpc_peering_connection_id = var.ext_service_peering_connection_id
}
Here, I would like to iterate over list of destination_cidr_block.
Simply put, I need a nested loop, count inside for_each.
I can't have both count and for_each in the same block, is there any workaround for this?
Or is there any way I can split this into two modules?
We can use setproduct to compute the Cartesian product of those two collections and create a map based on that. This map can be used to do for_each on it:
resource "aws_route" "ext_service_route" {
for_each = { for i, pair in tolist(setproduct(var.route_tables, var.ext_service_destination_cidr_blocks)) : "route-${i}" => { "name" : pair[0], "cidr" : pair[1] } }
route_table_id = each.value.name
destination_cidr_block = each.value.cidr
vpc_peering_connection_id = var.ext_service_peering_connection_id
}
You can also use mod operation to iterate over the two lists simultaneously (for example, when you are using older TF version that does not support for_each yet):
resource "aws_route" "ext_service_route" {
count = length(var.ext_service_destination_cidr_blocks) * length(var.route_tables)
route_table_id = element(var.route_tables, count.index + 1 % length(var.route_tables))
destination_cidr_block = element(var.ext_service_destination_cidr_blocks, count.index + 1 % length(var.ext_service_destination_cidr_blocks))
vpc_peering_connection_id = var.ext_service_peering_connection_id
}

Terraform how to fix attribute "route": set of object required

I have this terraform module route_table.tf I need to use. It looks like below:
resource "aws_route_table" "aws_route_table" {
# route {
# cidr_block = "0.0.0.0/0"
# gateway_id = var.GATEWAY_ID
# }
route = var.ROUTE
tags = var.ROUTE_TABLE_TAGS
vpc_id = var.VPC_ID
}
and I have defined the variable ROUTE as below in the inputs.tf:
variable "ROUTE" {
type = object({ cidr_block=string, gateway_id=string })
}
And I am passing those values in the main.tf as below:
module "route_tables_public" {
source = "./modules/route_tables"
ROUTE = {
cidr_block = "0.0.0.0/0"
gateway_id = var.GATEWAY_ID
}
ROUTE_TABLE_TAGS = { "Name" : "mlb-rt-public" , "Project" : "mlb"}
VPC_ID = module.ecs_vpc.vpc_id
}
But I am getting this error:
Inappropriate value for attribute "route": set of object required.
Can someone help me on this?
Your var.ROUTE is a single object, but it should be list of objects. So you can try:
variable "ROUTE" {
type = list(object({ cidr_block=string, gateway_id=string }))
}
and then
module "route_tables_public" {
source = "./modules/route_tables"
ROUTE = [{
cidr_block = "0.0.0.0/0"
gateway_id = var.GATEWAY_ID
}]
ROUTE_TABLE_TAGS = { "Name" : "mlb-rt-public" , "Project" : "mlb"}
VPC_ID = module.ecs_vpc.vpc_id
}
UPDATE
Your aws_route_table should be:
resource "aws_route_table" "aws_route_table" {
dynamic "route" {
for_each = var.ROUTE
content {
cidr_block = route.value.cidr_block
gateway_id = route.value.gateway_id
}
}
tags = var.ROUTE_TABLE_TAGS
vpc_id = var.VPC_ID
}
I would recommend against directly assigning variable values to complex resource type arguments like this, because if the schema of the route argument were to grow to include additional attributes in the future then your module would fail validation.
Instead, better to let the variable's type be independent of the route block schema and translate explicitly between them, which should then allow this to work as long as a new version of the provider doesn't introduce a new required argument for that block.
variable "route" {
type = object({
cidr_block = string
gateway_id = string
})
}
resource "aws_route_table" "aws_route_table" {
# ...
route {
cidr_block = var.route.cidr_block
gateway_id = var.route.gateway_id
}
}
This can work because it's constructing a new object that conforms to the route block schema using two attributes of your object, whereas your example fails because you tried to assign the input object directly to that argument, which would require that you match the target type exactly.
Please note that naming variables all in uppercase is not idiomatic Terraform style, so in the above example I renamed the variable to be route instead. ROUTE would also work, but would be an unusual way to name a Terraform variable.

Terraform - how to use for_each loop on a list of objects to create resources

I have an object containing the list of subnets I want to create.
variable "subnet-map" {
default = {
ec2 = [
{
cidr_block = "10.0.1.0/24"
availability_zone = "eu-west-1a"
}
],
lambda = [
{
cidr_block = "10.0.5.0/24"
availability_zone = "eu-west-1a"
},
{
cidr_block = "10.0.6.0/24"
availability_zone = "eu-west-1b"
},
{
cidr_block = "10.0.7.0/24"
availability_zone = "eu-west-1c"
}
],
secrets = [
{
cidr_block = "10.0.8.0/24"
availability_zone = "eu-west-1a"
},
{
cidr_block = "10.0.9.0/24"
availability_zone = "eu-west-1b"
},
{
cidr_block = "10.0.10.0/24"
availability_zone = "eu-west-1c"
}
],
rds = [
{
cidr_block = "10.0.11.0/24"
availability_zone = "eu-west-1a"
},
{
cidr_block = "10.0.12.0/24"
availability_zone = "eu-west-1b"
},
{
cidr_block = "10.0.13.0/24"
availability_zone = "eu-west-1c"
}
]
}
}
Earlier I was using the count loop construct. So I used to flatten the above structure into a list of objects
locals {
subnets = flatten([
for resource in keys(var.subnet-map) : [
for subnet in var.subnet-map[resource] : {
resource = resource
cidr_block = subnet.cidr_block
availability_zone = subnet.availability_zone
}
]
])
}
And then I would create the resources by doing
resource "aws_subnet" "aws-subnets" {
count = length(local.subnets)
vpc_id = aws_vpc.aws-vpc.id
cidr_block = local.subnets[count.index].cidr_block
availability_zone = local.subnets[count.index].availability_zone
tags = {
Name = "subnet-${local.subnets[count.index].resource}-${local.subnets[count.index].availability_zone}"
}
}
Now I want to use the for_each loop. But I cannot figure out how to do it. This is what I've done so far.
resource "aws_subnet" "subnets-dev" {
for_each = var.subnet-map
vpc_id = aws_vpc.vpc-dev.id
cidr_block = each.value.cidr_block
availability_zone = each.value.availability_zone
tags = {
Name = "subnet-dev-${each.value.resource}-${each.value.availability_zone}"
environment = "dev"
}
}
But it keeps giving an error saying
Error: Unsupported attribute
on vpc/main.tf line 93, in resource "aws_subnet" "subnets-dev":
93: Name = "subnet-dev-${each.value.resource}-${each.value.availability_zone}"
|----------------
| each.value is tuple with 3 elements
This value does not have any attributes.
How could I fix this?
I'm not sure I fully follow all of what you tried here because your initial snippet of var.subnet-map shows it being a map of maps of lists of objects, but later on when you used for_each = var.subnet-map it seems to have treated it as a map of lists instead. Did you remove that extra level of maps (the "default" key) before trying for_each here?
Working with your original definition of variable "subnet-map", your first step with for_each will be similar to what you did with count: you need to flatten the structure, this time into a map of objects rather than a list of objects. The easiest way to get there is to derive a map from your existing flattened list:
locals {
subnets = flatten([
for resource in keys(var.subnet-map) : [
for subnet in var.subnet-map[resource] : {
resource = resource
cidr_block = subnet.cidr_block
availability_zone = subnet.availability_zone
}
]
])
subnets_map = {
for s in local.subnets: "${s.resource}:${s.availability_zone}" => s
}
}
Here I assumed that your "resource" string and your availability zone together are a suitable unique identifier for a subnet. If not, you can adjust the "${s.resource}:${s.availability_zone}" part to whatever unique key you want to use for these.
Now you can use the flattened map as the for_each map:
resource "aws_subnet" "subnets-dev" {
for_each = local.subnets_map
vpc_id = aws_vpc.vpc-dev.id
cidr_block = each.value.cidr_block
availability_zone = each.value.availability_zone
tags = {
Name = "subnet-dev-${each.value.resource}-${each.value.availability_zone}"
environment = "dev"
}
}
This will give you instances with addresses like aws_subnet.subnets-dev["ec2:eu-west-1a"].
Note that if you are migrating from count with existing subnets that you wish to retain, you'll need to also do a one-time migration step to tell Terraform which indexes from the existing state correspond to which keys in the new configuration.
For example, if (and only if) index 0 was previously the one for ec2 in eu-west-1a, the migration command for that one would be:
terraform state mv 'aws_subnet.subnets-dev[0]' 'aws_subnet.subnets-dev["ec2:eu-west-1a"]'
If you're not sure how they correlate, you can run terraform plan after adding for_each and look at the instances that Terraform is planning to destroy. If you work through each one of those in turn, taking the address Terraform currently knows along with the resource and availability zone names shown in the Name tag, you can migrate each of them to its new address so that Terraform will no longer think you're asking for it to destroy the numbered instances and replace them with named ones.