How to grab two arbitrary subnet IDs from Terraform - amazon-web-services

Looking to deploy a Fargate application where the invoking function runs the ECS container in two arbitrarily selected subnets from the default VPC.
So far my template looks like this:
data "aws_subnets" "subnets" {
filter {
name = "vpc-id"
values = [var.vpc_id]
}
}
data "aws_subnet" "subnet" {
for_each = toset(data.aws_subnets.subnets.ids)
id = each.value
}
resource "aws_lambda_function" "ecs_invoker" {
function_name = "ecs_invoker"
...
environment {
variables = {
SUBNET_PRIMARY = data.aws_subnet.subnet[0]
SUBNET_SECONDARY = data.aws_subnet.subnet[1]
}
}
}
However, this produces the following error:
│ Error: Invalid index
│
│ on lambda.tf line 16, in resource "aws_lambda_function" "ecs_invoker":
│ 16: SUBNET_PRIMARY = data.aws_subnet.subnet[0]
│ ├────────────────
│ │ data.aws_subnet.subnet is object with 6 attributes
│
│ The given key does not identify an element in this collection value. An object only supports looking up attributes by name, not by numeric index.
╵
So how exactly should I grab two arbitrary subnet IDs from the default VPC?

Since you used for_each, data.aws_subnet.subnet will be a map, not a list. So to get two first subnet ids, you can do:
SUBNET_PRIMARY = values(data.aws_subnet.subnet)[0].id
SUBNET_SECONDARY = values(data.aws_subnet.subnet)[1].id
To get two random ids, you can do:
resource "random_shuffle" "subnets" {
input = values(data.aws_subnet.subnet)[*].id
result_count = 2
}
and then
SUBNET_PRIMARY = random_shuffle.subnets.result[0]
SUBNET_SECONDARY = random_shuffle.subnets.result[1]

Related

Add tag to launch template for ec2 nodes

I am trying to add tags to a launch template so that the ec2 nodes are tagged and named..
When I add the hardcoded tags inside the module it works, but the idea is to have dynamic tags and be able to merge with the local ones.
module
resource "aws_autoscaling_group" "ecs_asg" {
name = var.name_asg
max_size = var.max_size
min_size = var.min_size
.
.
.
service_linked_role_arn = var.service_linked_role_arn
tags = var.asg_tags
launch_template {
id = aws_launch_template.launch_template.id
version = "$Latest"
}
}
variables.tf
variable "asg_tags" {
type = map(string)
default = {}
}
main.tf
name_asg = "leo-nombre-asg"
max_size = var.max_size
min_size = var.min_size
.
.
.
asg_tags = merge(
local.tags,
{
propagate_at_launch=true,
},
)
locals.tf
locals {
tags = {
"Accountable" = "business"
"Deploy" = "terraform"
"Role" = "services"
}
}
terraform validate
│ Error: Incorrect attribute value type
│
│ on modules\ecs\main.tf line 38, in resource "aws_autoscaling_group" "ecs_asg":
│ 38: tags = var.asg_tags
│ ├────────────────
│ │ var.asg_tags is a map of string
│
│ Inappropriate value for attribute "tags": set of map of string required.
The two fixes necessary here are both for the type in the asg_tags parameter argument value:
asg_tags = [merge(local.tags, { "propagate_at_launch" = "true" })]
Here we use the list/set constructor to cast the type set(map(string)). Terraform will coerce to set instead of list with the constructor as long as the type is specified to be set instead. Since we need to fix the type declaration anyway to be compatible with the resource attribute schema, this is convenient to do:
variable "asg_tags" {
type = set(map(string))
default = {}
}

Iterate On Module Instances Output When Declared With `for_each`

Description
I'm attempting to iterate on the results of a terraform module and build a set of AWS glue crawlers and IAM roles based on how many source S3 buckets are set in the terraform module
I'm specifically trying to use this line
for_each = {for key, value in setproduct(local.bucket-names, module.crawler_source_buckets.id): key=>value}
or this line
for_each = {for key, value in setproduct(local.bucket-names, module.crawler_source_buckets.*.id): key=>value}
or this line
for_each = {for key, value in setproduct(local.bucket-names, module.crawler_source_buckets.[*].id): key=>value}
but these all result in different errors such as
│ Error: Invalid function argument
│
│ on catalog.tf line 11, in resource "aws_glue_crawler" "bucket_crawlers":
│ 11: for_each = {for key, value in setproduct(local.bucket-names, module.crawler_source_buckets.id): key=>value}
│ ├────────────────
│ │ while calling setproduct(sets...)
│ │ module.crawler_source_buckets.id is a object
│
│ Invalid value for "sets" parameter: a set or a list is required.
or
│ Error: Invalid template interpolation value
│
│ on catalog.tf line 14, in resource "aws_glue_crawler" "bucket_crawlers":
│ 14: path = "s3://${each.value[1]}"
│ ├────────────────
│ │ each.value[1] is object with 2 attributes
│
│ Cannot include the given value in a string template: string required.
which is making me question if the thing I want to do is even possible. Maybe the answer in this case is that I use the base variable declarations since AWS S3 Ids and ARNs follow a known standard pattern but I would like to know how to iterate on module outputs that are declared with a for_each pattern since that seems very useful.
Code
s3.tf
locals {
bucket-names = [
"${var.env_name}-failed-stripe-payment-reports",
"${var.env_name}-fraud-log-return-responses"]
}
module "crawler_source_buckets" {
source = "../../modules/s3-bucket-v2"
for_each = toset(local.bucket-names)
bucket_name = each.value
# There's more code but I'm choosing to leave it out for clairty
]
}
s3-bucket-v2 output.tf
output "arn" {
description = "The ARN of the main bucket."
value = aws_s3_bucket.main.arn
}
output "id" {
description = "The ID (name) of the main bucket."
value = aws_s3_bucket.main.id
}
# There's an extra 60 lines I'm choosing to leave out for clarity.
catalog.tf
resource "aws_glue_catalog_database" "catalog_databases" {
name = each.value
for_each = toset(local.bucket-names)
}
resource "aws_glue_crawler" "bucket_crawlers" {
database_name = each.value[0]
name = "${each.value[0]}-crawler"
role = aws_iam_role.glue_crawler_role.arn
for_each = {for key, value in setproduct(local.bucket-names, module.crawler_source_buckets.id): key=>value}
s3_target {
path = "s3://${each.value[1]}"
}
schedule = "cron(*/15 * * * ? *)"
}
Since you are using for_each in the module you have to access it using key. For example, module.crawler_source_buckets["key"].id. To get all ids you have to use the following:
values(module.crawler_source_buckets)[*].id
So it should be:
for_each = {for key, value in setproduct(local.bucket-names, values(module.crawler_source_buckets)[*].id): key=>value}

Accessing a list inside a list of maps in terraform

I am trying to access a list inside a list of maps in terraform. The structure is as below:
dynamic.tfvars
--------------
nacl=[
{
"vpc_name" = "vpc1"
"acl_name" = "acl1"
"subnet_name" = ["sub1-az1","sub2-az2"]
},
{
"vpc_name" = "vpc2"
"acl_name" = "acl2"
"subnet_name" = ["sub1-az1","sub2-az2"]
}
]
I am trying to get "subnet_name"(a list) from the list of maps in my child module and it does not work. Below is the piece of code that i am using.
main.tf
-------
data "aws_vpc" "vpc_nacl" {
count = length(var.nacl[*])
filter {
name = "tag:Name"
values = [element(var.nacl[*]["vpc_name"],count.index)]
}
}
locals {
lcl_vpc_nacl = data.aws_vpc.vpc_nacl.*.id
}
data "aws_subnet_ids" "example" {
count = length(var.nacl[*]["subnet_name"])
vpc_id = element(local.lcl_vpc_nacl,count.index)
filter {
name = "tag:Name"
values = [element(var.nacl[*]["subnet_name"],count.index)]
}
}
I am getting the below error when terraform plan is executed.
│ Error: Incorrect attribute value type
│
│ on Modules\NACL\Main.tf line 28, in data "aws_subnet_ids" "example":
│ 28: values = [element(var.nacl[*]["subnet_name"],count.index)]
│ ├────────────────
│ │ count.index is 1
│ │ var.nacl is list of object with 5 elements
│
│ Inappropriate value for attribute "values": element 0: string required.
Any fixes or suggestions would be highly appreciated. Thanks in advance.
From aws_subnet_ids data source documentation, I see values within filter expects a set.
Looking code you are almost there, but you are passing a list of lists to values which is why you have this error.
element((var.nacl[*]["subnet_name"]), count.index) extracts list of subnets in the format of list.
All you need is to convert into set and pass without square braces like below..
main.tf
-------
data "aws_subnet_ids" "example" {
count = length(var.nacl[*]["subnet_name"])
vpc_id = element(local.lcl_vpc_nacl,count.index)
filter {
name = "tag:Name"
values = toset(element((var.nacl[*]["subnet_name"]), count.index))
}
}
Give a try & let me know if this helps..

How to check, if Terraform will create resource

I use Terraform + AWS Lambda in my workflow, and i want to call some lambda with ARNs of my existent instances during every plan or apply action. I use this structure:
locals {
...
centralized_nodes = {...}
tags_map = {...}
backup_map = {
for node in keys(local.centralized_nodes):
node => aws_instance.centralized_node[node].arn
if can(local.tags_map[node]["backup"]) && !(can(aws_instance.centralized_node[node].root_block_device.0.tags["backup"]))
}
...
}
resource "aws_instance" "centralized_node" {
...
for_each = local.centralized_nodes
...
}
data "aws_lambda_invocation" "lambda_backup" {
for_each = local.backup_map
function_name = "lambdafunc"
input = jsonencode({
"resources" = [each.value]
})
}
it worked fine until i try to add node description object to centralized_nodes map and create new instance. When i add this, terraform shows me error during planning:
Error: Invalid for_each argument
│
│ on resources.tf line 223, in data "aws_lambda_invocation" "lambda_backup":
│ 223: for_each = local.backup_map
│ ├────────────────
│ │ local.backup_map will be known only after apply
│
│ The "for_each" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work around this, use the
│ -target argument to first apply only the resources that the for_each depends on.
Now i'm looking for terraform check if resource exists or will be created. i've tried adding arguments to map generator to exclude uncreated instances (it's OK in my workflow) like that:
if can(local.tags_map[node]["backup"]) && !(can(aws_instance.centralized_node[node].root_block_device.0.tags["backup"])) && can(aws_instance.centralized_node[node].arn)
But it doesn't work, terraform thinks it can reach ARN value for uncreated instance, but later.
Please, help me find the way to avoid such error.
Seems that your local.centralized_nodes is not pre-defined by rather generated automatically. If so, you can't use it with for_each.
The keys of the map (or all the values in the case of a set of strings) must be known values, or you will get an error message that for_each has dependencies that cannot be determined before apply, and a -target may be needed.
You can try converting your code to use count, as count does not have such a limitation. But then, count has its own issues which may prevent you from achieving your goals anyway.
I've found the only one way to get check if resources like instances and volumes already exist. It is using aws_instance and aws_instances datasources.
This is my solution:
locals {
...
centralized_nodes = {...}
tags_map = {...}
instance_name_root_volume_tags_map = {
for name, value in {for id, value in data.aws_instance.instance_id_arn_map : value.tags["Name"] => value}:
name => data.aws_ebs_volume.volume_instance_id_map[value.id].tags
}
backup_map = {
for name, value in {for id, value in data.aws_instance.instance_id_arn_map : value.tags["Name"] => value}:
name => value.arn
if !can(local.instance_name_root_volume_tags_map[name]["backup"]) && can({for node, value in local.tags_map: value["Name"] => value}[name]["backup"]) )
}
...
}
resource "aws_instance" "centralized_node" {
...
for_each = local.centralized_nodes
...
}
data "aws_lambda_invocation" "lambda_backup" {
for_each = local.backup_map
function_name = "lambdafunc"
input = jsonencode({
"resources" = [each.value]
})
}
data "aws_instances" "existant_instances_array" {
filter {
name = "tag:Name"
values = [for tagmap in local.hiera_tags_map : tagmap["Name"]]
}
}
data "aws_instance" "instance_id_arn_map" {
for_each = toset(data.aws_instances.existant_instances_array.ids)
instance_id = each.value
}
data "aws_ebs_volume" "volume_instance_id_map" {
for_each = data.aws_instance.instance_id_arn_map
filter {
name = "attachment.instance-id"
values = [each.value.id]
}
filter {
name = "attachment.device"
values = ["/dev/sda1"]
}
}

Extracting list of subnets result in error - is tuple with 1 element

I have a module that creates a VPC with public and private subnets
module "vpc" {
count = var.vpc_enabled ? 1 : 0
source = "./vpc"
}
and as an output of that module I'm extracting the private subnets
output "private_subnets" {
value = aws_subnet.private.*.id
}
Then I want to use that subnets list as an input of another module:
module "eks" {
source = "./eks"
name = var.name
private_subnets = var.vpc_enabled ? module.vpc.private_subnets : var.private_subnets_id
}
basically what I'm trying to achieve is that the user can choose if he want to create a new VPC or use as an input a list of subnets of their existing VPC.
The problem that I've right now is that I'm getting the following error in terraform plan:
on main.tf line 32, in module "eks":
32: private_subnets = var.vpc_enabled ? module.vpc.private_subnets : var.private_subnets_id
|----------------
| module.vpc is tuple with 1 element
This value does not have any attributes.
Does anyone knows how to fix this?
You are defining your vpc module with count. Thus you need to refer to individual instances of the module, even if you have only 1.
private_subnets = var.vpc_enabled ? module.vpc[0].private_subnets : var.private_subnets_id
Just to add Marcin's answer
I had a similar issue when working with dynamic blocks and locals in Terraform.
I had a locals block like this:
locals {
subnet_suffix = "dev-subnet"
delegation_settings = [{
subnet_delegation_name = "app-service-delegation"
subnet_service_delegation_name = "Microsoft.Web/serverFarms"
}]
}
And I was referencing the attributes this way:
module "subnet_public_1" {
source = "../../../modules/azure/subnet"
subnet_name = "${var.subnet_name}-public-1-${local.subnet_suffix}"
resource_group_name = data.azurerm_resource_group.main.name
virtual_network_name = data.azurerm_virtual_network.main.name
subnet_address_prefixes = var.subnet_address_prefixes.public_1
enforce_private_link_endpoint_network_policies = var.enforce_private_link_endpoint_network_policies.public_1
delegation_settings = [
{
subnet_delegation_name = local.delegation_settings.subnet_delegation_name
subnet_service_delegation_name = local.delegation_settings.subnet_service_delegation_name
}
]
tag_environment = var.tag_environment
}
And when I run terraform plan I get the error below:
│ Error: Unsupported attribute
│
│ on main.tf line 68, in module "subnet_public_1":
│ 68: subnet_delegation_name = local.delegation_settings.subnet_delegation_name
│ ├────────────────
│ │ local.delegation_settings is tuple with 1 element
│
│ This value does not have any attributes.
╵
╷
│ Error: Unsupported attribute
│
│ on main.tf line 69, in module "subnet_public_1":
│ 69: subnet_service_delegation_name = local.delegation_settings.subnet_service_delegation_name
│ ├────────────────
│ │ local.delegation_settings is tuple with 1 element
│
│ This value does not have any attributes.
Here's how I solved it:
All I had to do was to add the index to the attributes, in this case it was 0:
module "subnet_public_1" {
source = "../../../modules/azure/subnet"
subnet_name = "${var.subnet_name}-public-1-${local.subnet_suffix}"
resource_group_name = data.azurerm_resource_group.main.name
virtual_network_name = data.azurerm_virtual_network.main.name
subnet_address_prefixes = var.subnet_address_prefixes.public_1
enforce_private_link_endpoint_network_policies = var.enforce_private_link_endpoint_network_policies.public_1
delegation_settings = [
{
subnet_delegation_name = local.delegation_settings[0].subnet_delegation_name
subnet_service_delegation_name = local.delegation_settings[0].subnet_service_delegation_name
}
]
tag_environment = var.tag_environment
}
That's all