Iterate On Module Instances Output When Declared With `for_each` - amazon-web-services

Question

Description
I'm attempting to iterate on the results of a terraform module and build a set of AWS glue crawlers and IAM roles based on how many source S3 buckets are set in the terraform module
I'm specifically trying to use this line
for_each = {for key, value in setproduct(local.bucket-names, module.crawler_source_buckets.id): key=>value}
or this line
for_each = {for key, value in setproduct(local.bucket-names, module.crawler_source_buckets.*.id): key=>value}
or this line
for_each = {for key, value in setproduct(local.bucket-names, module.crawler_source_buckets.[*].id): key=>value}
but these all result in different errors such as
│ Error: Invalid function argument
│
│ on catalog.tf line 11, in resource "aws_glue_crawler" "bucket_crawlers":
│ 11: for_each = {for key, value in setproduct(local.bucket-names, module.crawler_source_buckets.id): key=>value}
│ ├────────────────
│ │ while calling setproduct(sets...)
│ │ module.crawler_source_buckets.id is a object
│
│ Invalid value for "sets" parameter: a set or a list is required.
or
│ Error: Invalid template interpolation value
│
│ on catalog.tf line 14, in resource "aws_glue_crawler" "bucket_crawlers":
│ 14: path = "s3://${each.value[1]}"
│ ├────────────────
│ │ each.value[1] is object with 2 attributes
│
│ Cannot include the given value in a string template: string required.
which is making me question if the thing I want to do is even possible. Maybe the answer in this case is that I use the base variable declarations since AWS S3 Ids and ARNs follow a known standard pattern but I would like to know how to iterate on module outputs that are declared with a for_each pattern since that seems very useful.
Code
s3.tf
locals {
bucket-names = [
"${var.env_name}-failed-stripe-payment-reports",
"${var.env_name}-fraud-log-return-responses"]
}
module "crawler_source_buckets" {
source = "../../modules/s3-bucket-v2"
for_each = toset(local.bucket-names)
bucket_name = each.value
# There's more code but I'm choosing to leave it out for clairty
]
}
s3-bucket-v2 output.tf
output "arn" {
description = "The ARN of the main bucket."
value = aws_s3_bucket.main.arn
}
output "id" {
description = "The ID (name) of the main bucket."
value = aws_s3_bucket.main.id
}
# There's an extra 60 lines I'm choosing to leave out for clarity.
catalog.tf
resource "aws_glue_catalog_database" "catalog_databases" {
name = each.value
for_each = toset(local.bucket-names)
}
resource "aws_glue_crawler" "bucket_crawlers" {
database_name = each.value[0]
name = "${each.value[0]}-crawler"
role = aws_iam_role.glue_crawler_role.arn
for_each = {for key, value in setproduct(local.bucket-names, module.crawler_source_buckets.id): key=>value}
s3_target {
path = "s3://${each.value[1]}"
}
schedule = "cron(*/15 * * * ? *)"
}

Answer 1

Since you are using for_each in the module you have to access it using key. For example, module.crawler_source_buckets["key"].id. To get all ids you have to use the following:
values(module.crawler_source_buckets)[*].id
So it should be:
for_each = {for key, value in setproduct(local.bucket-names, values(module.crawler_source_buckets)[*].id): key=>value}