Terraform aws secrets - nested object variable - amazon-web-services

Question

I have a terraform template that create aws ecs task.
I filled a variable with a list of object like this:
`
variables.tf
variable "microservices" {
description = "the microservices to implement"
type = list(object({
name = string,
port = number,
secrets = optional(list(object({
key = string,
arn = string
})))
}))
`
Then in my main.tf I have the following:
`
main.tf
resource "aws_ecs_task_definition" "task_definition" {
count = length("${var.microservices}")
family = "${var.microservices[count.index].name}-${var.environment}"
requires_compatibilities = ["FARGATE"]
network_mode = "awsvpc"
cpu = 1024
memory= 2048
execution_role_arn = "arn:aws:iam::xxxxx:role/service-role/xxxx-test-service-role"
container_definitions = jsonencode([
{
name = "${var.microservices[count.index].name}"
image = "${aws_ecr_repository.microservices_ecr_repos[count.index].repository_url}"
cpu = 1
essential = true
Ulimits = [{
Name = "nofile"
SoftLimit = 65535
HardLimit = 65535
}]
//length("${var.microservices[count.index].secrets}") > 0 ?
Secrets = [{
Name = length("${var.microservices[count.index].secrets}") > 0 ? "${var.microservices[count.index].secrets[0].key}" : 0
ValueFrom = length("${var.microservices[count.index].secrets}") > 0 ? "${var.microservices[count.index].secrets[0].arn}" : 0
//Name = "${var.microservices[count.index].secrets[0].key}"
//ValueFrom = "${var.microservices[count.index].secrets[0].arn}"
`
I don't understand how can I create Secrets parsing the variables.
The secrets can be optional (it could exist or not).
I should need a sort of for_each only in Secrets section in order to check if secret exist in input and then fill this filed.
An example of inputs is the following:
`
microservices = [
{
"name" = "api",
"port" = 3000,
"secrets" = [{ "key" = "test123", "arn" = "0123"},{ "key" = "testXXX", "arn" = "1010"}] },
{
"name" = "web",
"port" = 3000
"secrets" = [{ "key" = "test456", "arn" = "4567"}]
}]
`
Anyone approach this kind of issue/configuration? What I would like to achieve is to create a task definition in aws ecs with secrets field (or empty secrets section) based on microservices input.
I tested a different data structure like here:
flatten object made of nested list in terraform
But in this scenario I was able to create a new data structure but when I create the resource (e.g.) aws_ecs_task_definition with a For_each it replicate some configuration like ecs tasks with the same name:
`
locals {
microservices_and_secrets = merge([
for ecs_taks, group in var.microservices:
{
for secrets_key, secret in group["secrets"]:
"${ecs_taks}-${secrets_key}" => {
name = group["name"]
port = group["port"]
secret = secret
}
}
]...)
}
`
`
resource "aws_ecs_task_definition" "task_definition" {
for_each = local.microservices_and_secrets
family = "${each.value.name}-${var.environment}" <-- ISSUE with creation because it replicates the ecs task microservice name due to foreach
requires_compatibilities = ["FARGATE"]
network_mode = "awsvpc"
cpu = 1024
memory= 2048
`
The problem is also that with this solution I can't have a microservice without any secret. e.g. the issue is the following:
`
microservices = [
{
"name" = "api",
"port" = 3000,
"secrets" = [{ "key" = "test123", "arn" = "0123"},{ "key" = "testXXX", "arn" = "1010"}] },
{
"name" = "web",
"port" = 3000
"secrets" = [{ "key" = "test456", "arn" = "4567"}]
},
{
"name" = "ciaotask",
"port" = 3000
}
]
`
`
Error: Iteration over null value
│
│ on main-aws-ecs.tf line 153, in locals:
│ 152: {
│ 153: for secrets_key, secret in group["secrets"]:
│ 154: "${ecs_taks}-${secrets_key}" => {
│ 155: name = group["name"]
│ 156: port = group["port"]
│ 157: secret = secret
│ 158: }
│ 159: }
│ ├────────────────
│ │ group["secrets"] is null
│
│ A null value cannot be used as the collection in a 'for' expression.
`
Anyone could help how can I manage the ecs task creation based on microservice input posted above?
The question is, how can I create one aws_ecs_task_definition for each microservice present into microservices variable and it can have zero to n Secrets, starting from microservices variable list of objects.

Answer 1

I solved the issue.
I started from this guide https://codeburst.io/how-to-securely-use-aws-secrets-manager-to-inject-secrets-into-ecs-using-infrastructure-as-code-ff2b39b420b6
then I created a template file like this:
`container_definitions.json.tpl
[{
"name" : "${name}",
"image": "${image}",
"cpu" : 1,
"essential" : true,
"Ulimits" : [{
"Name" : "nofile",
"SoftLimit" : 65535,
"HardLimit" : 65535
}],
"Secrets" : ${secrets},
"Environment" : ${environment},
"LogConfiguration" : {
"LogDriver" : "awslogs",
"Options" : {
"awslogs-group" : "${awslogs-group}",
"awslogs-region" : "${aws_region}",
"awslogs-stream-prefix" : "ecs"
}
},
"portMappings" : [
{
"containerPort" : 3000,
"hostPort" : 3000
}
]
}]
`
in my main.tf instead I created the resources in this way:
`
*/
data "template_file" "container_definitions" {
count = length("${var.microservices}")
template = file("${path.module}/template_dir/container_definitions.json.tpl")
vars = {
aws_region = "${var.aws_region}"
cpu = 1
image = "${aws_ecr_repository.microservices_ecr_repos[count.index].repository_url}"
name = "${var.microservices[count.index].name}"
awslogs-group = "${aws_cloudwatch_log_group.cloudwatch_log_groups[count.index].id}"
environment = jsonencode("${var.microservices[count.index].environment}")
secrets = jsonencode("${var.microservices[count.index].secrets}")
}
}
/*
AWS ECS Task definition
*/
resource "aws_ecs_task_definition" "task_definition" {
count = length("${var.microservices}")
family = "${var.microservices[count.index].name}-${var.environment}"
requires_compatibilities = ["FARGATE"]
network_mode = "awsvpc"
cpu = "${var.microservices[count.index].cpu}"
memory= "${var.microservices[count.index].memory}"
execution_role_arn = "${aws_iam_role.task_execution_roles[count.index].arn}"
task_role_arn = "${aws_iam_role.task_execution_roles[count.index].arn}"
container_definitions = "${data.template_file.container_definitions[count.index].rendered}" //file("./containers_file/api.json")
}
`
In this way I was able to create a task definition in aws ecs with 0..n secrets and 0..n environment variables based on this (e.g.) input.
`
microservices = [
{
"name" = "api",
"port" = 3000,
"cpu" = 1024,
"memory" = 2048,
"secrets" = [{ "name" = "test123", "valuefrom" = "0123"},{ "name" = "testXXX", "valuefrom" = "1010"}] },
{
"name" = "web",
"port" = 3000,
"cpu" = 1024,
"memory" = 2048,
"secrets" = [{ "name" = "test456", "valuefrom" = "4567"}],
"environment" = [{ "name" = "weenv", "value" = "emi_is_ok" },{ "name" = "weenv123", "value" = "emi_is_ok123" } ]
},
{
"name" = "ciaotask",
"port" = 3000
"cpu" = 1024,
"memory" = 2048
}
]
`
I hope this could help someone else that ran in the same issue.