I'm trying to create two loops inside the resources block on a aws_iam_policy_document statement.
I do have two lists of strings
variable "all_services" {
description = "List of aws services"
type = list(string)
default = [
"a4b",
"access-analyzer",
"acm",
"acm-pca",
"aws-marketplace-management"
]
}
and
variable "all_accounts" {
description = "List of accounts to block"
type = list(string)
default = [
"11111111111",
"22222222222",
"33333333333"
]
}
I already got it with one loop to iterate on the accounts list
resources = [for account in local.accounts_to_protect: "arn:aws:ec2::${account}:*"]
Which has this output
+ Resource = [
+ "arn:aws:ec2::11111111111:*",
+ "arn:aws:ec2::22222222222:*",
+ "arn:aws:ec2::33333333333:*",
But I also try to iterate on the service.
I tried something like this with a for inside the for
resources = [for account in var.all_accounts:[
for services in var.all_services: ["arn:aws:${services}::${account}:*"]]]
It gives me this output
Error: Incorrect attribute value type
on block.account.tf line 60, in data "aws_iam_policy_document" "block_accounts":
60: resources = [for account in var.all_accounts:[
61: for services in var.all_services: ["arn:aws:${services}::${account}:*"]]]
|----------------
| local.accounts_to_protect is list of string with 2 elements
| var.all_services is list of string with 5 elements
Inappropriate value for attribute "resources": element 0: string required.
Related
I want to define a list of maps inside a variable so I can use for_each on multiple resources with conditionals based on the key values.
For example I have this locals.tf file where I define the list of maps
locals {
networking = [
{
name = "first"
domain = "first.${local.dns_name}"
port = 8080
group = "eighties"
},
{
name = "second"
domain = "second.${local.dns_name}"
port = 8081
group = "eighties"
},
{
name = "third"
domain = "third.${local.dns_name}"
port = 9090
group = "nineties"
},
{
name = "fourth"
port = 9091
group = "nineties"
}
]
}
In my other file, I can loop through the list of maps with for_each and for arguments:
resource "google_dns_record_set" "dns_records" {
for_each = {
for k in local.networking: k.domain => k
if k.domain != null
}
name = each.value.domain
type = "A"
ttl = 300
managed_zone = var.managed_zone
project = var.dns_project
rrdatas = [google_compute_global_address.default-forwarding-address.address]
}
Given this setup, I have 2 different situations that I might discuss:
Since the domain key doesn't exists for the fourth map, Terraform just stops.
Is it possible to skip the resource creations if domain does not exist in the fourth map? Or simply skip the errors.
Error: Unsupported attribute on dns-record.tf line 4, in resource "google_dns_record_set" "dns_records": 4: if k.domain != null This object does not have an attribute named "domain".
If I change the k.domain values for k.group and the conditional to be something like k.group == "eighties" so I can target a specific group, I receive this duplication error.
Two different items produced the key "eighties" in this 'for' expression. If duplicates are expected, use the ellipsis (...) after the value expression to enable grouping by key.
Are this errors manageable with the current setup or should I drop the list of maps idea?
You can check if the domain exist as follows:
{
for k in local.networking: k.domain => k
if contains(keys(k), "domain")
}
When running terraform plan or terraform apply with a list provided to for_each an error occurs saying
Error: Invalid for_each argument
on main.tf line 2, in resource "aws_ssm_parameter" "foo":
2: for_each = ["a", "b"]
The given "for_each" argument value is unsuitable: the "for_each" argument
must be a map, or set of strings, and you have provided a value of type tuple.
A minimum example to reproduce this error is this:
resource "aws_ssm_parameter" "foo" {
for_each = ["a", "b"]
name = "foo-${each.value}"
type = "String"
value = "bar-${each.value}"
}
Explanation
This error is often caused by passing a list to for_each, but for_each only works with unordered data-types, i.e. with sets and maps.
Solution
The resolution depends on the situation.
List of strings
If the list is just a list of strings, the easiest fix is to add a toset()-call to transform the list to a set that can be handled by for_each, like this
resource "aws_ssm_parameter" "foo" {
for_each = toset(["a", "b"])
name = "foo-${each.value}"
type = "String"
value = "bar-${each.value}"
}
List that can be rearranged to a map
If the input is a list, but easily be rearranged to a map this is usually the best way.
Say we have a list like this
locals {
animals = [
{
name = "Bello"
age = 3
type = "dog"
},
{
name = "Minga"
age = 4
type = "cat"
},
]
}
Then an appropriate re-structuring might be this
locals {
animals = {
Bello : {
age = 3
type = "dog"
},
Minga : {
age = 4
type = "cat"
}
}
}
which then allows you to define
resource "aws_ssm_parameter" "foo" {
for_each = local.animals
name = each.key
type = string
value = "This is a ${each.value.type}, ${each.value.age} years old."
}
List that you do not want to rearrange
Sometimes it is natural to have a list, e.g. comming from an output of a module that one does not control or from a resource that is defined with count. In such a situation, one can either work with count like this
resource "aws_ssm_parameter" "foo" {
count = length(local.my_list)
name = my_list[count.index].name
type = "String"
value = my_list[count.index].value
}
which works for a list of maps containing name and value as keys. Often times, though, it is more appropriate to transform the list to a map instead like this
resource "aws_ssm_parameter" "foo" {
for_each = { for x in local.my_list: x.id => x }
name = each.value.name
type = "String"
value = each.value.value
}
Here one should choose anything appropriate in place of x.id. If my_list is a list of objects, there is usually some common field like a name or key, that can be used. The advantage of this approach in favor of using count as above, is that this behaves better when inserting or removing elements from the list. count will not notice the insertion or deletion as such and will hence update all resources following the place where the insertion took place, while for_each really only adds or removes the resource with the new or deleted id.
I have a module for service-accounts in GCP being used to populate kubernetes secrets
Here is my module
resource "google_service_account" "service_account" {
count = var.enabled ? 1 : 0
account_id = var.account_id
display_name = var.display_name
}
resource "google_project_iam_member" "service_account_roles" {
count = var.enabled ? length(var.roles) : 0
role = "roles/${element(var.roles, count.index)}"
member = "serviceAccount:${google_service_account.service_account[0].email}"
}
resource "google_service_account_key" "service_account_key" {
count = var.enabled ? 1 : 0
service_account_id = google_service_account.service_account[0].name
}
'output.tf' contains the following
output "private_decoded_key" {
value = base64decode(
element(
concat(
google_service_account_key.service_account_key.*.private_key,
[""],
),
0,
),
)
description = "The base 64 decoded version of the credentials"
}
Since there is a conditional that none of these resources are created without the enabled flag, I had to handle it in TF 0.11.14 this way, and the tf0.12 autoupgrade tool didnt do much changes here.
How can I simplify this in Terraform 0.12.24,
I tried modifying the output to simply
value = base64decode(google_service_account_key.service_account_key[0].private_key)
But the problem there is that if the corresponding kubernetes cluster gets deleted during a deletion, and there are errors midway because terraform, I will not be able to cleanup the terraform state of the rest of the resources using
`terraform destroy'
Attempts to convert the count to for_each as shown below gave me the following errors
resource "google_service_account" "service_account" {
# count = var.enabled ? 1 : 0
for_each = var.enabled ? 1 : 0
account_id = var.account_id
display_name = var.display_name
}
resource "google_project_iam_member" "service_account_roles" {
# count = var.enabled ? length(var.roles) : 0
for_each = var.enabled ? toset(var.roles) : 0
# role = "roles/${element(var.roles, count.index)}"
role = "roles/${each.value}"
member = "serviceAccount:${google_service_account.service_account[0].email}"
}
for_each = var.enabled ? toset(var.roles) : 0
The true and false result expressions must have consistent types. The given
expressions are set of dynamic and number, respectively.
What am I doing wrong above ?
In the terraform version you mentioned (0.12.24) you should be able to use try() in your outputs.tf:
value = try(base64decode(google_service_account_key.service_account_key[0].private_key), "")
This would default to "" if google_service_account_key.service_account_key[0].private_key is not resolvable for any reason; you can also default to null of course.
Edit/Update: To answer second (edited) part of the question:
To get rid of the error that both sides need to have the same type, you need to use [] as an empty set instead of 0 when converting to for_each:
for_each = var.enabled ? toset(var.roles) : []
Please pay attention with existing infrastructure as you need to manipulate the state file when converting from count to for_each or terraform will try to destroy and create resources.
(I will cover this in more detail in part 3 of a series of stories I am currently working on about how to write terraform modules. You can find part 1 on medium and part 2 will be released next week.)
I have a module for google_service_account, and just converted to Terraform-12 (0.12.24)
resource "google_service_account" "service_account" {
count = var.enabled ? 1 : 0
account_id = var.account_id
display_name = var.display_name
}
output.tf retrieves the email
output "email" {
value = try(google_service_account.service_account.*.email, null)
# value = element( --> Commented out part works fine
# concat(google_service_account.service_account.*.email, [""]),
# 0,
# )
description = "The e-mail address of the service account. Usually use this when constructing IAM policies."
When using this module in another resource as follows
resource "google_storage_bucket_iam_member" "registry_bucket_iam" {
bucket = "artifacts.${var.project}.appspot.com"
role = "roles/storage.objectViewer"
member = "serviceAccount:${module.k8s-node-service-account.email}"
}
I get the following error
48: member = "serviceAccount:${module.k8s-node-service-account.email}"
|----------------
| module.k8s-node-account.email is tuple with 1 element
Cannot include the given value in a string template: string required.
How can this be resolved ?
google_service_account.service_account.*.email evaluates to an array of 1 or 0 elements dependeing on var.enabled - while google_service_account.service_account[0].email evaluates to a string or to an error if var.enabled is false. So when using try() you want to evaluate to the string or in case of an error default to null
changing your output to the following should lead to the expected result of having an email output of type string.
output "email" {
value = try(google_service_account.service_account[0].email, null)
}
I am trying to create "N" number of instances via Terraform and i want each of my instance to be attached with 3 EBS volumes. I am trying the following Terraform snippet to perform this:
resource "aws_instance" "provision-data-nodes" {
count = "${var.data_node_count}"
ami = "${var.ami_id}"
instance_type = "${var.machine_type}"
key_name = "elasticsearch-prod"
vpc_security_group_ids = ["${aws_security_group.es-sec-group.id}"]
#availability_zone = "${element(var.azs, count.index)}"
subnet_id = "${element(var.subnets, count.index)}"
tags {
Name = "${var.data_name}-${count.index+1}"
Type = "es-data"
}
root_block_device {
volume_size = 100
volume_type = "gp2"
}
}
resource "aws_ebs_volume" "data-ebs-volumes" {
count = "${var.data_node_count * 3}"
availability_zone = "${element(var.azs, count.index)}"
size = "${var.volume_size_data}"
type = "gp2"
tags {
Name = "${var.data_name}-${count.index+1}"
Type = "es-data-vols"
}
}
resource "aws_volume_attachment" "data-ebs-volumes-attach" {
count = "${var.data_node_count * 3}"
device_name = "${element(var.block_device_names, count.index)}"
#device_name = "${var.block_device_names}"
#volume_id = "${element(aws_ebs_volume.data-ebs-volumes.*.id,count.index)}"
volume_id = "${aws_ebs_volume.data-ebs-volumes.*.id[count.index]}"
#instance_id = "${element(aws_instance.provision-data-nodes.*.id,count.index)}"
instance_id = "${aws_instance.provision-data-nodes.*.id[count.index]}"
}
The variable data_node_count = 2, so that should actually create me 2*3=6 drives and so 3 drives should be attached to the first instance and the other 3 should be attached to next and so on as the data_node_count is increased.
Terraform outputs the following error when running the plan:
Error: Error running plan: 1 error(s) occurred:
* aws_volume_attachment.data-ebs-volumes-attach: 4 error(s) occurred:
* aws_volume_attachment.data-ebs-volumes-attach[2]: index 2 out of range for list aws_instance.provision-data-nodes.*.id (max 2) in:
${aws_instance.provision-data-nodes.*.id[count.index]}
* aws_volume_attachment.data-ebs-volumes-attach[5]: index 5 out of range for list aws_instance.provision-data-nodes.*.id (max 2) in:
${aws_instance.provision-data-nodes.*.id[count.index]}
* aws_volume_attachment.data-ebs-volumes-attach[3]: index 3 out of range for list aws_instance.provision-data-nodes.*.id (max 2) in:
${aws_instance.provision-data-nodes.*.id[count.index]}
* aws_volume_attachment.data-ebs-volumes-attach[4]: index 4 out of range for list aws_instance.provision-data-nodes.*.id (max 2) in:
${aws_instance.provision-data-nodes.*.id[count.index]}
So I am not able to attach the 3 volumes to each of the 2 instances.
You seem to be using element(list, index) and list[index] interchangeably here when actually they are subtly different.
While list[index] is a slightly more compact syntax it won't loop back through the list modulo the list length. element(list, index) on the other hand does do this.
So given the following list:
variable "list" {
default = [
1,
2,
]
}
This will error with an index out of range exception:
output "loop" {
value = "${var.list[3]}"
}
While this will return 2:
output "loop_element" {
value = "${element(var.list, 3)}"
}
You should also note that your looping of EBS volume attachments won't operate in the way you think it will:
The variable data_node_count = 2, so that should actually create me 2*3=6 drives and so 3 drives should be attached to the first instance and the other 3 should be attached to next and so on as the data_node_count is increased.
Instead the attachments will attach the first volume to the first instance, the second volume to the second instance, the third volume to the first instance, the fourth volume to the second instance and so on, alternating across the instances.