Terraform for loop inside for_each argument - amazon-web-services

Can someone please explain what a for loop inside a for_each argument does in Terraform? I am trying to create an AWS SSL certificate. I've seen other code like the below but I don't understand it:
resource "aws_acm_certificate" "nonprod_cert" {
domain_name = var.phz_domain_name
validation_method = "DNS"
}
resource "aws_route53_record" "nonprod_cert_record" {
for_each = {
for dvo in aws_acm_certificate.nonprod_cert.domain_validation_options : dvo.domain_name => {
name = dvo.resource_record_name
record = dvo.resource_record_value
type = dvo.resource_record_type
}
}
zone_id = var.phz_id
name = each.value.name
type = each.value.type
records = [each.value.record]
ttl = 60
}
resource "aws_acm_certificate_validation" "nonprod_cert_validated" {
certificate_arn = aws_acm_certificate.nonprod_cert.arn
validation_record_fqdns = [for record in aws_route53_record.nonprod_cert_record : record.fqdn]
depends_on = [
aws_acm_certificate.nonprod_cert,
aws_route53_record.nonprod_cert_record
]
}
The specific line that I don't understand is the one in the route53 record. I get that a for_each argument can be used to create multiple resources from a single block, but I can't find anywhere that explains what this for loop is doing inside of it. If someone could explain that would be great!

The inner for "loop" creates the data that the for_each then iterates over. Specifically the each.key will be the dvo.domain_name and the each.value will be the
{
name = dvo.resource_record_name
record = dvo.resource_record_value
type = dvo.resource_record_type
}
You could simply move that into a locals block beforehand and not have it in a single line:
locals {
records = {
for dvo in aws_acm_certificate.nonprod_cert.domain_validation_options : dvo.domain_name => {
name = dvo.resource_record_name
record = dvo.resource_record_value
type = dvo.resource_record_type
}
}
}
resource "aws_route53_record" "nonprod_cert_record" {
for_each = local.records
zone_id = var.phz_id
name = each.value.name
type = each.value.type
records = [each.value.record]
ttl = 60
}
That will have the exact same effect.

First they are using a for expression to convert one type of object into another type. In this case they are converting the list of domain_validation_options into a list of objects that can be used for creating aws_route53_record resources.
Next they are using for_each to create a new aws_route53_record resource for each element of the list that was generated by the for expression.
The key things to be aware of here are:
for is used to convert a list of objects into a list of different objects.
for_each is used to create multiple resources from a list, set, or map of values.
I highly recommend spending the time to go through the Terraform learning site, or at least the Terraform documentation to learn the basic keywords and overall syntax.

Related

Reference module with multiple resources

I am trying to create an AWS route53 hosted zone and add records to it. I added the following resources to a module main.tf
resource "aws_route53_zone" "zone" {
name = var.name
}
data "aws_route53_zone" "zone_data" {
name = var.name
}
resource "aws_route53_record" "record" {
zone_id = data.aws_route53_zone.zone_data.zone_id
name = var.record_name
type = var.record_type
ttl = var.record_ttl
records = var.record_value
}
Then I reference that module in a stack main.py as follows:
module "route53" {
source = "../../modules/route53"
name = "website.com"
type = "NS"
ttl = "30"
}
My issue is that building the stack will use the same name variable for both zone and record resources. How do I add another name to the stack module route53 for the record resource that is different from the zone resource?
If all you're trying to do in the module is create a zone and a record, you could use split to get the zone from the record name given. Like this:
main.tf
module "route53" {
source = "./modules/route53"
name = "something.website.com"
type = "NS"
ttl = "30"
}
modules/route53/main.tf
variable "name" {}
variable "type" {}
variable "ttl" {}
resource "aws_route53_zone" "this" {
name = split(".", var.name)[0]
}
resource "aws_route53_record" "this" {
zone_id = aws_route53_zone.this.zone_id
name = var.name
type = var.type
ttl = var.ttl
records = [var.name]
}
If however, you want multiple records in that zone, you could consider something like this, but this will depend heavily on what record configuration you're after.
main.tf
module "route53" {
source = "./modules/route53"
name = "website.com"
record_configs = {
something = {
type = "A"
records = ["192.168.0.99"]
}
other = {
type = "CNAME"
records = ["something.website.com"]
}
}
}
modules/route53/main.tf
variable "name" {}
variable "record_configs" {}
resource "aws_route53_zone" "this" {
name = var.name
}
resource "aws_route53_record" "this" {
for_each = var.record_configs
zone_id = aws_route53_zone.this.zone_id
name = each.key
type = each.value.type
records = each.value.records
}
If you have multiple records and names, the best way is to use for_each. For example:
variable "names" {
default = ["website.com", "website1.com", "website2.com"]
}
then
module "route53" {
source = "../../modules/route53"
for_each = toset(var.name)
name = each.key
type = "NS"
ttl = "30"
}
This way you can have same module for multiple names.
Randomly experimenting with solutions got me to add the resource variables' names as arguments to the module. This seems to allow referring to arguments of a specific resource in the root module if its argument name is the same as other resources (e.g. record_name vs name).
module "route53" {
source = "../../modules/route53"
name = "website.com"
record_name = "_auth.website.com"
record_type = "NS"
record_ttl = "30"
}

aws_acm_certificate seems to have changed its state output possibly due to a provider update- am I doing this wrong?

Terraform v0.12.12
+ provider.aws v3.0.0
+ provider.template v2.1.2
Before I was doing this:
resource "aws_route53_record" "derps" {
name = aws_acm_certificate.mycert[0].resource_record_name
type = aws_acm_certificate.mycert[0].resource_record_type
zone_id = var.my_zone_id
records = aws_acm_certificate.mycert[0].resource_record_value
ttl = 60
}
And that worked fine for me about a week ago.
I just did a plan and got an error:
records = [aws_acm_certificate.mycert.domain_validation_options[0].resource_record_value]
This value does not have any indices.
Now I don't pin provider versions, so I'm assuming I pulled a newer version and the resource changed.
After fighting with this and realizing it's not a list (even though when doing show state it sure looked like one) I am now doing this to make it a list:
resource "aws_route53_record" "derps" {
name = sort(aws_acm_certificate.mycert.domain_validation_options[*].resource_record_name)[0]
type = sort(aws_acm_certificate.mycert.domain_validation_options[*].resource_record_type)[0]
zone_id = var.my_zone_id
records = [sort(aws_acm_certificate.mycert.domain_validation_options[*].resource_record_value)[0]]
ttl = 60
}
This resulted in no changes which is good. But if I use the example for doing this from the docs they now use for_each: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/acm_certificate_validation
resource "aws_route53_record" "example" {
for_each = {
for dvo in aws_acm_certificate.example.domain_validation_options : dvo.domain_name => {
name = dvo.resource_record_name
record = dvo.resource_record_value
type = dvo.resource_record_type
zone_id = dvo.domain_name == "example.org" ? data.aws_route53_zone.example_org.zone_id : data.aws_route53_zone.example_com.zone_id
}
}
allow_overwrite = true
name = each.value.name
records = [each.value.record]
ttl = 60
type = each.value.type
zone_id = each.value.zone_id
}
resource "aws_acm_certificate_validation" "example" {
certificate_arn = aws_acm_certificate.example.arn
validation_record_fqdns = [for record in aws_route53_record.example : record.fqdn]
}
Is the above the correct way to do this now? Am I going to run into issues doing it the way I currently am? Doing it like above would result in a destroy/recreate (i guess I could import it myself but that's painful).
Is doing it my way not going to result in unexpected diffs?
Edit
So, more specific for my issue. This is what I see when I look at the state:
terraform state show aws_acm_certificate.mycert
...
domain_name = "*.mydom.com"
domain_validation_options = [
{
domain_name = "*.mydom.com"
resource_record_name = "_11111111111.mydom.com."
resource_record_type = "CNAME"
resource_record_value = "_1111111111.11111111.acm-validations.aws."
},
{
domain_name = "mydom.com"
resource_record_name = "_11111111111.mydom.com."
resource_record_type = "CNAME"
resource_record_value = "_1111111111.111111111.acm-validations.aws."
},
]
...
By using sort I'm effectively using count which of course results in a destroy/recreate if the order changes. But in my case I think that's unlikely?? I also don't fully understand the difference between just using the values from the wildcard validation config and using both of them.
The AWS Terraform provider was recently upgraded to version 3.0. This version comes with a list of breaking changes. I recommend consulting the AWS provider 3.0 upgrade guide.
The issue you are encountering is because the domain_validation_options attribute is now a set instead of a list. From that guide:
Since the domain_validation_options attribute changed from a list to a set and sets cannot be indexed in Terraform, the recommendation is to update the configuration to use the more stable resource for_each support instead of count
I recommend using the new foreach syntax, as the upgrade guide recommends, in order to avoid unexpected diffs. The guide states that you will need to use terraform state mv to move the old configuration state to the new configuration, in order to prevent the resources from being recreated.
This is the same problem we were facing just now.. we use for_each to define hosting for multiple sites according to provided local variables, now, since we already use for_each we can't use it for workaround of their changes.. unfortunate.
I didn't want to go with sort so I checked what Jimmy wrote but it didn't work for this case due to output being index, I fixed it by using [0] instead of [*]:
resource "aws_route53_record" "cert_validation" {
for_each = local.web_pages
allow_overwrite = true
name = tolist(aws_acm_certificate.cert[each.key].domain_validation_options)[0].resource_record_name
type = tolist(aws_acm_certificate.cert[each.key].domain_validation_options)[0].resource_record_type
records = [tolist(aws_acm_certificate.cert[each.key].domain_validation_options)[0].resource_record_value]
zone_id = var.aws_hosted_zone
ttl = 60
}
works for us now ;)
These 3 sets of code below work. (I used Terraform v0.15.0)
*The difference between 1st and 2nd is [0] and .0
1st:
resource "aws_route53_record" "myRecord" {
zone_id = aws_route53_zone.myZone.zone_id
name = tolist(aws_acm_certificate.myCert.domain_validation_options)[0].resource_record_name
type = tolist(aws_acm_certificate.myCert.domain_validation_options)[0].resource_record_type
records = [tolist(aws_acm_certificate.myCert.domain_validation_options)[0].resource_record_value]
ttl = "60"
allow_overwrite = true
}
2nd:
resource "aws_route53_record" "myRecord" {
zone_id = aws_route53_zone.myZone.zone_id
name = tolist(aws_acm_certificate.myCert.domain_validation_options).0.resource_record_name
type = tolist(aws_acm_certificate.myCert.domain_validation_options).0.resource_record_type
records = [tolist(aws_acm_certificate.myCert.domain_validation_options).0.resource_record_value]
ttl = "60"
allow_overwrite = true
}
3rd:
resource "aws_route53_record" "myRecord" {
for_each = {
for dvo in aws_acm_certificate.myCert.domain_validation_options : dvo.domain_name => {
name = dvo.resource_record_name
record = dvo.resource_record_value
type = dvo.resource_record_type
}
}
name = each.value.name
records = [each.value.record]
ttl = 60
type = each.value.type
zone_id = aws_route53_zone.myZone.zone_id
allow_overwrite = true
}
You can also cheat with tolist.
e.g. rewrite aws_acm_certificate.mycert.domain_validation_options[*].resource_record_name as tolist(aws_acm_certificate.mycert.domain_validation_options)[*].resource_record_name
I had to do this in a module, as this particular module resource already had a count in it, and I knew I'd only ever have one entry.

Creating multiple Alias records in Terraform with for_each

I am trying to create multiple alias DNS records in terraform using for_each. I am getting an error when specifying the name and zone ID of the Alias name. My question is how do I read from a variable defined as a set of strings? Below is my code block and variables defined:
resource "aws_route53_record" "alias_records" {
for_each = var.alias_names
zone_id = aws_route53_zone.zone.zone_id
name = each.key
type = "A"
alias {
name = var.alias_dns_names[each.key]
zone_id = var.alias_zone_ids[each.key]
evaluate_target_health = false
}
}
variable "alias_names" {
type = set(string)
}
variable "alias_dns_names" {
type = set(string)
}
variable "alias_zone_ids" {
type = set(string)
}
Error: Invalid index
on alias.tf line 8, in resource "aws_route53_record" "alias_records":
8: name = var.alias_dns_names[each.key]
This value does not have any indices.
Error: Invalid index
on alias.tf line 9, in resource "aws_route53_record" "alias_records":
9: zone_id = var.alias_zone_ids[each.key]
This value does not have any indices.
alias_names = [
"alias1",
"alias2",
"alias3"
}
alias_dns_names = [
"alias_dns_1",
"alias_dns_2",
"alias_dns_3"
}
alias_zone_ids = [
"alias_zone_1",
"alias_zone_2",
"alias_zone_3"
}
A set data structure is not ordered and its elements don't have any identifiers other than their values, so the set type is not appropriate for your use-case here.
Since you seem to be describing three different aspects of the same underlying object here, I think the most convenient way to expose that from your module would be as a map of objects, like this:
variable "aliases" {
type = map(object({
zone_id = string
dns_name = string
}))
}
You can then use this variable alone to capture all three of the values you need for each alias record:
aliases = {
alias1 = {
zone_id = "alias_zone_1"
dns_name = "alias_dns_1"
}
alias2 = {
zone_id = "alias_zone_2"
dns_name = "alias_dns_2"
}
alias3 = {
zone_id = "alias_zone_3"
dns_name = "alias_dns_3"
}
}
A map of any data type can be used directly with resource for_each, with each.key representing the map key and each.value representing the value. So you can then change your resource block like this:
resource "aws_route53_record" "alias_records" {
for_each = var.aliases
zone_id = aws_route53_zone.zone.zone_id
name = each.key
type = "A"
alias {
zone_id = each.value.zone_id
name = each.value.dns_name
evaluate_target_health = false
}
}
Because the elements of the map are objects, you can access them from each.value to get the zone_id and dns_name values that correspond with each key in the original map.

Struggling to automate terraform WAF

I'm trying to terraform WAF ACL and associated rules. The terraform stack I'm working on is identical in DEV, QA , and PROD, differences are all handled using different variables. So my idea is to store a list of CIDRs in a variable, and automatically create ALLOW rules for each. My limited knowledge is slowing me down though. It creates the ipsets perfectly, but the rules and ACL complain,
variable cloud_allowed_cidr_list = {type="list" default=["1.2.3.4/32","4.3.2.1/32"]}
resource "aws_waf_ipset" "ipset" {
count = "${length(var.cloud_allowed_cidr_list)}"
name = "ipset-${count.index}"
ip_set_descriptors {
type = "IPV4"
value = "${element(var.cloud_allowed_cidr_list, count.index)}"
}
}
resource "aws_waf_rule" "matchIPrule" {
count = "${length(var.cloud_allowed_cidr_list)}"
depends_on = ["aws_waf_ipset.ipset"]
name = "matchMancIPrule${count.index}"
metric_name = "matchMancIPrule${count.index}"
predicates {
data_id = "${aws_waf_ipset.ipset.*.id}"
negated = false
type = "IPMatch"
}
}
resource "aws_waf_web_acl" "waf_acl" {
depends_on = ["aws_waf_ipset.ipset", "aws_waf_rule.matchIPrule"]
name = "mancACL${count.index}"
metric_name = "mancACL${count.index}"
default_action {
type = "BLOCK"
}
rules {
action {
type = "ALLOW"
}
priority = "${count.index}"
rule_id = "${aws_waf_rule.matchIPrule.id}"
type = "REGULAR"
}
}
It fell apart when I realised that rules have multiple predicates, and the ACL has multiple rules .....how do you create that dynamically ? If anyone has any examples of doing something similar I'd be very grateful.
Since the release of 0.12 you can now do this using dynamic blocks.
No need to use count to iterate over your array.
resource "aws_waf_ipset" "ipset" {
name = "youripset"
dynamic "ip_set_descriptors" {
iterator = ip
for_each = var.cloud_allowed_cidr_list
content {
type = "IPV4"
value = ip.value
}
}
}

Terraform - Conditional Data Source

In terraform, is there any way to conditionally use a data source? For example:
data "aws_ami" "application" {
most_recent = true
filter {
name = "tag:environment"
values = ["${var.environment}"]
}
owners = ["self"]
}
I'm hoping to be able to pass in an environment variable via the command line, and based on that, determine whether or not to fetch this data source.
I know with resources you can use the count property, but it doesn't seem you can use that with data sources.
I would consider tucking this code away in a module, but modules also can't use the count parameter.
Lastly, another option would be to provide a "Default" value for the data source, if it returned null, but I don't think that's doable either.
Are there any other potential solutions for this?
You can use a conditional on data sources the same as you can with resources and also from Terraform 0.13+ on modules as well:
variable "lookup_ami" {
default = true
}
data "aws_ami" "application" {
count = var.lookup_ami ? 1 : 0
most_recent = true
filter {
name = "tag:environment"
values = [var.environment]
}
owners = ["self"]
}
One use case for this in Terraform 0.12+ is to utilise the lazy evaluation of ternary statements like with the following:
variable "internal" {
default = true
}
data "aws_route53_zone" "private_zone" {
count = var.internal ? 1 : 0
name = var.domain
vpc_id = var.vpc_id
private_zone = var.internal
}
data "aws_route53_zone" "public_zone" {
count = var.internal ? 0 : 1
name = var.domain
private_zone = var.internal
}
resource "aws_route53_record" "www" {
zone_id = var.internal ? data.aws_route53_zone.private_zone.zone_id : data.aws_route53_zone.public_zone.zone_id
name = "www.${var.domain}"
type = "A"
alias {
name = aws_elb.lb.dns_name
zone_id = aws_elb.lb.zone_id
evaluate_target_health = false
}
}
This would create a record in the private zone when var.internal is true and instead create a record in the public zone when var.internal is false.
For this specific use case you could also use Terraform 0.12+'s null to rewrite this more simply:
variable "internal" {
default = true
}
data "aws_route53_zone" "zone" {
name = var.domain
vpc_id = var.internal ? var.vpc_id : null
private_zone = var.internal
}
resource "aws_route53_record" "www" {
zone_id = data.aws_route53_zone.zone.zone_id
name = "www.${data.aws_route53_zone.zone.name}"
type = "A"
alias {
name = aws_elb.lb.dns_name
zone_id = aws_elb.lb.zone_id
evaluate_target_health = false
}
}
This would only pass the vpc_id parameter to the aws_route53_zone data source if var.internal is set to true as you can't set vpc_id when private_zone is false.
Old Terraform 0.11 and earlier answer:
You can in fact use a conditional on the count of data sources but I've yet to manage to work out a good use case for it when I've tried.
As an example I successfully had this working:
data "aws_route53_zone" "private_zone" {
count = "${var.internal == "true" ? 1 : 0}"
name = "${var.domain}"
vpc_id = "${var.vpc_id}"
private_zone = "true"
}
data "aws_route53_zone" "public_zone" {
count = "${var.internal == "true" ? 0 : 1}"
name = "${var.domain}"
private_zone = "false"
}
But then had issues in how to then select the output of it because Terraform will evaluate any variables in the ternary conditional before deciding which side of the ternary to use (instead of lazy evaluation). So something like this doesn't work:
resource "aws_route53_record" "www" {
zone_id = "${var.internal ? data.aws_route53_zone.private_zone.zone_id : data.aws_route53_zone.public_zone.zone_id}"
name = "www.example.com"
type = "A"
alias {
name = "${aws_elb.lb.dns_name}"
zone_id = "${aws_elb.lb.zone_id }"
evaluate_target_health = "false"
}
}
Because if internal is true then you get the private_zone data source but not the public_zone data source and so the second half of the ternary fails to evaluate because data.aws_route53_zone.public_zone.zone_id isn't defined and equally with the other way around too.
In your case you probably just want to conditionally use the data source so might be able to do something like this:
variable "dynamic_ami" { default = "true" }
variable "default_ami" { default = "ami-123456" }
data "aws_ami" "application" {
most_recent = true
filter {
name = "tag:environment"
values = ["${var.environment}"]
}
owners = ["self"]
}
resource "aws_instance" "app" {
ami = "${var.dynamic_ami == "true" ? data.aws_ami.application.id : var.default_ami}"
instance_type = "t2.micro"
}