I am trying to get the network interface ids of a VPC endpoint using the data resource of aws_network_interface, the code for which looks like
resource "aws_vpc_endpoint" "api-gw" {
vpc_id = var.vpc_id
service_name = "com.amazonaws.${var.aws_region}.execute-api"
vpc_endpoint_type = "Interface"
security_group_ids = [aws_security_group.datashop_sg.id]
private_dns_enabled = true
subnet_ids = [data.aws_subnet.private-1.id]
}
data "aws_network_interface" "endpoint-api-gw" {
count = length(aws_vpc_endpoint.api-gw.network_interface_ids)
id = tolist(aws_vpc_endpoint.api-gw.network_interface_ids)[count.index]
}
I get the following error
Error: Invalid count argument
│
│ in data "aws_network_interface" "endpoint-api-gw":
│ count = length(aws_vpc_endpoint.api-gw.network_interface_ids)
│
│ The "count" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. To work
│ around this, use the -target argument to first apply only the resources that the count depends on.
I have also tried the for_each and it gives similar error of it is dependent on resources. I am running out of ideas. It would be of great if someone can help
The error is clear:
count = length(aws_vpc_endpoint.api-gw.network_interface_ids)
is only known after apply. You can't do this. count value must be known at plan time. You have to run your TF in two stages:
Execute TF with -target to deploy only aws_vpc_endpoint.api-gw using option.
Execute it again, to deploy the rest.
Otherwise, you have to re-factor you code, and fully eliminate the dependency of the count on aws_vpc_endpoint.api-gw.network_interface_ids.
Related
I have a global Aurora RDS cluster that takes automated snapshot everyday. My DB instance looks like this :-
new-test ( Global Database )
new-test-db ( Primary Cluster )
new-test-db-0 ( Writer Instance )
I have enabled automated snapshots for the db. What i am trying to achieve is to get the ARN for my snapshot using data resource. My ARN is something like this :-
arn:aws:rds:us-west-2:123456789101:cluster-snapshot:rds:new-test-db-2022-08-23-08-06
This is what my data resource looks like :-
data "aws_db_cluster_snapshot" "db" {
for_each = toset(var.rds_sources)
db_cluster_identifier = each.key
most_recent = true
}
where var.rds_sources is a list of strings. But when i try to access the arn using :-
data.aws_db_cluster_snapshot.db[*].db_cluster_snapshot_arn
I keep running into
Error: Unsupported attribute
│
│ on ../main.tf line 73, in resource "aws_iam_policy" "source_application":
│ 73: cluster_data_sources = jsonencode(data.aws_db_cluster_snapshot.db[*].db_cluster_snapshot_arn)
│
│ This object does not have an attribute named "db_cluster_snapshot_arn".
Which is weird since the attribute is laid out in the official docs. Thank you for the help.
This is my provider file :-
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.75"
}
archive = "~> 2.2.0"
}
required_version = "~> 1.2.6"
}
Since the data source is using for_each, the result will be a map of key value pairs. In terraform, there is a built-in function values [1] which can be used to fetch the values of a map. The return value is a list, so in order to get all the values for all the keys the splat operator is used [2]. Then, since the data source is returning multiple attributes and only one is required (namely db_cluster_snapshot_arn), the final expression needed is as follows:
jsonencode(values(data.aws_db_cluster_snapshot.db)[*].db_cluster_snapshot_arn)
[1] https://www.terraform.io/language/functions/values
[2] https://www.terraform.io/language/expressions/splat
I am working quite a bit with terraform in hopes of building an in-house solution to standing up infra. So far, I have written most of the terraform code and am building out the Ansible code for post-processing of the instances that are stood up. I am shuttling over the dynamic inventory from terraform to Ansible using this little Go app that can be found here, https://github.com/adammck/terraform-inventory. All of that works well.
As I get more into the terraform code, I am trying to use a ternary conditional operator on the ssh key for Linux instances. The goal is to "reuse" this resource on multiple instances.
My resource looks like this ..
resource "aws_key_pair" "key" {
key_name = var.ssh_key
count = var.create_ssh_key ? 1 : 0
public_key = file("~/.ssh/${var.ssh_key}")
}
I've included the [count.index] within the key argument here ...
resource "aws_instance" "linux" {
ami = var.linux_ami
instance_type = var.linux_instance_type
count = var.linux_instance_number
subnet_id = data.aws_subnet.itops_subnet.id
key_name = aws_key_pair.key[count.index].key_name
...
$ terraform validate comes back clean.
$ terraform plan -var-file response-file.tfvars -var "create_ssh_key=false" does not.
The std error is as follows ...
$ terraform plan -var-file response-file.tfvars -var "create_ssh_key=false"
╷
│ Error: Invalid index
│
│ on instances.tf line 16, in resource "aws_instance" "linux":
│ 16: key_name = aws_key_pair.key[count.index].key_name
│ ├────────────────
│ │ aws_key_pair.key is empty tuple
│ │ count.index is 0
│
│ The given key does not identify an element in this collection value.
What am I missing?
Thanks for the feedback!
if count in aws_key_pair is 0, there is no key to reference later on at all.
So you have to check for that and use null to eliminate key_name in such a case:
resource "aws_instance" "linux" {
ami = var.linux_ami
instance_type = var.linux_instance_type
count = var.linux_instance_number
subnet_id = data.aws_subnet.itops_subnet.id
key_name = var.create_ssh_key ? aws_key_pair.key[0].key_name : null
I think the root problem in your example here is that your resource "aws_key_pair" "key" block and your resource "aws_instance" "linux" block both have different values for count, and so therefore it isn't valid to use the count.index of the second to access an instance of the first.
In your case, you seem to have zero key pairs (it says aws_key_pair.key is empty tuple) but you have at least one EC2 instance, and so your expression is trying to access the zeroth instance of the key pair resource, which then fails because it doesn't have a zeroth instance.
If you are using Terraform v0.15 or later then you can use the one function to concisely handle both the zero- and one-instance cases of the key pair resource, like this:
resource "aws_instance" "linux" {
# ...
key_name = one(aws_key_pair.key[*].key_name)
# ...
}
Breaking this down into smaller parts:
aws_key_pair.key[*].key_name is a splat expression which projects from the list of objects aws_key_pair.key into a list of just strings containing key names, by accessing .key_name on each of the objects. In your case, because your resource can only have count 0 or 1, this'll be either a zero-or-one-element list of key names.
one then accepts both of those possible results as follows:
If it's a one-element list, it'll return just that single element no longer wrapped in a list.
If it's a zero-element list, it'll return null which, in a resource argument, means the same thing as not specifying that argument at all.
The effect, then, will be that if you have one key pair then it'll associate that key pair, but if you have no key pairs then it'll leave that argument unset and thus create an instance that doesn't have a key pair at all.
I have a terraform module that does domain delegation. For several variables there is some validation against a hard-coded value to check that a user is using valid inputs, for example:
resource "null_resource" "validate_region" {
count = contains(local.regions, var.region) == true ? 0 : "Please provide a valid AWS region. E.g. (us-west-2)"
}
with local.regions being hard-coded and var.region being a user-set variable. The above code works in that when a user sets the variable wrong, it throws an error like this:
Error: Incorrect value type
on .terraform/foo/main.tf line 46, in resource "null_resource" "validate_region":
46: count = contains(local.regions, var.region) == true ? 0 : "Please provide a valid AWS region. E.g. (us-west-2)"
Invalid expression value: a number is required.
I now need to validate that the AWS account the user is currently using is the correct one. In this case it's up to the user to set the account id of the correct account in their variables, and my code needs to pull the account id of the account that's running the module and compare it against the user's variable. I've tried something like this:
data "aws_caller_identity" "account" {}
resource "null_resource" "validate_account" {
count = data.aws_caller_identity.account.account_id == var.primary_account_id ? 0 : "Please check that you are using the AWS creds for the primary account for this domain."
}
data "aws_route53_zone" "primary" {
name = local.primary_name
}
with various syntax changes on the "{data.aws_caller_identity.account.account_id == var.primary_account_id}" ? 0 part in an effort to get the logic to work, but no luck. I would like it to throw an error like the region validation does, where it will show the error message I wrote. Instead(depending on syntax), it will work as expected for the correct account and throw a Error: no matching Route53Zone found error for the incorrect account, OR it will throw a completely different error presumably because the syntax is screwing things up.
How do I get this to work? is it possible?
What I do is create an if statement in the locals block and source a file with the error message I want to display.
variable "stage" {
type = string
desciption = "The stage to run the deployment in"
}
locals {
stage_validation = var.stage == "prod" || var.stage == "dev"
? var.stage
: file("[Error] this module should only be ran for stages ['prod' or 'dev' ]")
}
The output of setting the stage variable to anything other than 'dev' or 'prod' is as bellow
╷
│ Error: Invalid function argument
│
│ on main.tf line 10, in locals:
│ 10: stage_validation = var.stage == "prod" || var.stage == "dev"
│ ? var.stage
│ : file("[Error] this module should only be ran for stages ['prod' or 'dev' ]")
│
│ Invalid value for "path" parameter: no file exists at This module should only be run for stages ['prod' or 'dev']; this function works only
│ with files that are distributed as part of the configuration source code, so if this file will be created by a resource in this
│ configuration you must instead obtain this result from an attribute of that resource.
╵
This is helpful because it allows you to write an error message that will be shown to the person trying to run the code.
An alternative option here that should simplify what you are doing is to set the region and account constraints up so that Terraform will automatically use the correct region and fail if the credentials are not for the correct account.
You can define this in the aws provider block. An example might look like this:
provider "aws" {
region = "eu-west-1"
allowed_account_ids = ["123456789012"]
}
Now if you attempt to use credentials for a different AWS account then Terraform will fail during the plan stage:
Error: AWS Account ID not allowed: 234567890123
I figured out that this block:
data "aws_route53_zone" "primary" {
name = local.primary_name
}
was running before the account validation resource block. Add in a depends_on like so:
data "aws_route53_zone" "primary" {
name = local.primary_name
depends_on = [null_resource.validate_account,
]
}
And it's all good.
I am trying to dynamically declare multiple aws_nat_gateway data sources by retrieving the list of public subnets through the aws_subnet_ids data source. However, when I try to set the count parameter to be equal to the length of the subnet IDs, I get an error saying The "count" value depends on resource attributes that cannot be determined until apply....
This is almost in direct contradiction to the example in their documentation!. How do I fix this? Is their documentation wrong?
I am using Terraform v0.12.
data "aws_vpc" "environment_vpc" {
id = var.vpc_id
}
data "aws_subnet_ids" "public_subnet_ids" {
vpc_id = data.aws_vpc.environment_vpc.id
tags = {
Tier = "public"
}
depends_on = [data.aws_vpc.environment_vpc]
}
data "aws_nat_gateway" "nat_gateway" {
count = length(data.aws_subnet_ids.public_subnet_ids.ids) # <= Error
subnet_id = data.aws_subnet_ids.public_subnet_ids.ids.*[count.index]
depends_on = [data.aws_subnet_ids.public_subnet_ids]
}
I expect to be able to apply this template successfully, but I am getting the following error:
Error: Invalid count argument
on ../src/variables.tf line 78, in data "aws_nat_gateway" "nat_gateway":
78: count = "${length(data.aws_subnet_ids.public_subnet_ids.ids)}"
The "count" value depends on resource attributes that cannot be determined
until apply, so Terraform cannot predict how many instances will be created.
To work around this, use the -target argument to first apply only the
resources that the count depends on.
It seems you are trying to fetch subnets that weren't created yet or they couldn't be determinated, the terraform cmd output suggests you add -target flag to create the VPC and subnets or do another task first, after that, you'll apply the nat_gateway resource. I suggest you use the AZs list instead of subnets ids, I'll add a simple example below.
variable "vpc_azs_list" {
default = [
"us-east-1d",
"us-east-1e"
]
}
resource "aws_nat_gateway" "nat" {
count = var.enable_nat_gateways ? length(var.azs_list) : 0
allocation_id = "xxxxxxxxx"
subnet_id = "xxxxxxxxx"
depends_on = [
aws_internet_gateway.main,
aws_eip.nat_eip,
]
tags = {
"Name" = "nat-gateway-name"
"costCenter" = "xxxxxxxxx"
"owner" = "xxxxxxxxx"
}
}
I hope will be useful to you and other users.
I'm running into a really confusing Terraform resource issue automating the generation and DNS validation of SSL certificates in ACM for a list of (Terraform-managed) hosted zones. Code can also be found in this gist.
I'm starting by bootstrapping hosted zones referencing this environment-specific variable.
hosted_zones = [
{
domain = "site1.com"
zone_id = "MANUALLY FILL"
}
]
The block I am using to build the zones seems to work reliably.
resource "aws_route53_zone" "zones" {
count = "${length(var.hosted_zones)}"
name = "${lookup(var.hosted_zones[count.index], "domain")}"
}
After the zones are built, I am manually copying the zone ID into the variable because I haven't come up with a clever way to automate it given a combination of limitations of HCL and my lack of experience with it.
I can reliably generate naked and splat certificates for each hosted zone using...
resource "aws_acm_certificate" "cert" {
count = "${length(var.hosted_zones)}"
domain_name = "${lookup(var.hosted_zones[count.index], "domain")}"
subject_alternative_names = ["*.${lookup(var.hosted_zones[count.index], "domain")}"]
validation_method = "DNS"
tags {
Project = "${var.project}"
Environment = "${var.environment}"
}
}
Where things get hairy is when I try to automate the DNS validation for the certs. There is a good example in the documentation for a single hosted zone, but I haven't been able to successfully port it to multiple hosted-zones. My attempt...
resource "aws_route53_record" "cert_validation" {
count = "${length(var.hosted_zones)}"
name = "${aws_acm_certificate.cert.*.domain_validation_options.0.resource_record_name[count.index]}"
type = "${aws_acm_certificate.cert.*.domain_validation_options.0.resource_record_type[count.index]}"
zone_id = "${var.zone_override != "" ? var.zone_override : lookup(var.hosted_zones[count.index], "zone_id")}"
records = ["${aws_acm_certificate.cert.*.domain_validation_options.0.resource_record_value[count.index]}"]
ttl = 60
}
resource "aws_acm_certificate_validation" "cert" {
count = "${length(var.hosted_zones)}"
certificate_arn = "${aws_acm_certificate.cert.*.arn[count.index]}"
validation_record_fqdns = ["${aws_route53_record.cert_validation.*.fqdn[count.index]}"]
}
The error I am seeing on first run is:
* module.acm.aws_route53_record.cert_validation: 1 error(s) occurred:
* module.acm.aws_route53_record.cert_validation: Resource 'aws_acm_certificate.cert' does not have attribute 'domain_validation_options.0.resource_record_value' for variable 'aws_acm_certificate.cert.*.domain_validation_options.0.resource_record_value'
The obnoxious part is that if I comment the validation resources, the apply succeeds, and then uncommenting them and re-running also succeeds.
I've tried (what feels like) every permutation of element() lookup(), list() and map() to target certificates by index in the output from the first resource block, but am running into documented "flat list" limitations and this is the closest I've gotten to success. I'd like to understand why the workaround is necessary so I can eliminate it. This feels like a syntax issue or me trying to get HCL to behave more like an OO language than it is.
Thank you for any experience that may help!
I had a similar scenario and the key to solving it was the use of locals and flatten(). The approach should also work for you such that you shouldn't need two passes to create the resources.
In this scenario there are multiple domains that each have subdomains that will appear in the subjectAltName section of the certificate. For example:
├── preview.example.com
│ ├── app.preview.example.com
│ └── www.preview.example.com
├── demo.example.com
│ ├── app.demo.example.com
│ └── www.demo.example.com
├── staging.example.com
│ ├── app.staging.example.com
│ └── www.staging.example.com
└── example.com
├── app.example.com
└── www.example.com
To achieve this we first set some variables:
variable "domains" {
type = "list"
default = [
"demo.example.com",
"preview.example.com",
"staging.example.com",
"example.com"
]
}
variable "subdomains" {
type = "list"
default = [
"app",
"www"
]
}
Next we create the certificate resources that contain the subdomains as SANs.
resource "aws_acm_certificate" "cert" {
count = "${length(var.domains)}"
domain_name = "${element(var.domains, count.index)}"
validation_method = "DNS"
subject_alternative_names = ["${
formatlist("%s.%s",
var.subdomains,
element(var.domains, count.index)
)
}"]
}
Next we're going to need a local variable to flatten the resulting set of domains and subdomains.
This is needed because terraform doesn't support nested list syntax as of version 0.11.7, neither
via the element() interpolation nor the `list[count].
locals {
dvo = "${flatten(aws_acm_certificate.cert.*.domain_validation_options)}"
}
We'll next need a lookup of the Route 53 zone that we can use in the subsequent Route 53 records:
data "aws_route53_zone" "zone" {
count = "${length(var.domains) > 0 ? 1 : 0}"
name = "example.com."
private_zone = false
}
We then create the Route 53 DNS records that will be populated with data from the certificate
resource for DNS validation. We're adding one to the subdomains so that we also have a
record for the base domain not included in the list of subdomains.
resource "aws_route53_record" "cert_validation" {
count = "${length(var.domains) * (length(var.subdomains) + 1)}"
zone_id = "${data.aws_route53_zone.zone.id}"
ttl = 60
name = "${lookup(local.dvo[count.index], "resource_record_name")}"
type = "${lookup(local.dvo[count.index], "resource_record_type")}"
records = ["${lookup(local.dvo[count.index], "resource_record_value")}"]
}
Finally we create the certificate validation resource that will wait for the certificate to be
issued.
resource "aws_acm_certificate_validation" "cert" {
count = "${length(var.domains) * (length(var.subdomains) + 1)}"
certificate_arn = "${element(aws_acm_certificate.cert.*.arn, count.index)}"
validation_record_fqdns = ["${aws_route53_record.cert_validation.*.fqdn}"]
}
The one caveat for this last resource is that it'll create one instance of the resource for every
certificate requested, but each instance will depend on all the FQDNs across all domains and
subdomains. This won't affect anything in AWS but the terraform code won't continue/complete
until all certs are issued.
This should work in a single apply run with no need to -target any resources in a first pass,
though there is an apparently known issue around how long it takes for the validations to
complete when
performed via terraform, and for this reason it may require a second pass, albeit without changing the code or plan/apply invocation.
So after a bit of experimenting, I ended up leveraging -target=aws_acm_certificate.cert as a workaround to avoid the missing attribute errors I was seeing. The syntax I was using above was correct, and the error was a result of the apply needing to complete for the certificate before the validation steps could reference the generated attributes.
In addition, I found an elegant solution for the MANUAL FILL step using zipmap. The result looks like this...
Variable:
hosted_zones = [
"foo.com"
]
Output from hosted_zones module:
output "hosted_zone_ids" {
value = "${zipmap(var.hosted_zones, aws_route53_zone.zones.*.zone_id)}"
}
Then, my certificate generation/validation module looks like this, where var.hosted_zone_map is the output of the previous zipmap that creates a map of hosted zone domain name to assigned zone ID:
resource "aws_acm_certificate" "cert" {
count = "${length(keys(var.hosted_zone_map))}"
domain_name = "${element(keys(var.hosted_zone_map), count.index)}"
subject_alternative_names = ["*.${element(keys(var.hosted_zone_map), count.index)}"]
validation_method = "DNS"
tags {
Project = "${var.project}"
Environment = "${var.environment}"
}
}
resource "aws_route53_record" "cert_validation" {
count = "${length(keys(var.hosted_zone_map))}"
zone_id = "${lookup(var.hosted_zone_map, element(keys(var.hosted_zone_map), count.index))}"
name = "${aws_acm_certificate.cert.*.domain_validation_options.0.resource_record_name[count.index]}"
type = "${aws_acm_certificate.cert.*.domain_validation_options.0.resource_record_type[count.index]}"
records = ["${aws_acm_certificate.cert.*.domain_validation_options.0.resource_record_value[count.index]}"]
ttl = 60
}
resource "aws_acm_certificate_validation" "cert" {
count = "${length(keys(var.hosted_zone_map))}"
certificate_arn = "${aws_acm_certificate.cert.*.arn[count.index]}"
validation_record_fqdns = ["${aws_route53_record.cert_validation.*.fqdn[count.index]}"]
}
The positioning of the splat was definitely the trickiest and least documented part of tracking this down, so hopefully this helps someone else out.
I know this question is old, but for anyone searching for answers today, Terraform's updated documentation for the AWS provider has a great example for this using a for_each loop. This applies to terraform 1.x, obviously, but maybe other recent Check out https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/acm_certificate_validation#dns-validation-with-route-53