I create an AWS RDS instance with different KMS CMKs depending on whether or not the environment is Production or Non-Production. So I have two resources that use the terraform count if:
count = "${var.bluegreen == "nonprod" ? 1 : 0}"
This spins up an RDS instance with different KMS keys with different addresses. I need to capture that endpoint (which I do with terraform show after the build finishes) so why doesn't this work in Terraform?
output "rds_endpoint" {
value = "${var.bluegreen == "nonprod" ? aws_db_instance.rds_nonprod.address : aws_db_instance.rds_prod.address}"
}
It is an error to access attributes of a resource that has count = 0, and unfortunately Terraform currently checks both "sides" of a conditional during its check step, so expressions like this can fail. Along with this, there is a current behavior that errors in outputs are not explicitly shown since outputs can get populated when the state isn't yet complete (e.g. as a result of using -target). These annoyances all sum up to a lot of confusion in this case.
Instead of using a conditional expression in this case, it works better to use "splat expressions", which evaluate to an empty list in the case where count = 0. This would looks something like the following:
output "rds_endpoint" {
value = "${element(concat(aws_db_instance.rds_nonprod.*.address, aws_db_instance.rds_prod.*.address), 0)}"
}
This takes the first element of a list created by concatenating together all of the nonprod addresses and all of the prod addresses. Due to how you've configured count on these resource blocks, the resulting list will only ever have one element and so it will just take that element.
In general, to debug issues with outputs it can be helpful to evaluate the expressions in terraform console, or somewhere else in a config, to bypass the limitation that errors are silently ignored on outputs.
Related
I have been trying to create a step function with a choice step that acts as a rule engine. I would like to compare a date variable (from the stale input JSON) to another date variable that I generate with a lambda function.
AWS documentation does not go into details about the Timestamp comparator functions, but I assumed that it can handle two input variables. Here is the relevant part of the code:
{
"Variable": "$.staleInputVariable",
"TimestampEquals": "$.generatedTimestampUsingLambda"
}
Here is the error that I am getting when trying to update(!!!) the stepFunction. I would like to highlight the fact that I don't even get to invoking the stepFunction as it fails while updating the function.
Resource handler returned message: "Invalid State Machine Definition: 'SCHEMA_VALIDATION_FAILED: String does not match RFC3339 timestamp at ..... (Service: AWSStepFunctions; Status Code: 400; Error Code: InvalidDefinition; Request ID: 97df9775-7d2d-4dd2-929b-470c8s741eaf; Proxy: null)" (RequestToken: 030aa97d-35a5-a6a5-0ac5-5698a8662bc2, HandlerErrorCode: InvalidRequest)
The stepfunction updates without the Timestamp matching, therefore, I suspect this bit of code.. Any guess?
EDIT (08.Jun.2021):
A comparison – Two fields that specify an input variable to compare,
the type of comparison, and the value to compare the variable to.
Choice Rules support comparison between two variables. Within a Choice
Rule, the value of Variable can be compared with another value from
the state input by appending Path to name of supported comparison
operators.
Source: AWS docs
It clearly states that two variables can be compared, but to no avail. Still trying :)
When I explained the problem to one of my peers, I realised that the AWS documentation mentions a Path postfix (which I confused with the $.). This Path needs to be added to the operatorName.
The following code works:
{
"Variable": "$.staleInputVariable",
"TimestampEqualsPath": "$.generatedTimestampUsingLambda"
}
Again, I would like to draw your attention to the "Path" word. That makes the magic!
Looks like you indeed found a way around your initial challenge from the thread linked below.
Using an amazon step function, how do I write a choice operator that references current time?
However, I thought you wanted to compare $.staleInputVariable to the current timestamp and I wince to think you had to configure a lambda function (and test it!) to do only that.
If so, you could have achieved that simply by using the Context Object or $$.:
{
"Variable": "$$.State.EnteredTime",
"TimestampEqualsPath": "$.staleInputVariable"
}
we have a module that builds a security proxy that hosts an elasticsearch site using terraform. In its code there is this;
elastic_search_endpoint = "${element(concat(module.es_cluster.elasticsearch_endpoint, list("")),0)}"
which as I understand, then goes and finds the es_cluster module and gets the elasticsearch endpoint that was outputted from that. This then allows the proxy to have this endpoint available so it can run elasticsearch.
But I don't actually understand what this piece of code is doing and why the 'element' and 'concat' functions are there. Why can't it just be like this?
elastic_search_endpoint = "${module.es_cluster.elasticsearch_endpoint}"
Let's break this up and see what each part does.
It's not shown in the example, but I'm going to assume that module.es_cluster.elasticsearch_endpoint is an output value that is a list of eitehr zero or one ElasticSearch endpoints, presumably because that module allows disabling the generation of an ElasticSearch endpoint.
If so, that means that module.es_cluster.elasticsearch_endpoint would either be [] (empty list) or ["es.example.com"].
Let's consider the case where it's a one-element list first: concat(module.es_cluster.elasticsearch_endpoint, list("")) in that case will produce the list ["es.example.com", ""]. Then element(..., 0) will take the first element, giving "es.example.com" as the final result.
In the empty-list case, concat(module.es_cluster.elasticsearch_endpoint, list("")) produces the list [""]. Then element(..., 0) will take the first element, giving "" as the final result.
Given all of this, it seems like the intent of this expression is to either return the one ElasticSearch endpoint, if available, or to return an empty string as a placeholder if not.
I expect this is written this specific way because it was targeting an earlier version of the Terraform language which had fewer features. A different way to write this expression in current Terraform (v0.14 is current as of my writing this) would be:
elastic_search_endpoint = (
length(module.es_cluster.elasticsearch_endpoint) > 0 ? module.es_cluster.elasticsearch_endpoint : ""
)
It's awkward that this includes the full output reference twice though. That might be justification for using the concat approach even in modern Terraform, although arguably the intent wouldn't be so clear to a future reader:
elastic_search_endpoint = (
concat(module.es_cluster.elasticsearch_endpoint, "")[0]
)
Modern Terraform also includes the possibility of null values, so if I were writing a module like yours today I'd probably prefer to return a null rather than an empty string, in order to be clearer that it's representing the absense of a value:
elastic_search_endpoint = (
length(module.es_cluster.elasticsearch_endpoint) > 0 ? module.es_cluster.elasticsearch_endpoint : null
)
elastic_search_endpoint = (
concat(module.es_cluster.elasticsearch_endpoint, null)[0]
)
First things first: who wrote that code? Why is not documented? Ask the guy!
Just from that code... There's not much to do. I'd say that since concat expects two lists, module.es_cluster.elasticsearch_endpoint is a list(string). Also, depending on some variables, it might be empty. Concatenating an empty string will ensure that there's something at 0 position
So the whole ${element(concat(module.es_cluster.elasticsearch_endpoint, list("")),0)} could be translated to length(module.es_cluster.elasticsearch_endpoint) > 0 ? module.es_cluster.elasticsearch_endpoint[0] : "" (which IMHO is much readable)
Why can't it just be like this?
elastic_search_endpoint = "${module.es_cluster.elasticsearch_endpoint}"
Probably because elastic_search_endpoint is an string and, as mentioned before, module.es_cluster.elasticsearch_endpoint is a list(string). You should provide a default value in case the list is empty
In terraform I have 2 data outputs:
data "aws_instances" "daas_resolver_ip_1" {
instance_tags = {
Name = "${var.env_type}.${var.environment}.ns1.${var.aws_region}.a."
}
}
data "aws_instances" "daas_resolver_ip_2" {
instance_tags = {
Name = "${var.env_type}.${var.environment}.ns2.${var.aws_region}.b."
}
}
I want to get the private_ip from each of those combine those into a list and be used as follows:
dhcp_options_domain_name_servers = ["${data.aws_instances.daas_resolver_ip_1.private_ip}", "${data.aws_instances.daas_resolver_ip_1.private_ip}"]
How can I achieve this? At the moment this is the error I get:
Error: module.pmc_environment.module.pmc_vpc.aws_vpc_dhcp_options.vpc: domain_name_servers: should be a list
I believe what you've encountered here is a common limitation of Terraform 0.11. If this is a new configuration then starting with Terraform 0.12 should avoid the problem entirely, as this limitation was addressed in the Terraform 0.12 major release.
The underlying problem here is that the private_ip values of at least one of these resources is unknown during planning (it will be selected by the remote system during apply) but then Terraform 0.11's type checker is failing because it cannot prove that these unknown values will eventually produce a list of strings as the dhcp_options_domain_name_servers requires.
Terraform 0.12 addresses this by tracking type information for unknown values and propagating types through expressions so that e.g. in this case it could know that the result is a list of two strings but the strings themselves are not known yet. From Terraform 0.11's perspective, this is just an unknown value with no type information at all, and is therefore not considered to be a list, causing this error message.
A workaround for Terraform 0.11 is to use the -target argument to ask Terraform to deal with the operations it needs to learn the private_ip values first, and then run Terraform again as normal once those values are known:
terraform apply -target=module.pmc_environment.module.pmc_vpc.data.aws_instances.daas_resolver_ip_1 -target=module.pmc_environment.module.pmc_vpc.data.aws_instances.daas_resolver_ip_2
terraform apply
The first terraform apply with -target set should deal with the two data resources, and then the subsequent terraform apply with no arguments should then be able to see what the two IP addresses are.
This will work only if all of the values contributing to the data resource configurations remain stable after the initial creation step. You'd need to repeat this two-step process on subsequent changes if any of var.env_type, var.environment, or var.aws_region become unknown as a result of other planned actions.
I am facing a problem with Terraform (v0.12) to create multiple instances using count variable and subnet id's list, where the count is greater than the length of the subnet id's list.
For example;
resource "aws_instance" "main" {
count = 20
ami = var.ami_id
instance_type = var.instance_type
# ...
subnet_id = var.subnet_ids_list[count.index]
}
Where my count is '20' and length(var.subnet_ids_list) is 2. It throws the following error:
count.index is 2
var.instance_subnet_id is tuple with 2 elements
The given key does not identify an element in this collection value.
I tried to make the "subnet_ids_list" as string with comma-separated and used "split", but it too give the same error.
Later thought to append subnet elements to "subnet_ids_list" in order to make it to "20". something like;
Python 2.7
>>> subnet_ids_list = subnet_ids_list * 10
Can someone help me with how to achieve similar with Terraform or any other approaches to solve this problem.
Original like;
subnet_ids_list = ["sub-1", "sub-2"]
Converted to - satisfy the value provided to count;
subnet_ids_list = ["sub-1", "sub-2", "sub-1", "sub-2",....., "sub-1", "sub-2",] (length=20).
I don't want to use AWS autoscaling groups for this purpose.
You can use the element function if you need to loop back through a list of things as mentioned in the linked documentation:
The index is zero-based. This function produces an error if used with
an empty list.
Use the built-in index syntax list[index] in most cases. Use this
function only for the special additional "wrap-around" behavior
described below.
> element(["a", "b", "c"], 3)
a
It doesn't make sense to create a new subnet whenever you need to spin up a new EC2. I'd recommend you to take a look at the official documentation about the basics of VPC and subnets: https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html#vpc-subnet-basics
For example, if you create a VPC with CIDR block 10.0.0.0/24, it supports 256 IP addresses. You can break this CIDR block into two subnets, each supporting 128 IP addresses. One subnet uses CIDR block 10.0.0.0/25 (for addresses 10.0.0.0 - 10.0.0.127) and the other uses CIDR block 10.0.0.128/25 (for addresses 10.0.0.128 - 10.0.0.255).
In your Terraform example, it looks like you have 2 subnets (private and public?), so your counter must be rather 0 or 1 when accessing subnet_ids_list. Even a better solution would be to tag your subnets: https://www.terraform.io/docs/providers/aws/r/subnet.html#inner
You might have another counter though to control number of instances. Hope it helps!
EDIT: Based on your comments, a Map would be a better data structure to control instance/subnet. Key could be the instance or the subnet itself, e.g. { "aws_instance" = "sub-1" }
Reference: https://www.terraform.io/docs/configuration-0-11/variables.html#maps
I'm using Terraform with AWS as a provider.
I want to use a ternary operator in my availability zones local variable.
The logic is simple:
If a variable exist - take it.
If not, use the availability zones data.
The following code:
data "aws_availability_zones" "available" {}
locals {
azs = "${length(var.azs) > 0 ? var.azs : data.aws_availability_zones.available.names}"
}
variable "azs" {
description = "A list of Availability zones in the region"
default = []
type = "list"
}
Generates the following error:
conditional operator cannot be used with list values.
Although its quiet a simple operation, It turns out like a familiar issue.
I followed the work-arounds in the mentioned thread, but they looked looked quiet complicated (Using compact split and join functions together).
Any suggestions for more simple solution?
Thank you.
you are close to the answer.
Not sure how you define the variable var.azs, I guess they are defined as string and connected with commas.
So you need adjust the code, join the list to string.
locals {
azs = "${length(var.azs) > 0 ? var.azs : join(",", data.aws_availability_zones.available.names)}"
}