Merge more than 2 tfvars file contents - amazon-web-services

I have usecase where I have 3 tfvars files
1.tfvars
api_stages=[
{
api_id= "xxxx"
stage="play"
}
]
2.tfvars
api_stages=[
{
api_id= "yyy"
stage="play"
}
]
3.tfvars
api_stages=[
{
api_id= "zzz"
stage="play"
}
]
and I am creating 1 single usage plan using this
resource "aws_api_gateway_usage_plan" "usage_plan" {
name = var.usage_plan_name
throttle_settings {
burst_limit = var.burst_limit
rate_limit = var.rate_limit
}
dynamic "api_stages" {
for_each = var.api_stages
content {
api_id = api_stages.value.api_id
stage = api_stages.value.stage
}
}
}
And this tfvars files will be created by different users.
Question: How can I merge this different tfvars file and attach all api_id and stages to single usage plan?
right now when I am doing plan, it's only taking last file's value.

.tfvars file processing happens before any configuration expressions are evaluated, so there is no way to do exactly what you asked about here: the rule for .tfvars files is always that the last definition of a particular variable "wins", as you've seen.
The some options that are close to what you want here would be:
Define three different variables, perhaps named to indicate the three different sources that are providing these values, and then concatenate the three of them together in a local value:
locals {
all_api_stages = concat(
var.a_api_stages,
var.b_api_stages,
var.c_api_stages,
)
}
This approach would be appropriate only if there being exactly three sets of variables is a meaningful constraint for your system.
Declare the actual stage objects inside your configuration and make the variable instead be a list of keys for them:
variable "api_stages" {
type = set(string)
}
locals {
api_stages = {
x = {
api_id = "xxxx"
stage = "play"
}
y = {
api_id = "yyyy"
stage = "play"
}
z = {
api_id = "zzzz"
stage = "play"
}
}
}
You can then use -var when running Terraform to choose the appropriate settings, instead of using -var-file:
terraform apply -var='api_stages=["x", "y"]'

Related

For loop in terraform

I am trying to get the value of function name from local.tf but I am not able to get it. I have terraform,tfvars in which I am giving the function name then it is passed to variable.tf. From varibale.tf I pass it to local.tf then to main.tf. I am not able to get the function name in main.tf. Any help would be appreciated.
terraform.tfvars
config = {
s3= {
//s3 configurations
}
s3_notifications = {
function_name = "test-lambda-mary"
}
}
variable.tf
variable "config" {
type = any
description = "S3 configuration block"
}
local.tf
function_name = {
for k, v in var.config :
k => lookup(v, "function_name", "")
}
module "all_notifications" {
source = "terraform-aws-modules/s3-bucket/aws//modules/notification"
for_each = var.config
bucket = module.s3_bucket[each.key].this_s3_bucket_id
lambda_notifications = {
lambda = {
function_name = local.function_name[each.key]
function_arn = "arn:aws:lambda:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:function:${local.function_name[each.key]}"
events = ["s3:ObjectCreated:*"]
}
}
}
error
"function_name" doesn't comply with restrictions ("^(arn:[\\w-]+:lambda:)?([a-z]{2}-(?:[a-z]+-){1,2}\\d{1}:)?(\\d{12}:)?(function:)?([a-zA-Z0-9-_]+)(:(\\$LATEST|[a-zA-Z0-9-_]+))?$"): ""
│
│ with module.all_notifications["s3"].aws_lambda_permission.allow["lambda"],
│ on .terraform/modules/all_notifications/modules/notification/main.tf line 63, in resource "aws_lambda_permission" "allow":
│ 63: function_name = each.value.function_name
If I put the function_name inside s3 braces it works absolutely fine but I need to have the fucntion name in s3_notification
That looks like a great hint. You're iterating over var.config which has 2 keys and only 1 of them has function_name defined. So when module is requested with s3 as a key, the function_value for that key will be empty string and AWS will fail the request as expected.
You can filter for_each = var.config to exclude such case, something like:
for_each = { for k, v in var.config: k => v if local.function_name[each.key] != ""}
Little nitpick: seems like the source of the module could be incorrectly written. Instead of terraform-aws-modules/s3-bucket/aws//modules/notification potentially it should be terraform-aws-modules/terraform-aws-s3-bucket//modules/notification. See https://github.com/terraform-aws-modules/terraform-aws-s3-bucket
A stab in the dark as I haven't used that module before.
But by looking at your error message:
"function_name" doesn't comply with restrictions ("^(arn:[\\w-]+:lambda:)?
it looks like for function_name you should pass Lambda's ARN, not name (contrary to what the variable name says).
BTW, is function_arn even a parameter here?
This error is coming since terraform module is expecting a valid and mandatory function name to create [aws_lambda_permission][1] resource using terraform-aws-modules/s3-bucket/aws//modules/notification module.
In your case you are looping the module on var.config which consist s3 and s3_notifications part and while first iteration its getting function name as null therefore its throwing this error.
"function_name" doesn't comply with restrictions ("^(arn:[\\w-]+:lambda:)?([a-z]{2}-(?:[a-z]+-){1,2}\\d{1}:)?(\\d{12}:)?(function:)?([a-zA-Z0-9-_]+)(:(\\$LATEST|[a-zA-Z0-9-_]+))?$"): ""
Better split the variable specific to s3_notification as list and iterate the module as below using count.
variable "s3_config" {
type = any
default = {
s3 = {
}
}
}
variable "s3_notify_lambda_func" {
type = list
default = ["test-lambda-mary","test"]. #N Numbers of functions
}
data "aws_region" "current" {}
data "aws_caller_identity" "current" {}
module "all_notifications" {
source = "terraform-aws-modules/s3-bucket/aws//modules/notification"
count = length(var.s3_notify_lambda_func) > 0 ? length(var.s3_notify_lambda_func) : 0
bucket = module.s3_bucket[count.index].this_s3_bucket_id
lambda_notifications = {
lambda = {
function_name = var.s3_notify_lambda_func[count.index]
function_arn = "arn:aws:lambda:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:function:${var.s3_notify_lambda_func[count.index]}"
events = ["s3:ObjectCreated:*"]
}
}
}

Terraform - don't create resource if data source does not exist

I'm using the following set up to iterate through my locals. Certain parameters should only be filled in if terraform can grab the data resource. If the data resource DOES NOT EXIST, then it is noted in the parameter and then the resource creation is skipped.
#Only get the data resource if it exists#################################
data "aws_ssm_parameter" "example_parameter" {
count = "${var.does_ssm_parameter_exist == true ? 1 : 0}"
name = "ssm_parameter"
}
#List of parameters for all config rules
locals {
config_rule_params = {
"access_keys_rotated" = {
"input_parameters" = "example"
},
"acm_certificate_expiration_check" = {
#ERROR! Get input parameters from data source if it exists#################################
"input_parameters" = "${var.does_ssm_parameter_exist == "true" ? "${data.aws_ssm_parameter.example_parameter[count.index].value}" : "DOES_NOT_EXIST"}"
}
}
#Only create config rule if input parameters exist
resource "aws_config_config_rule" "parameterised_config_rules" {
for_each = {
for rule, params in local.config_rule_params : rule => params
if params.input_parameters != "DOES_NOT_EXIST"
}
input_parameters = each.value.input_parameters
}
Unfortunately, it seems like I cannot use count.index in this way:
Error: Reference to "count" in non-counted context
"input_parameters" = "${var.does_ssm_parameter_exist == "true" ? "${data.aws_ssm_parameter.example_parameter[count.index].value}" : "DOES_NOT_EXIST"}"
The "count" object can be used only in "resource" and "data" blocks, and only when the "count" argument is set.
Your use of count.index in locals is incorrect. count can be used in resources and modules only, not locals. Thus you have to explicitly specify which parameter index do you want as follows:
"input_parameters" = "${var.does_ssm_parameter_exist == "true" ? "${data.aws_ssm_parameter.example_parameter[0].value}" : "DOES_NOT_EXIST"}"
Depending on the nature of your example_parameter you many need to have regular loop or use splat expression to get all its values.

Outputs from for_each loop for each resource

I am having a hard time figuring out how to make an output for each target group resource that this code creates.
I'd like to be able to reference each one individually in other modules. It sounds like for_each stores it as a map, so my question is how would I get the arn for targetgroup1 and targetgroup2?
Terraform normally refers to outputs by resource name, so I am struggling with that in this scenario and also how to refer to these individual arns.
Would I also need to work the outputs into the for_each or could I drop it into the output.tf file?
locals {
target_groups_beta = {
targetgroup1 = {
name = "example",
path = "/",
environment = "Beta"
}
targetgroup2 = {
name = "example2",
path = "/",
environment = "Beta"
}
}
}
resource "aws_lb_target_group" "target-group" {
for_each = local.target_groups_beta
name = "example-${each.value.name}-"
port = 80
protocol = "HTTP"
vpc_id = var.vpc-id
deregistration_delay = 5
tags = {
Environment = "${each.value.environment}"
}
health_check{
healthy_threshold = 2
unhealthy_threshold = 2
timeout = 10
interval = 15
path = each.value.path
}
}
I receive the following error when trying to do it in the output.tf file without a key value, but when I input one such as value = "${aws_lb_target_group.target-group[0].arn}" it says it's invalid. Error without key value below:
Error: Missing resource instance key
on modules\targetgroups\output.tf line 2, in output "tg_example_beta":
2: value = "${aws_lb_target_group.target-group.arn}"
Because aws_lb_target_group.target-group has "for_each" set, its attributes
must be accessed on specific instances.
For example, to correlate with indices of a referring resource, use:
aws_lb_target_group.target-group[each.key]
The aws_lb_target_group.target-group generated will be a map, with key values of targetgroup2 and targetgroup1.
Therefore, to get the individual target group details you can do:
output "target-group1-arn" {
value = aws_lb_target_group.target-group["targetgroup1"].arn
}
To return both as a map:
output "target-groups-arn-alternatice" {
value = {for k, v in aws_lb_target_group.target-group: k => v.arn}
}
target-groups-arn-alternatice = {
"targetgroup1" = "arn:aws:elasticloadbalancing:us-east-1:xxxx:targetgroup/example-example/285b26e15221b113"
"targetgroup2" = "arn:aws:elasticloadbalancing:us-east-1:xxxx:targetgroup/example-example2/075bd58359e4c4b2"
}
To return both as a list (order will be same as for keys function):
output "target-groups-arn" {
value = values(aws_lb_target_group.target-group)[*].arn
}
target-groups-arn = [
"arn:aws:elasticloadbalancing:us-east-1:xxxx:targetgroup/example-example/285b26e15221b113",
"arn:aws:elasticloadbalancing:us-east-1:xxxx:targetgroup/example-example2/075bd58359e4c4b2",
]

Deal will dict Using Maps in terraform

I'm looking for way to define a list of ssh keys in a variables file so that I could retrieve them in the tf module code for my compute instance like this :
metadata = {
ssh-keys = join("\n", [for user, key in var.ssh_keys : "${user}:${key}"])
}
Here is the content of the variables file I wrote to achieve that :
variable "ssh_keys" {
type = "map"
default = {
{
user = "amary"
key = "${file("/Users/nixmind/.ssh/amary.pub")}"
}
{
user = "nixmind"
key = "${file("/Users/nixmind/.ssh/nixmind.pub")}"
}
}
}
But I'm having this error :
Error: Missing attribute value
on variables.tf line 8, in variable "ssh_keys":
4:
5:
6:
7:
8:
9:
Expected an attribute value, introduced by an equals sign ("=").
I'm not sure to really get what to do there.
There are a few different problems here. I'll talk about them one at a time.
The first is that your default expression is not using correct map syntax. Here's a corrected version:
variable "ssh_keys" {
type = map(string)
default = {
amary = file("/Users/nixmind/.ssh/amary.pub")
nixmind = file("/Users/nixmind/.ssh/nixmind.pub")
}
}
The second problem is that a variable default value cannot include function calls, so the calls to file above are invalid. There are a few different options here about how to deal with this, but if this is a variable in a root module then I expect it would be most convenient to have the variable be a map of filenames rather than a map of the contents of those files, and then the module itself can read the contents of those files in a later step:
variable "ssh_key_files" {
type = map(string)
default = {
amary = "/Users/nixmind/.ssh/amary.pub"
nixmind = "/Users/nixmind/.ssh/nixmind.pub"
}
}
Your for expression for building the list of "user:key" strings was correct with how you had the variable defined before, but with the adjustment I've made above to use filenames instead of contents we'll need an extra step to actually read the files:
locals {
ssh_keys = { for u, fn in var.ssh_key_files : u => file(fn) }
}
We can then use local.ssh_keys to get the map from username to key needed for the metadata expression:
metadata = {
ssh-keys = join("\n", [for user, key in local.ssh_keys : "${user}:${key}"])
}
If you do want this module to accept already-loaded SSH key data rather than filenames then that is possible but the variable will need to be required rather than having a default, because it'll be up to the calling module to load the files.
The definition without the default value will look like this:
variable "ssh_keys" {
type = map(string)
}
Then your calling module, if there is one (that is, if this isn't a root module) can be the one to call file to load those in:
module "example" {
source = "./modules/example"
ssh_keys = {
amary = file("/Users/nixmind/.ssh/amary.pub")
nixmind = file("/Users/nixmind/.ssh/nixmind.pub")
}
}
The above is a reasonable interface for a shared module that will be called from another module like this, but it's not a convenient design for a root module because the "caller" in that case is the person or script running the terraform program, and so providing the data from those files would require reading them outside of Terraform and passing in the results.
My problem was semantic not syntactic, cause I can't use a map of map as tried to. Instead I used a liste and the error desapeared.
variable ssh_keys {
type = list(object({
user=string
key=string
}))
default = [
{
"user" = "amaret93"
"key" = "/Users/valerietchala/.ssh/amaret93.pub"
},
{
"user" = "nixmind"
"key" = "/Users/valerietchala/.ssh/nixmind.pub"
}
]
}
But the Martin's answer above is also a more good approach

How to output all the resources of one type in Terraform?

I have a bunch of aws_ecr_repositories defined in my Terraform code:
resource "aws_ecr_repository" "nginx_images" {
name = "nginx-test"
}
resource "aws_ecr_repository" "oracle_images" {
name = "oracle-test"
}
I want to be able to have an output that can list all the aws_ecr_repository resources into one output. This is what I tried:
output "ecr_repository_urls" {
value = "[${aws_ecr_repository.*.repository_url}]"
}
This does not work because Terraform does not seem to allow wildcards on the resource names. Is it possible to have an output like this? My current solution is to just list outputs for every resource defined.
Terraform's splat syntax is for keeping track of each thing created by a resource using the count meta parameter.
If you want to be able to get at all of the respoitory URLs you could have a single aws_ecr_repository resource and use the count meta parameter with something like this:
variable "images" {
default = [
"nginx-test",
"oracle-test",
]
}
resource "aws_ecr_repository" "images" {
count = "${length(var.images)}"
name = "${var.images[count.index]}"
}
output "ecr_repository_urls" {
value = "[${aws_ecr_repository.images.*.repository_url}]"
}
You can combine them manually as a list:
output "ecr_repository_urls" {
value = ["${aws_ecr_repository.nginx_images.repository_url}", "${aws_ecr_repository.oracle_images.repository_url}"]
}
Although it probably won't be pretty in code.
You could also do something like this:
variable "ecr_repos" {
default = {
"0" = "foo"
"1" = "bar"
}
}
resource "aws_ecr_repository" "images" {
count = "${length(var.ecr_repos)}"
name = "${lookup(var.ecr_repos,count.index)}-test"
}
output "ecr_repository_urls" {
value = "${aws_ecr_repository.images.*.repository_url}"
}
But the problem is if the list order changes it's going to recreate resources and get really ugly really fast since each repo is assigned to an index number.