How to get IP addresses from several variables in Terraform - amazon-web-services

I tried to do so
private_ip_ws0 = ["10.0.10.3"]
private_ip_ws1 = ["10.0.20.3"]
private_ip_db0 = ["10.0.11.3"]
private_ip_db1 = ["10.0.21.3"]
resource "aws_network_interface" "ws_interface" {
count = 2
subnet_id = aws_subnet.ws_net_public[count.index].id
private_ips = "${var.private_ip_ws}[count.index]"[0]
tags = {
Name = "${var.environment}${count.index}-Interface"
}
}
But I get error:
Error: Reference to undeclared input variable
│
│ on vpc.tf line 55, in resource "aws_network_interface" "ws_interface":
│ 55: private_ips = "${var.private_ip_ws}[count.index]"[0]
│
│ An input variable with the name "private_ip_ws" has not been declared. Did you mean "private_ip_ws1"?
How to write code correctly?
And why in one place Terraform allows you to write like this
cidr_block = "${var.public_cidr}${count.index+1}0.0/24"
but in another similar expression it doesn't allow to write:
private_ips = "${var.private_ip_ws}[count.index]"[0]
Why?

You can't reference a variable name via a variable like that. That just isn't a feature supported by Terraform at all. The correct way to do this would be to convert your variables into arrays, but if you want to keep the variables the same you will need to wrap them in a local array before using them:
private_ip_ws0 = ["10.0.10.3"]
private_ip_ws1 = ["10.0.20.3"]
private_ip_db0 = ["10.0.11.3"]
private_ip_db1 = ["10.0.21.3"]
locals {
private_ip_ws = [private_ip_ws0, private_ip_ws1]
}
resource "aws_network_interface" "ws_interface" {
count = 2
subnet_id = aws_subnet.ws_net_public[count.index].id
private_ips = local.private_ip_ws[count.index]
tags = {
Name = "${var.environment}${count.index}-Interface"
}
}
And why in one place Terraform allows you to write like this
cidr_block = "${var.public_cidr}${count.index+1}0.0/24"
but in another similar expression it doesn't allow to write:
private_ips = "${var.private_ip_ws}[count.index]"[0]
Because in the first example you are just doing basic string interpolation, which is perfectly valid Terraform syntax.
In the second example you are building a dynamic variable name and then trying to reference the value inside that variable, which is completely invalid Terraform syntax.

Related

How to define a resource block with for_each that refers to other resources

I create few resources in the terraform -
resource "aws_dms_replication_instance" "foobar_instance_1" {
}
resource "aws_dms_replication_instance" "foobar_instance_2" {
}
resource "aws_dms_endpoint" "foobar_source_1" {
}
resource "aws_dms_endpoint" "foobar_source_2" {
}
Then define a replication task that's dependent on above two resources -
resource "aws_dms_replication_task" "foobar_task_1" {
replication_instance_arn = aws_dms_replication_instance.foobar_instance_1.replication_instance_arn
source_endpoint_arn = aws_dms_endpoint.foobar_source_1.endpoint_arn
.
.
.
}
I want to use a tf variable to automate a similar aws_dms_replication_task creation using for_each. I create a Map variable in variables.tf that
looks like this -
variable "task_map" {
type = map(object({
source_endpoint = string
repl_instance = string
}))
default = {
"dms_attr_map" = {
source_ep = "foobar_source_1"
repl_instance = "foobar_instance_1"
},
"dms_attr_map2" = {
source_ep = "foobar_source_2"
repl_instance = "foobar_instance_2"
}
}
}
Now, I go on to create a resource block to loop over task_map and create replication_task
resource "aws_dms_replication_task" "for_each_task_1" {
for_each = var.task_map
replication_instance_arn = aws_dms_replication_instance.${each.value["repl_instance"]}.replication_instance_arn
source_endpoint_arn = aws_dms_endpoint.${each.value["source_ep"]}.endpoint_arn
.
.
.
}
As I execute my plan, terrraform plan throws an error that referring to replication_instance using ${each.value["repl_instance"]} is wrong.
Error message -
│ Error: Invalid character │ │ On ../modules/cdc/main.tf line 236:
This character is not used within the │ language. ╵
╷ │ Error: Invalid attribute name │ │ On ../modules/cdc/main.tf line
233: An attribute name is required after a │ dot. ╵
Error messages point to the specific line where I use for_each to refer to replication_instance and source_endpoint
aws_dms_endpoint.${each.value["source_ep"]}.endpoint_arn
aws_dms_replication_instance.${each.value["repl_instance"]}.replication_instance_arn
How do I refer to the resources created using their names using a for_each.
Thank you.
How do I refer to the resources created using their names using a for_each.
You can't do this. In other words, you can't create dynamic references to resources in the form of:
aws_dms_endpoint.${each.value["source_ep"]}.endpoint_arn
Instead you have to use maps or lists to create your aws_dms_replication_instance and aws_dms_endpoint. For example:
resource "aws_dms_replication_instance" "foobar_instance" {
for_each = toset(["foobar_instance_1", "foobar_instance_2"])
}
resource "aws_dms_endpoint" "foobar_source" {
for_each = toset(["foobar_source_1", "foobar_source_2"])
}
end then refer to them as follows:
resource "aws_dms_replication_task" "for_each_task_1" {
for_each = var.task_map
replication_instance_arn = aws_dms_replication_instance.foobar_instance[each.value["repl_instance"]].replication_instance_arn
source_endpoint_arn = aws_dms_endpoint.foobar_source[each.value["source_ep"]].endpoint_arn
.
.
.
}
When thinking about problems like this it's important to recognize that a reference expression like aws_dms_replication_instance.foobar_instance_1 is an indivisible unit: aws_dms_replication_instance alone doesn't exist as a separate data structure that you can query dynamically, because Terraform needs to be able to determine exactly which resources a particular expression depends on in order to produce the dependency graph, before evaluating any expressions.
However, you can construct your own mapping data structure which incorporates the set of resources you are interested in:
locals {
source_endpoints = {
"foobar_1" = aws_dms_endpoint.foobar_source_1
"foobar_2" = aws_dms_endpoint.foobar_source_2
}
replication_instances = {
"foobar_1" = aws_dms_replication_instance.foobar_instance_1
"foobar_2" = aws_dms_replication_instance.foobar_instance_2
}
}
You can then use local.source_endpoints and local.replication_instances as mappings to look up the keys specified by your module's caller:
resource "aws_dms_replication_task" "for_each_task_1" {
for_each = var.task_map
replication_instance_arn = local.replication_instances[each.value.repl_instance].replication_instance_arn
source_endpoint_arn = local.source_endpoints[each.value.source_ep].endpoint_arn
# ...
}
This can work because a local value is also an object that participates in the dependency graph. Terraform can see that local.source_endpoints depends on both aws_dms_endpoint.foobar_source_1 and aws_dms_endpoint.foobar_source_2, and so therefore indirectly anything which depends on local.source_endpoints must effectively depend on those resources.
Although it's not crucial to your question, I want to note that this design means that your module will declare the full set of aws_dms_endpoint and aws_dms_replication_instance objects, even if some of them don't have any var.task_map elements referring to them.
I wouldn't worry about that if the only practical use of the module involves referring to all of them, but if that isn't true then a variant of this design is to allow the caller to also specify via input variables which endpoints and replication instances they need, and use for_each on all three of these resources.
If you do that then you can avoid constructing the intermediate data structure, because for_each resources are already naturally maps which support the same sort of dynamic lookup:
resource "aws_dms_replication_instance" "example" {
for_each = var.replication_instances
# ...
}
resource "aws_dms_endpoint" "example" {
for_each = var.source_endpoints
# ...
}
resource "aws_dms_replication_task" "for_each_task_1" {
for_each = var.task_map
replication_instance_arn = aws_dms_replication_instance.example[each.value.repl_instance].replication_instance_arn
source_endpoint_arn = aws_dms_endpoint.example[each.value.source_ep].endpoint_arn
# ...
}
In this variant, the caller of the module controls both which replication instances and endpoints exist and which of those each of the tasks use, so the burden for the user of your module is greater but they also get the flexibility of not declaring objects they won't actually use, in case these objects have a significant cost.

Terraform, How to output multiple private ips

I use "terraform-aws-modules/ec2-instance/aws" to provision two ec2 instances. How to output these two private IPs?
module "ec2_jekins_agent" {
source = "terraform-aws-modules/ec2-instance/aws"
version = "4.1.4"
for_each = toset(["one", "two"])
name = "jenkins-agent-${each.key}"
vpc_security_group_ids = [aws_security_group.sg_jenkins_agent.id]
subnet_id = data.terraform_remote_state.vpc.outputs.private_subnets[0]
}
The code below does not work.
output "jenkins_agent_private_ips" {
value = [module.ec2_jekins_agent.private_ip]
}
Error: Unsupported attribute
module.ec2_jekins_agent is object with 2 attributes
This object does not have an attribute named "private_ip".
Since you've used for_each, you can use values:
output "jenkins_agent_private_ips" {
value = values(module.ec2_jekins_agent)[*].private_ip
}

For loop in terraform

I am trying to get the value of function name from local.tf but I am not able to get it. I have terraform,tfvars in which I am giving the function name then it is passed to variable.tf. From varibale.tf I pass it to local.tf then to main.tf. I am not able to get the function name in main.tf. Any help would be appreciated.
terraform.tfvars
config = {
s3= {
//s3 configurations
}
s3_notifications = {
function_name = "test-lambda-mary"
}
}
variable.tf
variable "config" {
type = any
description = "S3 configuration block"
}
local.tf
function_name = {
for k, v in var.config :
k => lookup(v, "function_name", "")
}
module "all_notifications" {
source = "terraform-aws-modules/s3-bucket/aws//modules/notification"
for_each = var.config
bucket = module.s3_bucket[each.key].this_s3_bucket_id
lambda_notifications = {
lambda = {
function_name = local.function_name[each.key]
function_arn = "arn:aws:lambda:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:function:${local.function_name[each.key]}"
events = ["s3:ObjectCreated:*"]
}
}
}
error
"function_name" doesn't comply with restrictions ("^(arn:[\\w-]+:lambda:)?([a-z]{2}-(?:[a-z]+-){1,2}\\d{1}:)?(\\d{12}:)?(function:)?([a-zA-Z0-9-_]+)(:(\\$LATEST|[a-zA-Z0-9-_]+))?$"): ""
│
│ with module.all_notifications["s3"].aws_lambda_permission.allow["lambda"],
│ on .terraform/modules/all_notifications/modules/notification/main.tf line 63, in resource "aws_lambda_permission" "allow":
│ 63: function_name = each.value.function_name
If I put the function_name inside s3 braces it works absolutely fine but I need to have the fucntion name in s3_notification
That looks like a great hint. You're iterating over var.config which has 2 keys and only 1 of them has function_name defined. So when module is requested with s3 as a key, the function_value for that key will be empty string and AWS will fail the request as expected.
You can filter for_each = var.config to exclude such case, something like:
for_each = { for k, v in var.config: k => v if local.function_name[each.key] != ""}
Little nitpick: seems like the source of the module could be incorrectly written. Instead of terraform-aws-modules/s3-bucket/aws//modules/notification potentially it should be terraform-aws-modules/terraform-aws-s3-bucket//modules/notification. See https://github.com/terraform-aws-modules/terraform-aws-s3-bucket
A stab in the dark as I haven't used that module before.
But by looking at your error message:
"function_name" doesn't comply with restrictions ("^(arn:[\\w-]+:lambda:)?
it looks like for function_name you should pass Lambda's ARN, not name (contrary to what the variable name says).
BTW, is function_arn even a parameter here?
This error is coming since terraform module is expecting a valid and mandatory function name to create [aws_lambda_permission][1] resource using terraform-aws-modules/s3-bucket/aws//modules/notification module.
In your case you are looping the module on var.config which consist s3 and s3_notifications part and while first iteration its getting function name as null therefore its throwing this error.
"function_name" doesn't comply with restrictions ("^(arn:[\\w-]+:lambda:)?([a-z]{2}-(?:[a-z]+-){1,2}\\d{1}:)?(\\d{12}:)?(function:)?([a-zA-Z0-9-_]+)(:(\\$LATEST|[a-zA-Z0-9-_]+))?$"): ""
Better split the variable specific to s3_notification as list and iterate the module as below using count.
variable "s3_config" {
type = any
default = {
s3 = {
}
}
}
variable "s3_notify_lambda_func" {
type = list
default = ["test-lambda-mary","test"]. #N Numbers of functions
}
data "aws_region" "current" {}
data "aws_caller_identity" "current" {}
module "all_notifications" {
source = "terraform-aws-modules/s3-bucket/aws//modules/notification"
count = length(var.s3_notify_lambda_func) > 0 ? length(var.s3_notify_lambda_func) : 0
bucket = module.s3_bucket[count.index].this_s3_bucket_id
lambda_notifications = {
lambda = {
function_name = var.s3_notify_lambda_func[count.index]
function_arn = "arn:aws:lambda:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:function:${var.s3_notify_lambda_func[count.index]}"
events = ["s3:ObjectCreated:*"]
}
}
}

Terraform does not work input and environment variables

I have a glue job in aws.
I made a loop.
In variables.tf
variable "list_of_jobs" {
type = list(string)
default = ["myjob1","myjob2","myjob3"]
}
In glue.tf
resource "aws_glue_job" "this" {
for_each = toset(var.list_of_jobs)
name = each.value
role_arn = var.role_arn
command {
name = "pythonshell"
python_version = 3
script_location = "s3://mybucket/${each.value}/run.py"
}
}
In main.tf
variable "region" {}
variable "list_of_jobs" {}
module "my_glue" {
source = "../terraform-glue"
region = var.region
list_of_jobs = var.list_of_jobs
}
This loop works fine, and I have 3 glue jobs after execution of terraform apply.
The problem, when I am trying to make:
export TF_VAR_list_of_jobs='["myjob1","myjob2","myjob3"]'
In this case, when I am making terraform apply, I am receiving this:
Error: Invalid function argument
on ../terraform-glue/glue.tf line 2, in resource "aws_glue_job" "this":
2: for_each = toset(var.list_of_jobs)
|----------------
| var.list_of_jobsis "[\"myjob1\",\"myjob2\",\"myjob3\"]"
Invalid value for "v" parameter: cannot convert string to set of any single
type.
Input variables, does not work too. Only variable from variables.tf. Could You help me please ? I am trying to resolve this during all night.
It does not work because you need to provide type constrain for your complex variable if you want to pass it though env variables:
variable "list_of_jobs" {
type = list(string)
default = ["myjob1","myjob2","myjob3"]
}

Outputs from for_each loop for each resource

I am having a hard time figuring out how to make an output for each target group resource that this code creates.
I'd like to be able to reference each one individually in other modules. It sounds like for_each stores it as a map, so my question is how would I get the arn for targetgroup1 and targetgroup2?
Terraform normally refers to outputs by resource name, so I am struggling with that in this scenario and also how to refer to these individual arns.
Would I also need to work the outputs into the for_each or could I drop it into the output.tf file?
locals {
target_groups_beta = {
targetgroup1 = {
name = "example",
path = "/",
environment = "Beta"
}
targetgroup2 = {
name = "example2",
path = "/",
environment = "Beta"
}
}
}
resource "aws_lb_target_group" "target-group" {
for_each = local.target_groups_beta
name = "example-${each.value.name}-"
port = 80
protocol = "HTTP"
vpc_id = var.vpc-id
deregistration_delay = 5
tags = {
Environment = "${each.value.environment}"
}
health_check{
healthy_threshold = 2
unhealthy_threshold = 2
timeout = 10
interval = 15
path = each.value.path
}
}
I receive the following error when trying to do it in the output.tf file without a key value, but when I input one such as value = "${aws_lb_target_group.target-group[0].arn}" it says it's invalid. Error without key value below:
Error: Missing resource instance key
on modules\targetgroups\output.tf line 2, in output "tg_example_beta":
2: value = "${aws_lb_target_group.target-group.arn}"
Because aws_lb_target_group.target-group has "for_each" set, its attributes
must be accessed on specific instances.
For example, to correlate with indices of a referring resource, use:
aws_lb_target_group.target-group[each.key]
The aws_lb_target_group.target-group generated will be a map, with key values of targetgroup2 and targetgroup1.
Therefore, to get the individual target group details you can do:
output "target-group1-arn" {
value = aws_lb_target_group.target-group["targetgroup1"].arn
}
To return both as a map:
output "target-groups-arn-alternatice" {
value = {for k, v in aws_lb_target_group.target-group: k => v.arn}
}
target-groups-arn-alternatice = {
"targetgroup1" = "arn:aws:elasticloadbalancing:us-east-1:xxxx:targetgroup/example-example/285b26e15221b113"
"targetgroup2" = "arn:aws:elasticloadbalancing:us-east-1:xxxx:targetgroup/example-example2/075bd58359e4c4b2"
}
To return both as a list (order will be same as for keys function):
output "target-groups-arn" {
value = values(aws_lb_target_group.target-group)[*].arn
}
target-groups-arn = [
"arn:aws:elasticloadbalancing:us-east-1:xxxx:targetgroup/example-example/285b26e15221b113",
"arn:aws:elasticloadbalancing:us-east-1:xxxx:targetgroup/example-example2/075bd58359e4c4b2",
]