I am trying to get the value of function name from local.tf but I am not able to get it. I have terraform,tfvars in which I am giving the function name then it is passed to variable.tf. From varibale.tf I pass it to local.tf then to main.tf. I am not able to get the function name in main.tf. Any help would be appreciated.
terraform.tfvars
config = {
s3= {
//s3 configurations
}
s3_notifications = {
function_name = "test-lambda-mary"
}
}
variable.tf
variable "config" {
type = any
description = "S3 configuration block"
}
local.tf
function_name = {
for k, v in var.config :
k => lookup(v, "function_name", "")
}
module "all_notifications" {
source = "terraform-aws-modules/s3-bucket/aws//modules/notification"
for_each = var.config
bucket = module.s3_bucket[each.key].this_s3_bucket_id
lambda_notifications = {
lambda = {
function_name = local.function_name[each.key]
function_arn = "arn:aws:lambda:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:function:${local.function_name[each.key]}"
events = ["s3:ObjectCreated:*"]
}
}
}
error
"function_name" doesn't comply with restrictions ("^(arn:[\\w-]+:lambda:)?([a-z]{2}-(?:[a-z]+-){1,2}\\d{1}:)?(\\d{12}:)?(function:)?([a-zA-Z0-9-_]+)(:(\\$LATEST|[a-zA-Z0-9-_]+))?$"): ""
│
│ with module.all_notifications["s3"].aws_lambda_permission.allow["lambda"],
│ on .terraform/modules/all_notifications/modules/notification/main.tf line 63, in resource "aws_lambda_permission" "allow":
│ 63: function_name = each.value.function_name
If I put the function_name inside s3 braces it works absolutely fine but I need to have the fucntion name in s3_notification
That looks like a great hint. You're iterating over var.config which has 2 keys and only 1 of them has function_name defined. So when module is requested with s3 as a key, the function_value for that key will be empty string and AWS will fail the request as expected.
You can filter for_each = var.config to exclude such case, something like:
for_each = { for k, v in var.config: k => v if local.function_name[each.key] != ""}
Little nitpick: seems like the source of the module could be incorrectly written. Instead of terraform-aws-modules/s3-bucket/aws//modules/notification potentially it should be terraform-aws-modules/terraform-aws-s3-bucket//modules/notification. See https://github.com/terraform-aws-modules/terraform-aws-s3-bucket
A stab in the dark as I haven't used that module before.
But by looking at your error message:
"function_name" doesn't comply with restrictions ("^(arn:[\\w-]+:lambda:)?
it looks like for function_name you should pass Lambda's ARN, not name (contrary to what the variable name says).
BTW, is function_arn even a parameter here?
This error is coming since terraform module is expecting a valid and mandatory function name to create [aws_lambda_permission][1] resource using terraform-aws-modules/s3-bucket/aws//modules/notification module.
In your case you are looping the module on var.config which consist s3 and s3_notifications part and while first iteration its getting function name as null therefore its throwing this error.
"function_name" doesn't comply with restrictions ("^(arn:[\\w-]+:lambda:)?([a-z]{2}-(?:[a-z]+-){1,2}\\d{1}:)?(\\d{12}:)?(function:)?([a-zA-Z0-9-_]+)(:(\\$LATEST|[a-zA-Z0-9-_]+))?$"): ""
Better split the variable specific to s3_notification as list and iterate the module as below using count.
variable "s3_config" {
type = any
default = {
s3 = {
}
}
}
variable "s3_notify_lambda_func" {
type = list
default = ["test-lambda-mary","test"]. #N Numbers of functions
}
data "aws_region" "current" {}
data "aws_caller_identity" "current" {}
module "all_notifications" {
source = "terraform-aws-modules/s3-bucket/aws//modules/notification"
count = length(var.s3_notify_lambda_func) > 0 ? length(var.s3_notify_lambda_func) : 0
bucket = module.s3_bucket[count.index].this_s3_bucket_id
lambda_notifications = {
lambda = {
function_name = var.s3_notify_lambda_func[count.index]
function_arn = "arn:aws:lambda:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:function:${var.s3_notify_lambda_func[count.index]}"
events = ["s3:ObjectCreated:*"]
}
}
}
Related
I create few resources in the terraform -
resource "aws_dms_replication_instance" "foobar_instance_1" {
}
resource "aws_dms_replication_instance" "foobar_instance_2" {
}
resource "aws_dms_endpoint" "foobar_source_1" {
}
resource "aws_dms_endpoint" "foobar_source_2" {
}
Then define a replication task that's dependent on above two resources -
resource "aws_dms_replication_task" "foobar_task_1" {
replication_instance_arn = aws_dms_replication_instance.foobar_instance_1.replication_instance_arn
source_endpoint_arn = aws_dms_endpoint.foobar_source_1.endpoint_arn
.
.
.
}
I want to use a tf variable to automate a similar aws_dms_replication_task creation using for_each. I create a Map variable in variables.tf that
looks like this -
variable "task_map" {
type = map(object({
source_endpoint = string
repl_instance = string
}))
default = {
"dms_attr_map" = {
source_ep = "foobar_source_1"
repl_instance = "foobar_instance_1"
},
"dms_attr_map2" = {
source_ep = "foobar_source_2"
repl_instance = "foobar_instance_2"
}
}
}
Now, I go on to create a resource block to loop over task_map and create replication_task
resource "aws_dms_replication_task" "for_each_task_1" {
for_each = var.task_map
replication_instance_arn = aws_dms_replication_instance.${each.value["repl_instance"]}.replication_instance_arn
source_endpoint_arn = aws_dms_endpoint.${each.value["source_ep"]}.endpoint_arn
.
.
.
}
As I execute my plan, terrraform plan throws an error that referring to replication_instance using ${each.value["repl_instance"]} is wrong.
Error message -
│ Error: Invalid character │ │ On ../modules/cdc/main.tf line 236:
This character is not used within the │ language. ╵
╷ │ Error: Invalid attribute name │ │ On ../modules/cdc/main.tf line
233: An attribute name is required after a │ dot. ╵
Error messages point to the specific line where I use for_each to refer to replication_instance and source_endpoint
aws_dms_endpoint.${each.value["source_ep"]}.endpoint_arn
aws_dms_replication_instance.${each.value["repl_instance"]}.replication_instance_arn
How do I refer to the resources created using their names using a for_each.
Thank you.
How do I refer to the resources created using their names using a for_each.
You can't do this. In other words, you can't create dynamic references to resources in the form of:
aws_dms_endpoint.${each.value["source_ep"]}.endpoint_arn
Instead you have to use maps or lists to create your aws_dms_replication_instance and aws_dms_endpoint. For example:
resource "aws_dms_replication_instance" "foobar_instance" {
for_each = toset(["foobar_instance_1", "foobar_instance_2"])
}
resource "aws_dms_endpoint" "foobar_source" {
for_each = toset(["foobar_source_1", "foobar_source_2"])
}
end then refer to them as follows:
resource "aws_dms_replication_task" "for_each_task_1" {
for_each = var.task_map
replication_instance_arn = aws_dms_replication_instance.foobar_instance[each.value["repl_instance"]].replication_instance_arn
source_endpoint_arn = aws_dms_endpoint.foobar_source[each.value["source_ep"]].endpoint_arn
.
.
.
}
When thinking about problems like this it's important to recognize that a reference expression like aws_dms_replication_instance.foobar_instance_1 is an indivisible unit: aws_dms_replication_instance alone doesn't exist as a separate data structure that you can query dynamically, because Terraform needs to be able to determine exactly which resources a particular expression depends on in order to produce the dependency graph, before evaluating any expressions.
However, you can construct your own mapping data structure which incorporates the set of resources you are interested in:
locals {
source_endpoints = {
"foobar_1" = aws_dms_endpoint.foobar_source_1
"foobar_2" = aws_dms_endpoint.foobar_source_2
}
replication_instances = {
"foobar_1" = aws_dms_replication_instance.foobar_instance_1
"foobar_2" = aws_dms_replication_instance.foobar_instance_2
}
}
You can then use local.source_endpoints and local.replication_instances as mappings to look up the keys specified by your module's caller:
resource "aws_dms_replication_task" "for_each_task_1" {
for_each = var.task_map
replication_instance_arn = local.replication_instances[each.value.repl_instance].replication_instance_arn
source_endpoint_arn = local.source_endpoints[each.value.source_ep].endpoint_arn
# ...
}
This can work because a local value is also an object that participates in the dependency graph. Terraform can see that local.source_endpoints depends on both aws_dms_endpoint.foobar_source_1 and aws_dms_endpoint.foobar_source_2, and so therefore indirectly anything which depends on local.source_endpoints must effectively depend on those resources.
Although it's not crucial to your question, I want to note that this design means that your module will declare the full set of aws_dms_endpoint and aws_dms_replication_instance objects, even if some of them don't have any var.task_map elements referring to them.
I wouldn't worry about that if the only practical use of the module involves referring to all of them, but if that isn't true then a variant of this design is to allow the caller to also specify via input variables which endpoints and replication instances they need, and use for_each on all three of these resources.
If you do that then you can avoid constructing the intermediate data structure, because for_each resources are already naturally maps which support the same sort of dynamic lookup:
resource "aws_dms_replication_instance" "example" {
for_each = var.replication_instances
# ...
}
resource "aws_dms_endpoint" "example" {
for_each = var.source_endpoints
# ...
}
resource "aws_dms_replication_task" "for_each_task_1" {
for_each = var.task_map
replication_instance_arn = aws_dms_replication_instance.example[each.value.repl_instance].replication_instance_arn
source_endpoint_arn = aws_dms_endpoint.example[each.value.source_ep].endpoint_arn
# ...
}
In this variant, the caller of the module controls both which replication instances and endpoints exist and which of those each of the tasks use, so the burden for the user of your module is greater but they also get the flexibility of not declaring objects they won't actually use, in case these objects have a significant cost.
I have a list of accounts declared under variable "acct". I need to create as many ARNs as the accounts below using this list. How to do it? The below code gives me attribute error:
A reference to a resource type must be followed by at least one attribute access, specifying the resource name.
variable "acct"{
type = list(string)
default = ["111111111111","22222222222",.....]
}
data "aws_arn" "SrcArn" {
account = [for acc in var.acct : acc ]
arn = ["arn:aws:logs:us-east-1:${account}:*"]
}
I need to create a list of arns which then can be used further down in the code. Can these then be referenced below like this:
condition {
test = "ArnLike"
values = data.aws_arn.SrcArn
variable = "aws:SourceArn"
}
If you want to use aws_arn, you can use to this.
variable "acct"{
type = list(string)
default = ["111111111111","22222222222",.....]
}
data "aws_arn" "SrcArn" {
count = var.acct
arn = ["arn:aws:logs:us-east-1:${var.acct[count.index]}:*"]
}
And if you wanna make arn_list,
locals {
# only use to variable.
# not use to data "aws_arn"
arn_list_only_use_to_variable = [for account in var.acct: "arn:aws:logs:us-east-1:${account}:*"]
# use to data "aws_arn"
src_arn = data.aws_arn.SrcArn
arn_list_2_use_to_data_aws_arn = [for account in src_arn: account]
}
...
condition {
test = "ArnLike"
values = local.arn_list_only_use_to_variable
variable = "aws:SourceArn"
}
...
I have a glue job in aws.
I made a loop.
In variables.tf
variable "list_of_jobs" {
type = list(string)
default = ["myjob1","myjob2","myjob3"]
}
In glue.tf
resource "aws_glue_job" "this" {
for_each = toset(var.list_of_jobs)
name = each.value
role_arn = var.role_arn
command {
name = "pythonshell"
python_version = 3
script_location = "s3://mybucket/${each.value}/run.py"
}
}
In main.tf
variable "region" {}
variable "list_of_jobs" {}
module "my_glue" {
source = "../terraform-glue"
region = var.region
list_of_jobs = var.list_of_jobs
}
This loop works fine, and I have 3 glue jobs after execution of terraform apply.
The problem, when I am trying to make:
export TF_VAR_list_of_jobs='["myjob1","myjob2","myjob3"]'
In this case, when I am making terraform apply, I am receiving this:
Error: Invalid function argument
on ../terraform-glue/glue.tf line 2, in resource "aws_glue_job" "this":
2: for_each = toset(var.list_of_jobs)
|----------------
| var.list_of_jobsis "[\"myjob1\",\"myjob2\",\"myjob3\"]"
Invalid value for "v" parameter: cannot convert string to set of any single
type.
Input variables, does not work too. Only variable from variables.tf. Could You help me please ? I am trying to resolve this during all night.
It does not work because you need to provide type constrain for your complex variable if you want to pass it though env variables:
variable "list_of_jobs" {
type = list(string)
default = ["myjob1","myjob2","myjob3"]
}
I am having a hard time figuring out how to make an output for each target group resource that this code creates.
I'd like to be able to reference each one individually in other modules. It sounds like for_each stores it as a map, so my question is how would I get the arn for targetgroup1 and targetgroup2?
Terraform normally refers to outputs by resource name, so I am struggling with that in this scenario and also how to refer to these individual arns.
Would I also need to work the outputs into the for_each or could I drop it into the output.tf file?
locals {
target_groups_beta = {
targetgroup1 = {
name = "example",
path = "/",
environment = "Beta"
}
targetgroup2 = {
name = "example2",
path = "/",
environment = "Beta"
}
}
}
resource "aws_lb_target_group" "target-group" {
for_each = local.target_groups_beta
name = "example-${each.value.name}-"
port = 80
protocol = "HTTP"
vpc_id = var.vpc-id
deregistration_delay = 5
tags = {
Environment = "${each.value.environment}"
}
health_check{
healthy_threshold = 2
unhealthy_threshold = 2
timeout = 10
interval = 15
path = each.value.path
}
}
I receive the following error when trying to do it in the output.tf file without a key value, but when I input one such as value = "${aws_lb_target_group.target-group[0].arn}" it says it's invalid. Error without key value below:
Error: Missing resource instance key
on modules\targetgroups\output.tf line 2, in output "tg_example_beta":
2: value = "${aws_lb_target_group.target-group.arn}"
Because aws_lb_target_group.target-group has "for_each" set, its attributes
must be accessed on specific instances.
For example, to correlate with indices of a referring resource, use:
aws_lb_target_group.target-group[each.key]
The aws_lb_target_group.target-group generated will be a map, with key values of targetgroup2 and targetgroup1.
Therefore, to get the individual target group details you can do:
output "target-group1-arn" {
value = aws_lb_target_group.target-group["targetgroup1"].arn
}
To return both as a map:
output "target-groups-arn-alternatice" {
value = {for k, v in aws_lb_target_group.target-group: k => v.arn}
}
target-groups-arn-alternatice = {
"targetgroup1" = "arn:aws:elasticloadbalancing:us-east-1:xxxx:targetgroup/example-example/285b26e15221b113"
"targetgroup2" = "arn:aws:elasticloadbalancing:us-east-1:xxxx:targetgroup/example-example2/075bd58359e4c4b2"
}
To return both as a list (order will be same as for keys function):
output "target-groups-arn" {
value = values(aws_lb_target_group.target-group)[*].arn
}
target-groups-arn = [
"arn:aws:elasticloadbalancing:us-east-1:xxxx:targetgroup/example-example/285b26e15221b113",
"arn:aws:elasticloadbalancing:us-east-1:xxxx:targetgroup/example-example2/075bd58359e4c4b2",
]
I have a bunch of aws_ecr_repositories defined in my Terraform code:
resource "aws_ecr_repository" "nginx_images" {
name = "nginx-test"
}
resource "aws_ecr_repository" "oracle_images" {
name = "oracle-test"
}
I want to be able to have an output that can list all the aws_ecr_repository resources into one output. This is what I tried:
output "ecr_repository_urls" {
value = "[${aws_ecr_repository.*.repository_url}]"
}
This does not work because Terraform does not seem to allow wildcards on the resource names. Is it possible to have an output like this? My current solution is to just list outputs for every resource defined.
Terraform's splat syntax is for keeping track of each thing created by a resource using the count meta parameter.
If you want to be able to get at all of the respoitory URLs you could have a single aws_ecr_repository resource and use the count meta parameter with something like this:
variable "images" {
default = [
"nginx-test",
"oracle-test",
]
}
resource "aws_ecr_repository" "images" {
count = "${length(var.images)}"
name = "${var.images[count.index]}"
}
output "ecr_repository_urls" {
value = "[${aws_ecr_repository.images.*.repository_url}]"
}
You can combine them manually as a list:
output "ecr_repository_urls" {
value = ["${aws_ecr_repository.nginx_images.repository_url}", "${aws_ecr_repository.oracle_images.repository_url}"]
}
Although it probably won't be pretty in code.
You could also do something like this:
variable "ecr_repos" {
default = {
"0" = "foo"
"1" = "bar"
}
}
resource "aws_ecr_repository" "images" {
count = "${length(var.ecr_repos)}"
name = "${lookup(var.ecr_repos,count.index)}-test"
}
output "ecr_repository_urls" {
value = "${aws_ecr_repository.images.*.repository_url}"
}
But the problem is if the list order changes it's going to recreate resources and get really ugly really fast since each repo is assigned to an index number.