I've got a terraform plan that creates a number of resources in a for_each loop, and I need another resource to depend_on those first ones. How can I do it without having to explicitly list them?
Here's the first resource (AWS API Gateway resource):
locals {
apps = toset(["app1", "app2", "app3"])
}
resource "aws_api_gateway_integration" "lambda" {
for_each = local.apps
rest_api_id = aws_api_gateway_rest_api.primary.id
resource_id = aws_api_gateway_resource.lambda[each.key].id
http_method = aws_api_gateway_method.lambda_post[each.key].http_method
integration_http_method = "POST"
type = "AWS_PROXY"
uri = aws_lambda_function.lambda[each.key].invoke_arn
}
Now I need to wait for all the 3 apps integrations before creating an API Gateway deployment:
resource "aws_api_gateway_deployment" "primary" {
rest_api_id = aws_api_gateway_rest_api.primary.id
depends_on = [
aws_api_gateway_integration.lambda["app1"],
aws_api_gateway_integration.lambda["app2"],
aws_api_gateway_integration.lambda["app3"],
]
However the list of apps keeps growing and I don't want to maintain it manually over here. Everywhere else I can simply use for or for_each together with local.apps but I can't figure out how to dynamically build the list for depends_on? Any ideas?
You have to do it manually. TF docs clearly explain that depends_on must be explicitly defined:
You only need to explicitly specify a dependency
Dependencies in Terraform are always between static blocks (resource, data, and module blocks mainly) and not between individual instances of those objects.
Therefore listing individual instances like you did in your example is redundant:
depends_on = [
aws_api_gateway_integration.lambda["app1"],
aws_api_gateway_integration.lambda["app2"],
aws_api_gateway_integration.lambda["app3"],
]
The above is exactly equivalent to declaring a dependency on the resource as a whole:
depends_on = [
aws_api_gateway_integration.lambda,
]
The for_each argument itself typically has its own dependencies (e.g. local.apps in your example) and so Terraform needs to construct the dependency graph before evaluating for_each. This means that there is only one node in the dependency graph representing the entire resource (including its for_each expression), and the individual instances are not represented in the plan-time dependency graph at all.
There is another way:
Create a module with for_each approach called aws_api_gateway_integration
Modify module to support all parameters needed
Then you can just call new objects to create - example
At the end you can put depends_on on a module - depends_on = [
module.logic_app
]
Related
I want to create multiple aws_iam_policy_document resources with for_each, to be later assumed by several roles, as follows:
# Policy to allow services to assume the role
data "aws_iam_policy_document" "this" {
for_each = var.lambda_configuration
statement {
actions = ["sts:AssumeRole"]
principals {
type = "Service"
identifiers = [
"lambda.amazonaws.com",
"apigateway.amazonaws.com",
]
}
}
}
# IAM role for executing the Lambda function
resource "aws_iam_role" "this" {
for_each = var.lambda_configuration
name = "my_lambda_${each.key}_Executor_Role"
description = "Role for executing my_lambda-${each.key} function"
assume_role_policy = data.aws_iam_policy_document.assume_role_policy_[each.key].json
}
How should I interpolate this
assume_role_policy = data.aws_iam_policy_document.assume_role_policy_[each.key].json
to make the correct matching with the roles?
The correct syntax:
data.aws_iam_policy_document.this[each.key].json
Note that this is not interpolation, as you mentioned in your question. It is just a value lookup. And I have no idea where you got assume_role_policy_ from, but that is not valid HCL syntax.
This seems like a good situation for Chaining for_each Between Resources, which (subjectively) tends to lead to a configuration where it's easier for an unfamiliar reader to follow the way data is flowing between the resources.
data "aws_iam_policy_document" "this" {
for_each = var.lambda_configuration
statement {
actions = ["sts:AssumeRole"]
principals {
type = "Service"
identifiers = [
"lambda.amazonaws.com",
"apigateway.amazonaws.com",
]
}
}
}
# IAM role for executing the Lambda function
resource "aws_iam_role" "this" {
for_each = data.aws_iam_policy_document.this
name = "my_lambda_${each.key}_Executor_Role"
description = "Role for executing my_lambda-${each.key} function"
assume_role_policy = each.value.json
}
Using the first resource itself as the for_each for the second one means that the two will always have exactly the same keys (because a resource with for_each appears in expressions as a map of objects using the keys from the for_each map), and that each.value inside the second block refers to the corresponding instance of the first block.
This has essentially the same behavior as using the index operator to select an element from data.aws_iam_policy_document.this using each.key, but makes the relationship between these two resources a little more direct and so (again, subjectively) easier to read and maintain in future.
I created tf file that takes input from cli and then use this as name for aws lambda, and api gateway.
Currently inputing different name just replace name in currently working one.
My goal is that every time i input new name new lambda and gateway should be created. Is it possible?
variable "repo_name" {
type = string
}
resource "aws_lambda_function" "lambda" {
function_name = var.repo_name
handler = "lambda_function.lambda_handler"
runtime = "python3.9"
role = ""
filename = "python.zip"
}
This can be done in a couple of different ways, but the easiest one would be creating a local variable and just add additional names to it whenever you need a new function and API Gateway. That would look something like this:
locals {
repo_names = ["one", "two", "three"]
}
Now, the second part can be solved with count [1] or for_each [2] meta-arguments. It's usually a matter of preference, but I would suggest using for_each:
resource "aws_lambda_function" "lambda" {
for_each = toset(local.repo_names)
function_name = each.value
handler = "lambda_function.lambda_handler"
runtime = "python3.9"
role = ""
filename = "python.zip"
}
You would then follow a similar approach for creating different API Gateway resources. Note that for_each has to be used with sets or maps, hence the toset built-in function [3] usage. Also, make sure you understand how each object works [4]. In case of a set, the each.key and each.value are the same which is not the case when using maps.
[1] https://developer.hashicorp.com/terraform/language/meta-arguments/count
[2] https://developer.hashicorp.com/terraform/language/meta-arguments/for_each
[3] https://developer.hashicorp.com/terraform/language/functions/toset
[4] https://developer.hashicorp.com/terraform/language/meta-arguments/for_each#the-each-object
I am getting Cycle errors when adding dependencies on the previous count index
I want to define an API path /test1/{id} using AWS Terraform with the id resource depending on the test1. If the resource is test1, the set the parent as the root API gateway resource.
locals {
resources = ["test1", "{id}"]
}
resource "aws_api_gateway_rest_api" "root_api" {
name = "dev-api"
}
resource "aws_api_gateway_resource" "dev_gateway_test_resources" {
count = length(local.resources)
path_part = local.resources[count.index]
parent_id = count.index == 0 ? aws_api_gateway_rest_api.root_api.root_resource_id : aws_api_gateway_resource.dev_gateway_test_resources[count.index - 1].id
rest_api_id = aws_api_gateway_rest_api.root_api.id
}
The response:
Error: Cycle: aws_api_gateway_resource.dev_gateway_test_resources[1], aws_api_gateway_resource.dev_gateway_test_resources[0]
I have tried the same logic using for_each however I still see the same error
What you have there is a resource which is referencing itself. As of today, this is not supported in Terraform, even if it can be easily seen that the resource does not actually has a reference to itself, but to another resource of the same type created in a loop. There is an issue for supporting self referencing: GitHub
Now, I don't know how many item can have your resources list, but in this case you would probably want to duplicate your aws_api_gateway_resource:
resource "aws_api_gateway_resource" "dev_gateway_test_root" {
path_part = "test"
parent_id = aws_api_gateway_rest_api.root_api.root_resource_id
rest_api_id = aws_api_gateway_rest_api.root_api.id
}
resource "aws_api_gateway_resource" "dev_gateway_test_resources" {
path_part = "{id}"
parent_id = aws_api_gateway_resource.dev_gateway_test_root.id
rest_api_id = aws_api_gateway_rest_api.root_api.id
}
What you want to achieve is to have an endpoint such as /test/{id}. From my experience API gateway endpoints tend to grow in width, meaning that you will have something like:
/
/test1
/{id}
/test2
/{id}
/test3
...
/testN
Instead of having a really lengthy endpoint, like /test/child1/child2/.../{id}.
You can have a loop for each level in the tree, they wont have self-referencing. You can not really have a loop between different levels of the tree.
I'm having a complex set of complex objects, e.g. DynamoDB tables received via data
data "aws_dynamodb_table" "table" {
for_each = toset([
"table-1",
"table-2",
"table-3",
])
name = each.value
}
I want to use some of the table properties in different places, e.g. creating event source mappings to lambda functions, which is quite easy:
resource "aws_lambda_event_source_mapping" "mapping" {
for_each = data.aws_dynamodb_table.table
batch_size = 100
event_source_arn = each.value.stream_arn
function_name = // some function ARN
starting_position = "LATEST"
}
Now where I'm stuck: how do it just get a list of some specific field like the table ARN, so that I can use that directly in a IAM policy for the resources?
statement {
actions = [
"dynamodb:PutItem",
"dynamodb:UpdateItem",
"dynamodb:Query",
"dynamodb:Scan"
]
# that's not how it works
resources = tolist(data.aws_dynamodb_table.table.*.arn)
effect = "Allow"
}
I'm sure the terraform documentation gives me an answer on that, but I'm not able to find it.
Since your data.aws_dynamodb_table.table is a map due to the use of for_each, you have to use values to get all the values for the arn:
resources = values(data.aws_dynamodb_table.table)[*].arn
I have a template file that I use to on a list:
variable "users" {
type = "list"
default = [
"blackwidow",
"hulk",
"marvel",
]
}
// This will loop through the users list above and render out code for
// each item in the list.
data "template_file" "init" {
template = file("user_template.tpl")
count = length(var.users)
vars = {
username = var.users[count.index]
bucketid = aws_s3_bucket.myFTP_Bucket.id
}
}
The template file has multiple aws resources like
- "aws_transfer_user"
- "aws_s3_bucket_object"
- "aws_transfer_ssh_key"
etc... In fact it can have more stuff than just that. It also has some terraform variables in there too.
This data template works great in rendering out the contents of the template file, substituting in the names of my users.
But that's all terraform does.
Terraform doesn't instantiate the rendered content of the template file. It just merely keeps it as a string and keeps it in memory. Kind of like the C preprocessor doing substitution, but not 'including' the file.
Kind of frustrating. I'd like Terraform to instantiate the contents of my rendered template file. How do I do this?
The template_file data source (along with the templatefile function that has replaced it for Terraform 0.12) are for string templating, not for modular Terraform configuration.
To produce a set of different resource instances per item in a collection, we use resource for_each:
variable "users" {
type = set(string)
default = [
"blackwidow",
"hulk",
"marvel",
]
}
resource "aws_transfer_user" "example" {
for_each = var.users
# ...
}
resource "aws_transfer_user" "example" {
for_each = var.users
# ...
}
resource "aws_s3_bucket_object" "example" {
for_each = var.users
# ...
}
resource "aws_transfer_ssh_key" "example" {
for_each = aws_transfer_user.example
# ...
}
Inside each of those resource blocks you can use each.key to refer to each one of the usernames. Inside the resource "aws_transfer_ssh_key" "example" block, because I used aws_transfer_user.example as the repetition expression, you can also use each.value to access the attributes of the corresponding aws_transfer_user object. That for_each expression also serves to tell Terraform that aws_transfer_ssh_key.example depends on aws_transfer_user.example.