I am creating an AWS http APIgateway with JWT authorisation from cognito using Terraform.
I have built this out, using one app client and it works.
But i would like to add another app client. i call the clientid from AWS SSM.
below is the jwt_configuration extract from the module we have built.
extprovider is passed from the main resource tf as a variable.
eg extprovider=production
jwt_configuration {
audience = [ data.aws_ssm_parameter.clientid.value ]
issuer = data.aws_ssm_parameter.user_pool_endpoint.value
}
my data.tf
data aws_ssm_parameter clientid {
name = "/${var.workflow}/cognito/${var.extprovider}/clientid"
}
The new setup will have the extprovider name of production_odd and production_even, this part must be done within the module, as the name of production is used elsewhere.
i had thought my approach would be;
locals {
extprovider_new = flatten([ for e in var.extprovider : tolist(["${e}odd","${e}even"])])
}
data aws_ssm_parameter clientid1 {
name = "/${var.workflow}/cognito/${var.extprovider_new[0]}/clientid"
}
data aws_ssm_parameter clientid2 {
name = "/${var.workflow}/cognito/${var.extprovider_new[1]}/clientid"
}
jwt_configuration {
audience = [ data.aws_ssm_parameter.clientid1.value,data.aws_ssm_parameter.clientid2.value ]
issuer = data.aws_ssm_parameter.user_pool_endpoint.value
}
However, i am getting the error;
Error: Iteration over non-iterable value
on ../modules/apiresource/locals.tf line 3, in locals:
3: extprovider_new = flatten([ for e in var.extprovider : tolist(["${e}odd","${e}even"])])
├────────────────
│ var.extprovider is "foo"
A value of type string cannot be used as the collection in a 'for' expression.
Error: Iteration over non-iterable value
on ../modules/apiresource/locals.tf line 3, in locals:
3: extprovider_new = flatten([ for e in var.extprovider : tolist(["${e}odd","${e}even"])])
├────────────────
│ var.extprovider is "bar"
A value of type string cannot be used as the collection in a 'for' expression.
Variable is a string, it needs to be a list;
extprovider_new = ["${var.extprovider}odd", "${var.extprovider}even"]
}```
Related
I have a terraform project which contains a main module and a submodule. Something like this:
\modules
dev.tfvars
\app
\main
main.tf, output.tf, providers.tf, variables.tf
\sub-modules
\eventbridge
main.tf, output.tf, variables.tf
Both variables.tf files have this variable defined:
variable "secrets" {
description = "map for secret manager"
type = map(string)
}
The top level main.tf has this defined:
module "a-eventbridge-trigger" {
source = "../sub-modules/eventbridge"
secrets = var.secrets
}
The submodule main.tf has this:
resource "aws_cloudwatch_event_connection" "auth" {
name = "request-token"
description = "Gets token"
authorization_type = "OAUTH_CLIENT_CREDENTIALS"
auth_parameters {
oauth {
authorization_endpoint = "${var.apiurl}"
http_method = "POST"
oauth_http_parameters {
body {
key = "grant_type"
value = "client_credentials"
is_value_secret = true
}
body {
key = "client_id"
value = var.secrets.Client_Id
is_value_secret = true
}
body {
key = "client_secret"
value = var.secrets.Client_Secret
is_value_secret = true
}
}
}
}
}
However, when run it throws this error:
Error: error creating EventBridge connection (request-token): InvalidParameter: 2 validation error(s) found.
- missing required field, CreateConnectionInput.AuthParameters.OAuthParameters.ClientParameters.ClientID.
- missing required field, CreateConnectionInput.AuthParameters.OAuthParameters.ClientParameters.ClientSecret.
A file dump ahead of the terrform apply command successfully dumps out the contents of the tfvars file, so I know it exists at time of execution.
The top level output.tf successfully writes out the complete values of the secrets variable after execution, so I know the top level module receives the variables.
In the submodule, the resources defined after the aws_cloudwatch_event_connection block do get created and they also use variables received from the same tfvars file.
Is this a problem with how I am providing the variables or with my definition of the resources itself? (Or something else?)
client_parameters is missing on your configuration, you need to set it in auth_parameters.oauth
resource "aws_cloudwatch_event_connection" "auth" {
name = "request-token"
description = "Gets token"
authorization_type = "OAUTH_CLIENT_CREDENTIALS"
auth_parameters {
oauth {
authorization_endpoint = "${var.apiurl}"
http_method = "POST"
client_parameters {
client_id = var.secrets.Client_Id
client_secret = var.secrets.Client_Secret
}
oauth_http_parameters {
body {
key = "grant_type"
value = "client_credentials"
is_value_secret = true
}
}
}
}
}
I am trying to use Terraform's public modules and I created a lot of IAM users like so:
module "iam_finance_users" {
source = "terraform-aws-modules/iam/aws//modules/iam-user"
version = "5.9.1"
for_each = var.finance_users
name = each.value
force_destroy = true
create_iam_access_key = false
password_reset_required = true
}
My variables.tf file has the following:
variable "finance_users" {
type = set(string)
default = ["mary.johnson#domain.com","mike.smith#domain.com"....]
}
Now I am trying to add these users to a group like so
module "iam_group_finance" {
source = "terraform-aws-modules/iam/aws//modules/iam-group-with-policies"
version = "5.9.1"
name = "finance"
group_users = ["${module.iam_finance_users.iam_user_name}"]
attach_iam_self_management_policy = true
custom_group_policy_arns = [
"${module.iam_read_write_policy.arn}",
]
}
But no matter what I try I still keep getting errors. I have a list of users in a variable and I want to add all those users created in that module to the group module. I know I'm close but I can't quite seem to close this.
Since you used for_each in the module, you have to use values to access elements of all instances of the module created:
group_users = values(module.iam_finance_users)[*].iam_user_name
It's often helpful to know what the error is. In versions of Terraform v1.0 or newer, the errors become fairly explanatory. The error you were likely getting was something similar to:
│ Error: Unsupported attribute
│
│ on main.tf line 14, in output "iam_finance_users":
│ 14: value = module.iam_finance_users.iam_user_name
│ ├────────────────
│ │ module.iam_finance_users is object with 2 attributes
│
│ This object does not have an attribute named "iam_user_name".
The solutions, which both work, can be tested using the following example code:
Module 1: modules/iam-user
variable "name" {
type = string
}
output "iam_user_name" {
value = "IAM-${var.name}"
}
Module 2: modules/iam-group-with-policies
variable "group_users" {
type = set(string)
}
output "iam-users" {
value = var.group_users
}
Root module (your code):
variable "finance_users" {
type = set(string)
default = ["mary.johnson#domain.com", "mike.smith#domain.com"]
}
module "iam_finance_users" {
source = "../modules/iam-user"
for_each = var.finance_users
name = each.value
}
module "iam_group_finance" {
source = "../modules/iam-group-with-policies"
# This works...
group_users = values(module.iam_finance_users)[*].iam_user_name
# This also works...
group_users = [for o in module.iam_finance_users : o.iam_user_name]
}
output "group_users" {
value = module.iam_group_finance.iam-users
}
You can easily test each module along the way using terraform console.
E.g.:
terraform console
> module.iam_finance_users
{
"mary.johnson#domain.com" = {
"iam_user_name" = "IAM-mary.johnson#domain.com"
}
"mike.smith#domain.com" = {
"iam_user_name" = "IAM-mike.smith#domain.com"
}
}
Here you can see why it didn't work. The module doesn't spit out a list in that way, so you have to iterate through the module to get your variable. This is why both the for method as well as the values method works.
Look at how differently a single module is handled:
module "iam_finance_users" {
source = "../modules/iam-user"
name = tolist(var.finance_users)[1]
}
terraform console
> module.iam_finance_users
{
"iam_user_name" = "IAM-mike.smith#domain.com"
}
Notice that when not inside a loop (foreach), that the single iam_user_name is accessible.
You could try something like below:
This is untested, so you might need to tweak a bit. Reference thread: https://discuss.hashicorp.com/t/for-each-objects-to-list/36609/2
module "iam_group_finance" {
source = "terraform-aws-modules/iam/aws//modules/iam-group-with-policies"
version = "5.9.1"
name = "finance"
group_users = [ for o in module.iam_finance_users : o.iam_user_name ]
attach_iam_self_management_policy = true
custom_group_policy_arns = [
module.iam_read_write_policy.arn
]
}
I have a resource that creates multiple s3 access points depending on the input provided. The input is a map with s3 uri as the key and parsed bucket name as the value.
Example:
{
"s3://my_bucket/model1.tar.gz" -> "my_bucket",
"s3://my_bucket_2/model2.tar.gz" -> "my_bucket_2",
"s3://my_bucket/model3.tar.gz" -> "my_bucket"
}
I then use for_each to iterate through each element in the map to create s3 access points. Unfortunately, there are 2 "my_bucket" values in the map, which means it will attempt to create s3 access points for that designated bucket twice, and thus will error out with message:
AccessPointAlreadyOwnedByYou: Your previous request to create the named accesspoint succeeded and you already own it.
How can I check that the access point exists first before creating the resource?
Code Example:
resource "aws_s3_access_point" "s3_access_point" {
for_each = var.create ? local.uri_bucket_map : {}
bucket = each.value
name = format("%s-%s", each.value, "access-point")
}
output "s3_access_point_arn" {
description = "The arn of the access point"
value = { for uri, ap in aws_s3_access_point.s3_access_point : uri => ap.arn }
}
Desired Output:
{
"s3://my_bucket/model1.tar.gz" -> <access point uri>,
"s3://my_bucket_2/model2.tar.gz" -> <access point uri>,
"s3://my_bucket/model3.tar.gz" -> <access point uri>
}
I would invert your uri_bucket_map:
locals {
uri_bucket_map_inverse = {
for k,v in local.uri_bucket_map: v => k...
}
}
giving:
{
"my_bucket" = [
"s3://my_bucket/model1.tar.gz",
"s3://my_bucket/model3.tar.gz",
]
"my_bucket_2" = [
"s3://my_bucket_2/model2.tar.gz",
]
}
then just create access points as:
resource "aws_s3_access_point" "s3_access_point" {
for_each = var.create ? local.uri_bucket_map_inverse : {}
bucket = each.key
name = format("%s-%s", each.key, "access-point")
}
and the output would use both the APs and the inverted list map:
output "s3_access_point_arn" {
description = "The arn of the access point"
value = merge([for bucket_name, ap in aws_s3_access_point.s3_access_point:
{ for uri in local.uri_bucket_map_inverse[bucket_name]:
uri => ap.arn
}
]...)
}
I have two conditions need to be fulfilled:
Grant users permission to specific project-id based on env. For example: my-project-{env} (env: stg/prd)
I want to loop over the variables, instead of writing down repetitive resource for each user.
Example:
variable some_ext_users {
type = map(any)
default = {
user_1 = { email_id = "user_1#gmail.com" }
user_2 = { email_id = "user_2#gmail.com" }
}
}
To avoid repetitive resource made on each user (imagine 100++ users), I decided to list them in variable as written above.
Then, I'd like to assign these user GCS permission, e.g:
resource "google_storage_bucket_iam_member" "user_email_access" {
for_each = var.some_ext_users
count = var.env == "stg" ? 1 : 0
provider = google-beta
bucket = "my-bucketttt"
role = "roles/storage.objectViewer"
member = "user:${each.value.email_id}"
}
The error I'm getting is clear :
Error: Invalid combination of "count" and "for_each" on
../../../modules/my-tf.tf line 54, in resource
"google_storage_bucket_iam_member" "user_email_access": 54:
for_each = var.some_ext_users The "count" and "for_each"
meta-arguments are mutually-exclusive, only one should be used to be
explicit about the number of resources to be created.
My question is, what is the workaround in order to satisfy the requirements above if count and for_each can't be used together?
You could control the user list according to the environment, rather than trying to control the resource. So, something like this:
resource "google_storage_bucket_iam_member" "user_email_access" {
for_each = var.env == "stg" ? var.some_ext_users : {}
provider = google-beta
bucket = "my-bucketttt"
role = "roles/storage.objectViewer"
member = "user:${each.value.email_id}"
}
The rule for for_each is to assign it a map that has one element per instance you want to declare, so the best way to think about your requirement here is that you need to write an expression that produces a map with zero elements when your condition doesn't hold.
The usual way to project and filter collections in Terraform is for expressions, and indeed we can use a for expression with an if clause to conditionally filter out unwanted elements, which in this particular case will be all of the elements:
resource "google_storage_bucket_iam_member" "user_email_access" {
for_each = {
for name, user in var.some_ext_users : name => user
if var.env == "stg"
}
# ...
}
Another possible way to structure this would be to include the environment keywords as part of the data structure, which would keep all of the information in one spot and potentially allow you to have entries that apply to more than one environment at once:
variable "some_ext_users" {
type = map(object({
email_id = string
environments = set(string)
}))
default = {
user_1 = {
email_id = "user_1#gmail.com"
environments = ["stg"]
}
user_2 = {
email_id = "user_2#gmail.com"
environments = ["stg", "prd"]
}
}
}
resource "google_storage_bucket_iam_member" "user_email_access" {
for_each = {
for name, user in var.some_ext_users : name => user
if contains(user.environments, var.env)
}
# ...
}
This is a variation of the example in the "Filtering Elements" documentation I linked above, which uses an is_admin flag in order to declare different resources for admin users vs. non-admin users. In this case, notice that the if clause refers to the symbols declared in the for expression, which means we can now get a different result for each element of the map, whereas the first example either kept all elements or no elements.
I have the following code..
data "aws_secretsmanager_secret" "db_password" {
name = "${var.db_secret}"
}
data "aws_secretsmanager_secret_version" "db_password" {
secret_id = "${data.aws_secretsmanager_secret.db_password.id}"
}
master_password = "${data.aws_secretsmanager_secret_version.db_password.secret_string}"
which returns the secret_string in this case of
secret_string = {"Test":"TestPassword"}
how do i cut out and use the TestPassword section of the secret for use as my master_password?
I had to fake up your Secrets endpoint but this test endpoint returns the same json:
So in tf...
data "external" "secret_string" {
program = ["curl", "http://echo.jsontest.com/Test/Testpassword"]
}
output "json_data_key" {
value = "${data.external.secret_string.result}"
}
output "PASSWORD" {
value = "${lookup(data.external.secret_string.result, "Test")}"
}
that last output is what you were after?
${lookup(data.external.secret_string.result, "Test")}
Which gives you:
data.external.secret_string: Refreshing state...
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
Outputs:
PASSWORD = Testpassword
json_data_key = {
Test = Testpassword
}
So it is certainly possible to parse json before 0.12......
Considering this is JSON, you probably need to wait for jsondecode in Terraform v0.12 to solve the problem.
jsondecode function Github issue