terraform: is there a way to create iam policy statements dynamically? - amazon-web-services

Terraform version: 0.11
I am running multiple eks clusters and trying to enable IAM Roles-based service account in all cluster following this doc:
https://www.terraform.io/docs/providers/aws/r/eks_cluster.html#enabling-iam-roles-for-service-accounts
This works when I hardcode the cluster name in the policy statement and create multiple statements
data "aws_iam_policy_document" "example_assume_role_policy" {
# for cluster 1
statement {
actions = ["sts:AssumeRoleWithWebIdentity"]
effect = "Allow"
condition {
test = "StringEquals"
variable = "${replace(aws_iam_openid_connect_provider.example1.url, "https://", "")}:sub"
values = ["system:serviceaccount:kube-system:aws-node"]
}
principals {
identifiers = ["${aws_iam_openid_connect_provider.example1.arn}"]
type = "Federated"
}
}
}
Since I have multiple clusters, I want to be able to generate the statement dynamically
so I made the following changes:
I created a count variable and changed values in principals and and condition
count = "${length(var.my_eks_cluster)}"
condition {
test = "StringEquals"
variable = "${replace(element(aws_iam_openid_connect_provider.*.url, count.index), "https://", "")}:sub"
values = ["system:serviceaccount:kube-system:aws-node"]
}
principals {
identifiers = ["${element(aws_iam_openid_connect_provider.*.url, count.index)}"]
type = "Federated"
}
Terraform now is able to find the clusters BUT also generate multiple policies.
And this will not work, since in the following syntax, the assume_role_policy doesn't take the list
resource "aws_iam_role" "example" {
assume_role_policy = "${data.aws_iam_policy_document.example_assume_role_policy.*.json}"
name = "example"
}
It seems like instead of creating multiple policies, I need to generate multiple statements in one policy (so I can add to one iam_role). Has anyone done something similar before ? Thanks.

You only want one policy, so you should not use the count argument in your policy.
What you want to have instead is multiple statements, like this
data "aws_iam_policy_document" "example" {
statement {
# ...
}
statement {
# ...
}
}
Now you could hard-code this directly (maybe that would be a good start to test if it works). If you want to generate this dynamically from a variable you would need a dynamic-block as described here: https://www.terraform.io/docs/configuration/expressions.html
In your case that would probably be
data "aws_iam_policy_document" "example" {
dynamic "statement" {
for_each = aws_iam_openid_connect_provider
content {
actions = ["sts:AssumeRoleWithWebIdentity"]
effect = "Allow"
condition {
test = "StringEquals"
variable = "${replace(statement.value.url, "https://", "")}:sub"
values = ["system:serviceaccount:kube-system:aws-node"]
}
principals {
identifiers = ["${statement.value.arn}"]
type = "Federated"
}
}
}
}
I think that "dynamic" is only available since TF 0.12, though.

Related

Adding S3 bucket policy to multiple buckets with for_each Terraform module

I have the following module which works fine.
Module
resource "aws_s3_bucket" "buckets" {
bucket = var.s3_buckets
}
resource "aws_s3_bucket_acl" "buckets" {
bucket = var.s3_buckets
acl = "private"
}
Root module
module "s3_buckets" {
source = "./modules/s3"
for_each = toset([
"bucket-test1-${var.my_env}",
"bucket-test2-${var.my_env}",
])
s3_buckets = each.value
}
I would like to add get the following policy to all the buckets in the list. Obviously the count option below does not work.
data "aws_iam_policy_document" "buckets" {
count = length(var.s3_buckets)
statement {
sid = "AllowSSlRequestsOnly"
actions = ["s3:*"]
effect = "Deny"
condition {
test = "Bool"
variable = "aws:SecureTransport"
values = ["false"]
}
principals {
type = "*"
identifiers = ["*"]
}
resources = ["arn:aws:s3:::${var.s3_buckets}"]
}
}
resource "aws_s3_bucket_policy" "buckets" {
bucket = var.s3_buckets
policy = data.aws_iam_policy_document.buckets[count.index].json
}
I'm thinking that I need another for_each and use a resource for the IAM policy, I have seen an example such as below, but in the current form I'm providing a string instead a set of strings. Any ideas?
resource "aws_s3_bucket_policy" "buckets" {
for_each = var.s3_buckets
bucket = each.key
policy = jsonencode({
Version = "2012-10-17"
Id = "AllowSSlRequestsOnly",
Statement = [
{
Sid = "AllowSSlRequestsOnly"
Effect = "Deny"
Principal = "*"
Action = "s3:*"
Resource = each.value.arn
Condition = {
Bool = {
"aws:SecureTransport": "false"
}
}
}
]
})
}
If you are adding the policy inside the module (which you probably are, otherwise it doesn't make much sense to attach the policies outside, since you have full control) - then why do you need to mingle with count() at all?
Create the policy and attach to the bucket like:
data "aws_iam_policy_document" "buckets" {
statement {
sid = "AllowSSlRequestsOnly"
actions = ["s3:*"]
effect = "Deny"
condition {
test = "Bool"
variable = "aws:SecureTransport"
values = ["false"]
}
principals {
type = "*"
identifiers = ["*"]
}
resources = ["arn:aws:s3:::${var.s3_buckets}"]
}
}
resource "aws_s3_bucket_policy" "buckets" {
bucket = var.s3_buckets
policy = data.aws_iam_policy_document.buckets.json
}
Couple more points:
It's a singular bucket that you are passing to the module, yet the variable is named s3_buckets, it is confusing.
Using var.s3_buckets for all dependent resources is not the best practice. Create the bucket with var.s3.buckets, after which use the outputs of the resource. This and examples of policies reside here: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket
good luck ☺️

How to refer parent resource in child resource when using terraform map (and for_each)

I have the following code which works fine.
resource "aws_ses_email_identity" "main_from_email" {
email = "some#email.com"
}
data "aws_iam_policy_document" "main_from_email_policy_document" {
statement {
actions = ["SES:SendEmail", "SES:SendRawEmail"]
resources = [aws_ses_email_identity.main_from_email.arn]
principals {
identifiers = ["*"]
type = "AWS"
}
}
}
resource "aws_ses_identity_policy" "email_notif_policy" {
identity = aws_ses_email_identity.main_from_email.arn
name = "${local.namespace}-ses_main_from_email_policy"
policy = data.aws_iam_policy_document.main_from_email_policy_document.json
}
The above code is working fine. But the email is hard coded. I would like to have the (email) resources to be created based on some configuration provided in terraform.tfvars as follows:
clientemails = {
"client1" = { "email" = "client1#gmail.com" }
"client2" = { "email" = "client2#gmail.com" }
}
I modified main resource as follows:
resource "aws_ses_email_identity" "main_from_email_map" {
for_each = var.clientemails
email = each.value.email
}
But, I don't know how I can modify "aws_iam_policy_document" and "aws_ses_identity_policy" to follow "aws_ses_email_identity".
How do I modify my terraform script to honor "clientemails" configuration?
This should be relatively straight forward to do:
resource "aws_ses_email_identity" "main_from_email_map" {
for_each = var.clientemails
email = each.value.email
}
data "aws_iam_policy_document" "main_from_email_policy_document" {
statement {
actions = ["SES:SendEmail", "SES:SendRawEmail"]
resources = values(aws_ses_email_identity.main_from_email_map)[*].arn
principals {
identifiers = ["*"]
type = "AWS"
}
}
}
resource "aws_ses_identity_policy" "email_notif_policy" {
for_each = aws_ses_email_identity.main_from_email_map
identity = each.value.arn
name = "${local.namespace}-ses_main_from_email_policy"
policy = data.aws_iam_policy_document.main_from_email_policy_document.json
}
Here, there will be a single policy which will allow all the same actions for all the SES email identity ARNs. To achieve this, since the SES email identity resource was created using for_each meta-argument, the values built-in function [1] was used to fetch all the values of the ARN attribute for all the keys (hence the [*] part).
There could be a way to modify the data source with for_each if needed (i.e., to create a policy per client email):
data "aws_iam_policy_document" "main_from_email_policy_document" {
for_each = aws_ses_email_identity.main_from_email_map
statement {
actions = ["SES:SendEmail", "SES:SendRawEmail"]
resources = [each.value.arn]
principals {
identifiers = ["*"]
type = "AWS"
}
}
}
Then, you would also have to fix the SES identity policy:
resource "aws_ses_identity_policy" "email_notif_policy" {
for_each = aws_ses_email_identity.main_from_email_map
identity = each.value.arn
name = "${local.namespace}-ses_main_from_email_policy"
policy = data.aws_iam_policy_document.main_from_email_policy_document[each.key].json
}
The code where for_each meta-argument is used with the resources (e.g., aws_ses_email_identity.main_from_email_map) is called for_each chaining [2].
[1] https://developer.hashicorp.com/terraform/language/functions/values
[2] https://developer.hashicorp.com/terraform/language/meta-arguments/for_each#chaining-for_each-between-resources

Get a list of created resources in terraform

I am creating AWS ECR repositories via terraform
resource "aws_ecr_repository" "repo1" {
name = "repo1"
image_tag_mutability = "MUTABLE"
image_scanning_configuration {
scan_on_push = true
}
}
resource "aws_ecr_repository" "repo2" {
name = "repo2"
image_tag_mutability = "MUTABLE"
image_scanning_configuration {
scan_on_push = true
}
}
Now I want to attach a policy to all ECR repositories.
Question is, is there a dynamic way to create a list of all the resources (of type ECR) created using the terraform script? If yes then we can have a for_each on that list and attach a policy.
Or is there any better way to do it?
P.S. I know I can attach policy by writing the following for each. I want to avoid duplication and avoid a case where policy is not attached if the block is missed by someone
resource "aws_ecr_lifecycle_policy" "insights_repository_policy" {
repository = aws_ecr_repository.insights_repository.name
policy = local.ecr_cleanup_policy
}
Edit: Question 2
There are some accounts I want to give access to. If I use list of repositories to create and then I want to assign policies for each account then it would make nested for loops. Is there a cleaner solution for that?
local {
accounts = {test=account_id_123, prod=account_id_456}
}
resource "aws_ecr_repository_policy" "access-permission" {
for_each = local.accounts
policy = <<POLICY
...
POLICY
repository = aws_ecr_repository.repo_template.name
}
Not in your form. It would be better if you used for_each or count. For example:
variable "repos" {
default = ["repo1", "repo2"]
}
resource "aws_ecr_repository" "repo" {
for_each = to_set(var.repos)
name = each.key
image_tag_mutability = "MUTABLE"
image_scanning_configuration {
scan_on_push = true
}
}
then you can do:
resource "aws_ecr_lifecycle_policy" "insights_repository_policy" {
for_each = aws_ecr_repository.repo
repository = each.value.name
policy = local.ecr_cleanup_policy
}

How to loop through a list of s3 buckets and create and attach a number of policies for each bucket?

I am learning about terraform modules and my objective is to build module which takes in a collection of s3 Buckets, and then creates and applies to them some iam policies.
What I have tried so far was to have some sort of a for loop, where I generate the policies and attach them to the buckets. For reference, my code looks something like this:
data "aws_iam_policy_document" "foo_iam_policy" {
statement {
sid = ""
effect = "Allow"
resources = [
for arn in var.s3_buckets_arn :
"${arn}/*"
]
actions = [
"s3:GetObject",
"s3:GetObjectVersion",
]
}
statement {
sid = ""
effect = "Allow"
resources = var.s3_buckets_arn
actions = ["s3:*"]
}
}
resource "aws_iam_policy" "foo_iam_policy" {
name = "foo-iam-policy"
path = "/"
description = "IAM policy for foo to access S3"
policy = data.aws_iam_policy_document.foo_iam_policy.json
}
data "aws_iam_policy_document" "foo_assume_rule_policy" {
statement {
effect = "Allow"
actions = [
"sts:AssumeRole"]
principals {
type = "AWS"
identifiers = [
var.foo_iam_user_arn]
}
condition {
test = "StringEquals"
values = var.foo_external_ids
variable = "sts:ExternalId"
}
}
}
resource "aws_iam_role" "foo_role" {
name = "foo-role"
assume_role_policy = data.aws_iam_policy_document.foo_assume_rule_policy.json
}
resource "aws_iam_role_policy_attachment" "foo_attach_s3_policy" {
role = aws_iam_role.foo_role.name
policy_arn = aws_iam_policy.foo_iam_policy.arn
}
data "aws_iam_policy_document" "foo_policy_source" {
for_each = toset(var.s3_buckets_arn)
// arn = each.key
statement {
sid = "VPCAllow"
effect = "Allow"
resources = [
each.key,
"${each.key}/*",
]
actions = [
"s3:*"]
condition {
test = "StringEquals"
variable = "aws:SourceVpc"
values = [
"vpc-01010101"]
}
principals {
type = "*"
identifiers = [
"*"]
}
}
}
I don't know if what I have tried makes much sense, or if there is a better way to loop through buckets and generate policies. My question is: what is the best practice for such cases where one wants to provide a list of buckets and loop through them to attach policies?
On a side note, I have encountered an error with my approach:
The “for_each” value depends on resource attributes that cannot be
determined (Terraform)
To attach a bucket policy to a bucket you should use aws_s3_bucket_policy, not aws_iam_policy_document. Also if the buckets already exist, probably it would be better to fetch their data first using data source aws_s3_bucket:
data "aws_s3_bucket" "selected" {
# s3_buckets_names easier to use then s3_buckets_arns
for_each = toset(var.s3_buckets_names)
bucket = each.value
}
Then, you can iterate over the selected buckets and add your policy to it:
resource "aws_s3_bucket_policy" "bucket_policie" {
for_each = data.aws_s3_bucket.selected
bucket = each.key
policy = "your policy document"
}

How can I attach a generic policy to more than 1 bucket, while having the bucket arn dynamically?

I am learning terraform and currently attempting to attach a policy to a created bucket. More specifically, the policy I want to attach has the same permissions/structure but the only difference is the resources section. I will illustrate with an example:
Let's say I create an s3 bucket like:
module "happy_bucket" {
source = "outer space"
name = "happy-bucket"
}
And another bucket like:
module "sad_bucket" {
source = "outer space"
name = "sad-bucket"
}
And now I have a policy that looks like:
data "aws_iam_policy_document" "some_policy" {
statement {
effect = "Allow"
resources = [module.some_bucket.bucket_arn]
actions = ["s3:GetObject", "s3:GetObjectVersion"]
}
}
And now I would like to attach "some_policy" to both "sad-bucket" and "happy-bucket". But I want to do that without having to repeat myself by creating the policy two times (because I need the .bucket_arn to be based on the bucket). In other words, I want to create one generic policy, and attach it to the 2 buckets I created (while picking up the arn dynamically).
Bucket policies require principals, so you need to add that to your some_policy. Having said that, if you want to keep using aws_iam_policy_document you use for_each to iterate over your buckets.
For example, if your module is:
variable "name" {}
resource "aws_s3_bucket" "b" {
bucket = var.name
}
output "bucket_arn" {
value = aws_s3_bucket.b.arn
}
output "bucket_name" {
value = aws_s3_bucket.b.id
}
then in parent module, you can:
module "happy_bucket" {
source = "./modules/buckets"
name = "happy-bucket-231123124ff"
}
module "sad_bucket" {
source = "./modules/buckets"
name = "sad-bucket-231123124ff"
}
locals {
my_buckets = {for bucket in [module.happy_bucket, module.sad_bucket]:
bucket.bucket_name => bucket}
}
data "aws_iam_policy_document" "some_policy" {
for_each = local.my_buckets
statement {
effect = "Allow"
resources = ["${each.value.bucket_arn}/*"]
actions = ["s3:GetObject", "s3:GetObjectVersion"]
principals {
type = "AWS"
identifiers = ["*"]
}
}
}
resource "aws_s3_bucket_policy" "bucket_policie" {
for_each = local.my_buckets
bucket = each.key
policy = data.aws_iam_policy_document.some_policy[each.key].json
}