Apply same data for two resources in loop terraform - amazon-web-services

I want to create two Amazon SNS topics with the same aws_iam_policy_document, aws_sns_topic_policy & time_sleep configs.
This is my terraform, my_sns_topic.tf:
resource "aws_sns_topic" "topic_a" {
name = "topic-a"
}
resource "aws_sns_topic" "topic_b" {
name = "topic-b"
}
data "aws_iam_policy_document" "topic_notification" {
version = "2008-10-17"
statement {
sid = "__default_statement_ID"
actions = [
"SNS:Publish"
]
# Cut off some lines for simplification.
## NEW LINE ADDED
statement {
sid = "allow_snowflake_subscription"
principals {
type = "AWS"
identifiers = [var.storage_aws_iam_user_arn]
}
actions = ["SNS:Subscribe"]
resources = [aws_sns_topic.topic_a.arn] # Troubles with this line
}
}
resource "aws_sns_topic_policy" "topic_policy_notification" {
arn = aws_sns_topic.topic_a.arn
policy = data.aws_iam_policy_document.topic_policy_notification.json
}
resource "time_sleep" "topic_wait_10s" {
depends_on = [aws_sns_topic.topic_a]
create_duration = "10s"
}
As you can see here, I set up the configuration only for topic-a. I want to loop this over to apply for topic-b as well.

It would be better to use map and for_each, instead of separately creating "a" and "b" topics:
variable "topics" {
default = ["a", "b"]
}
resource "aws_sns_topic" "topic" {
for_each = toset(var.topics)
name = "topic-${each.key}"
}
data "aws_iam_policy_document" "topic_notification" {
version = "2008-10-17"
statement {
sid = "__default_statement_ID"
actions = [
"SNS:Publish"
]
# Cut off some lines for simplification.
}
resource "aws_sns_topic_policy" "topic_policy_notification" {
for_each = toset(var.topics)
arn = aws_sns_topic.topic[each.key].arn
policy = data.aws_iam_policy_document.topic_policy_notification.json
}
resource "time_sleep" "topic_wait_10s" {
for_each = toset(var.topics)
depends_on = [aws_sns_topic.topic[each.key]]
create_duration = "10s"
}

Related

Adding S3 bucket policy to multiple buckets with for_each Terraform module

I have the following module which works fine.
Module
resource "aws_s3_bucket" "buckets" {
bucket = var.s3_buckets
}
resource "aws_s3_bucket_acl" "buckets" {
bucket = var.s3_buckets
acl = "private"
}
Root module
module "s3_buckets" {
source = "./modules/s3"
for_each = toset([
"bucket-test1-${var.my_env}",
"bucket-test2-${var.my_env}",
])
s3_buckets = each.value
}
I would like to add get the following policy to all the buckets in the list. Obviously the count option below does not work.
data "aws_iam_policy_document" "buckets" {
count = length(var.s3_buckets)
statement {
sid = "AllowSSlRequestsOnly"
actions = ["s3:*"]
effect = "Deny"
condition {
test = "Bool"
variable = "aws:SecureTransport"
values = ["false"]
}
principals {
type = "*"
identifiers = ["*"]
}
resources = ["arn:aws:s3:::${var.s3_buckets}"]
}
}
resource "aws_s3_bucket_policy" "buckets" {
bucket = var.s3_buckets
policy = data.aws_iam_policy_document.buckets[count.index].json
}
I'm thinking that I need another for_each and use a resource for the IAM policy, I have seen an example such as below, but in the current form I'm providing a string instead a set of strings. Any ideas?
resource "aws_s3_bucket_policy" "buckets" {
for_each = var.s3_buckets
bucket = each.key
policy = jsonencode({
Version = "2012-10-17"
Id = "AllowSSlRequestsOnly",
Statement = [
{
Sid = "AllowSSlRequestsOnly"
Effect = "Deny"
Principal = "*"
Action = "s3:*"
Resource = each.value.arn
Condition = {
Bool = {
"aws:SecureTransport": "false"
}
}
}
]
})
}
If you are adding the policy inside the module (which you probably are, otherwise it doesn't make much sense to attach the policies outside, since you have full control) - then why do you need to mingle with count() at all?
Create the policy and attach to the bucket like:
data "aws_iam_policy_document" "buckets" {
statement {
sid = "AllowSSlRequestsOnly"
actions = ["s3:*"]
effect = "Deny"
condition {
test = "Bool"
variable = "aws:SecureTransport"
values = ["false"]
}
principals {
type = "*"
identifiers = ["*"]
}
resources = ["arn:aws:s3:::${var.s3_buckets}"]
}
}
resource "aws_s3_bucket_policy" "buckets" {
bucket = var.s3_buckets
policy = data.aws_iam_policy_document.buckets.json
}
Couple more points:
It's a singular bucket that you are passing to the module, yet the variable is named s3_buckets, it is confusing.
Using var.s3_buckets for all dependent resources is not the best practice. Create the bucket with var.s3.buckets, after which use the outputs of the resource. This and examples of policies reside here: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket
good luck ☺️

How to create multiple policies and policy attachments for each iam group terraform

So I have multiple IAM groups which I am looping through as follows :
resource "aws_iam_group" "all_iam_groups" {
for_each = var.iam_user_groups
name = "${local.csi}-${each.key}"
path = "/"
}
This will create multiple IAM groups. Now for each IAM groups I will have an IAM Policy which I will attatch. Rather than having to do it manually and create multiple resources what is the best approach to take here. So far I have been doing it as follows :
resource "aws_iam_policy" "finance_read_only" {
name = "${local.csi}-finance-read-only"
path = "/"
policy = data.aws_iam_policy_document.finance_read_only.json
}
resource "aws_iam_policy" "security_read_only" {
name = "${local.csi}-security-read-only"
path = "/"
policy = data.aws_iam_policy_document.security_read_only.json
}
resource "aws_iam_group_policy_attachment" "security_read_only" {
group = aws_iam_group.security_team.name
policy_arn = aws_iam_policy.security_read_only.arn
}
resource "aws_iam_group_policy_attachment" "finance_read_only" {
group = aws_iam_group.finance_team.name
policy_arn = aws_iam_policy.finance_read_only.arn
}
For example purposes I have added in the group names but I know i can use a for-each loop to go through the names but am not sure how I would be able to look through the IAM Policies and policy attatchments.
I do understand that I will have multiple aws_iam_policy_document which is fine.
There's a direct relationship between the following resources: aws_iam_policy , the data source for aws_iam_policy_document, the aws_iam_group_policy_attachment and the aws_iam_group resource.
My suggestion is to create a variable of a list of objects type, where you define different attributes that are needed for the resources required.
Example:
variable "policies" {
type = list(object({
name = string
statement = object
}))
default = [
{
name = "finance-read-only"
statement = {
sid = "XXX"
actions = ["ec2:XXX"]
resources = ["XXX"]
}
},
{
name = "security-read-only"
statement = {
sid = "XXX"
actions = ["ec2:XXX"]
resources = ["XXX"]
}
}
]
}
data "aws_iam_policy_document" "this" {
for_each = { for policy in var.policies : policy.name => policy }
statement = each.value.statement
}
resource "aws_iam_policy" "this" {
for_each = { for policy in var.policies : policy.name => policy }
name = format("${local.csi}-%s", each.key)
path = "/"
policy = data.aws_iam_policy_document.this[each.key].json
}
resource "aws_iam_group" "this" {
for_each = { for policy in var.policies : policy.name => policy }
name = each.key
}
resource "aws_iam_group_policy_attachment" "security_read_only" {
for_each = { for policy in var.policies : policy.name => policy }
group = aws_iam_group.this[each.key].name
policy_arn = aws_iam_policy.this[each.key].arn
}

Terraform give IAM access to specific folder only in S3

How can I create an IAM user using terraform with access to one folder in s3
I found a lot of answers that give access to the entire bucket but I want access to one folder only
resource "aws_iam_policy" "username-s3-access" {
name = "username-s3-access"
path = "/"
policy = data.aws_iam_policy_document.username-s3-access.json
}
resource "aws_iam_user" "username-s3-access" {
name = "username-s3-access"
path = "/machine/"
}
resource "aws_iam_user_policy_attachment" "username-s3-access" {
user = aws_iam_user.username-s3-access.name
policy_arn = aws_iam_policy.username-s3-access.arn
}
I found the solution based on this official documentation in AWS site https://aws.amazon.com/premiumsupport/knowledge-center/s3-folder-user-access/
This code will give full assess to one folder in the bucket
scroll down to find the code for readonly access to one folder
Note: change the bucket_arn and the folder_name
variable "bucket_arn" {
type = string
default = "bucket_arn"
}
variable "folder_name" {
type = string
default = "s3_folder_name"
}
data "aws_iam_policy_document" "username-s3-access" {
statement {
sid = "AllowUserToSeeBuckets"
effect = "Allow"
resources = ["arn:aws:s3:::*"]
actions = [
"s3:ListAllMyBuckets",
"s3:GetBucketLocation",
]
}
statement {
sid = "AllowBucketListing"
effect = "Allow"
resources = [ var.bucket_arn ]
actions = ["s3:ListBucket"]
}
statement {
sid = "AllowFolderListing"
effect = "Allow"
resources = [ var.bucket_arn ]
actions = ["s3:ListBucket"]
condition {
test = "StringEquals"
variable = "s3:prefix"
values = [
"",
"${var.folder_name}"
]
}
condition {
test = "StringEquals"
variable = "s3:delimiter"
values = ["/"]
}
}
statement {
sid = "AllowAllS3ActionsInUserFolder"
effect = "Allow"
resources = ["${var.bucket_arn}/${var.folder_name}/*"]
actions = ["s3:*"]
}
}
resource "aws_iam_policy" "username-s3-access" {
name = "username-s3-access"
path = "/"
policy = data.aws_iam_policy_document.username-s3-access.json
}
resource "aws_iam_user" "username-s3-access" {
name = "username-s3-access"
path = "/machine/"
}
resource "aws_iam_user_policy_attachment" "username-s3-access" {
user = aws_iam_user.username-s3-access.name
policy_arn = aws_iam_policy.username-s3-access.arn
}
if you want to give read only access to the folder then replace the last statement
statement {
sid = "AllowReadOnlyInUserFolder"
effect = "Allow"
resources = ["${var.bucket_name}/${var.folder_name}/*"]
actions = [
"s3:GetObject",
"s3:ListBucket",
]
}

Amazon SNS (Terraform) Invalid parameter: Policy statement action out of service scope

I want to create an IAM policy document and attached two values taken from snowflake error integration as Trusted Relationship in the policy. Following this tutorial Step 5.
The idea is that to add SF_AWS_IAM_USER_ARN and SF_AWS_EXTERNAL_ID created from
SNOWFLAKE NOTIFICATION INTEGRATION to the policy.
The integration is succesfully created.
This is part of my code:
resource "random_id" "random" {
byte_length = 8
}
resource "aws_sns_topic" "my_sns_topic" {
name = "${var.bucket_name}-errors-${random_id.random.id}"
}
data "aws_iam_policy_document" "snowflake_notification_error" {
version = "2008-10-17"
statement {
sid = "__default_statement_ID"
actions = [
"SNS:GetTopicAttributes",
"SNS:SetTopicAttributes",
"SNS:AddPermission",
"SNS:RemovePermission",
"SNS:DeleteTopic",
"SNS:Subscribe",
"SNS:ListSubscriptionsByTopic",
"SNS:Publish",
"SNS:Receive",
]
principals {
type = "AWS"
identifiers = ["*"]
}
resources = [aws_sns_topic.my_sns_topic.arn]
condition {
test = "StringEquals"
variable = "AWS:SourceOwner"
values = [data.aws_caller_identity.current.account_id]
}
}
statement {
sid = "allow_s3_notification"
principals {
type = "Service"
identifiers = ["s3.amazonaws.com"]
}
actions = ["SNS:Publish"]
resources = [aws_sns_topic.my_sns_topic.arn]
condition {
test = "ArnLike"
variable = "aws:SourceArn"
values = [data.aws_s3_bucket.bucket.arn]
}
}
statement {
sid = "allow_snowflake_subscription"
principals {
type = "AWS"
identifiers = [snowflake_storage_integration.integration.storage_aws_iam_user_arn]
}
actions = ["SNS:Subscribe"]
resources = [aws_sns_topic.my_sns_topic.arn]
}
# Error starts in this block I believe
# The json file looks like in the tutorial shown.
statement {
sid = "allow_error_integration"
principals {
type = "AWS"
identifiers = [snowflake_notification_integration.error_integration.aws_sns_iam_user_arn]
}
actions = ["sts:AssumeRole"]
condition {
test = "StringEquals"
variable = "sts:ExternalId"
values = [snowflake_notification_integration.error_integration.aws_sns_external_id]
}
resources = [aws_sns_topic.my_sns_topic.arn]
}
}
# ERROR HERE
resource "aws_sns_topic_policy" "snowflake_s3_pipe_notification_error" {
arn = aws_sns_topic.my_sns_topic.arn
policy = data.aws_iam_policy_document.snowflake_notification_error.json
}
The error is:
Error: InvalidParameter: Invalid parameter: Policy statement action out of service scope!status code: 400, request id: 5c75a285-294b-56b7-ad4d-f915d5e0b01b
with module.datalake_dev["my-snowpipe"].module.s3_integration.aws_sns_topic_policy.snowflake_notification_error, on ../snowflake/s3_integration/s3_integration/error_integration.tf line 79, in resource "aws_sns_topic_policy" "snowflake_notification_error":
79: resource "aws_sns_topic_policy" "snowflake_notification_error" {
The action "SNS:Receive" is not allowed to be in the policy statement. All of the allowed actions are listed in the AWS documentation at https://docs.aws.amazon.com/sns/latest/dg/sns-access-policy-language-api-permissions-reference.html#sns-valid-policy-actions

How to loop through a list of s3 buckets and create and attach a number of policies for each bucket?

I am learning about terraform modules and my objective is to build module which takes in a collection of s3 Buckets, and then creates and applies to them some iam policies.
What I have tried so far was to have some sort of a for loop, where I generate the policies and attach them to the buckets. For reference, my code looks something like this:
data "aws_iam_policy_document" "foo_iam_policy" {
statement {
sid = ""
effect = "Allow"
resources = [
for arn in var.s3_buckets_arn :
"${arn}/*"
]
actions = [
"s3:GetObject",
"s3:GetObjectVersion",
]
}
statement {
sid = ""
effect = "Allow"
resources = var.s3_buckets_arn
actions = ["s3:*"]
}
}
resource "aws_iam_policy" "foo_iam_policy" {
name = "foo-iam-policy"
path = "/"
description = "IAM policy for foo to access S3"
policy = data.aws_iam_policy_document.foo_iam_policy.json
}
data "aws_iam_policy_document" "foo_assume_rule_policy" {
statement {
effect = "Allow"
actions = [
"sts:AssumeRole"]
principals {
type = "AWS"
identifiers = [
var.foo_iam_user_arn]
}
condition {
test = "StringEquals"
values = var.foo_external_ids
variable = "sts:ExternalId"
}
}
}
resource "aws_iam_role" "foo_role" {
name = "foo-role"
assume_role_policy = data.aws_iam_policy_document.foo_assume_rule_policy.json
}
resource "aws_iam_role_policy_attachment" "foo_attach_s3_policy" {
role = aws_iam_role.foo_role.name
policy_arn = aws_iam_policy.foo_iam_policy.arn
}
data "aws_iam_policy_document" "foo_policy_source" {
for_each = toset(var.s3_buckets_arn)
// arn = each.key
statement {
sid = "VPCAllow"
effect = "Allow"
resources = [
each.key,
"${each.key}/*",
]
actions = [
"s3:*"]
condition {
test = "StringEquals"
variable = "aws:SourceVpc"
values = [
"vpc-01010101"]
}
principals {
type = "*"
identifiers = [
"*"]
}
}
}
I don't know if what I have tried makes much sense, or if there is a better way to loop through buckets and generate policies. My question is: what is the best practice for such cases where one wants to provide a list of buckets and loop through them to attach policies?
On a side note, I have encountered an error with my approach:
The “for_each” value depends on resource attributes that cannot be
determined (Terraform)
To attach a bucket policy to a bucket you should use aws_s3_bucket_policy, not aws_iam_policy_document. Also if the buckets already exist, probably it would be better to fetch their data first using data source aws_s3_bucket:
data "aws_s3_bucket" "selected" {
# s3_buckets_names easier to use then s3_buckets_arns
for_each = toset(var.s3_buckets_names)
bucket = each.value
}
Then, you can iterate over the selected buckets and add your policy to it:
resource "aws_s3_bucket_policy" "bucket_policie" {
for_each = data.aws_s3_bucket.selected
bucket = each.key
policy = "your policy document"
}