Terraform always says changes on templatefile for s3 bucket policy - amazon-web-services

I am having issues with my s3 bucket policy, it seems to add the policy correctly and even verified it in AWS and it shows the exact policy set in the policy.tpl but it keeps saying there are changes
I've tried changing the action and resource into arrays which I've heard may help.. tried removing the "Version" from the policy, the SID, keeps saying there are changes everytime i run it
policy.tf
resource "aws_s3_bucket_policy" "bucket" {
bucket = aws_s3_bucket.bucket.id
policy = local.policy
}
locals.tf
locals {
template_dir = "${path.module}/templates"
template_vars = {
encrypt = var.s3_require_encryption_enabled
bucket_arn = aws_s3_bucket.bucket.arn
extra_statements = var.s3_bucket_policy
}
policy = templatefile("${local.template_dir}/policy.tpl", local.template_vars)
}
templates/policy.tpl
{
"Version": "2008-10-17",
"Statement": [
{
"Sid" : "",
"Effect" : "Deny",
"Principal" : "*",
"Action" : "s3:*",
"Resource" : "${bucket_arn}/*",
"Condition": {
"Bool": {
"aws:SecureTransport": "false"
}
}
}
]
}
in AWS
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": "arn:aws:s3:::test-bucket-us-east-1/*",
"Condition": {
"Bool": {
"aws:SecureTransport": "false"
}
}
}
]
}
says
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
~ update in-place
Terraform will perform the following actions:
#aws_s3_bucket_policy.bucket will be updated in-place
~ resource "aws_s3_bucket_policy" "bucket" {
bucket = "test-bucket-us-east-1"
id = "test-bucket-us-east-1"
+ policy = jsonencode(
{
+ Statement = [
+ {
+ Action = "s3:*"
+ Condition = {
+ Bool = {
+ aws:SecureTransport = "false"
}
}
+ Effect = "Deny"
+ Principal = "*"
+ Resource = "arn:aws:s3:::test-bucket-us-east-1/*"
+ Sid = ""
},
]
+ Version = "2008-10-17"
}
)
}
Plan: 0 to add, 1 to change, 0 to destroy.

Based on the comments, the underlying bucket policy had issues.
PutBucketPolicy has
Content-MD5
The MD5 hash of the request body.
For requests made using the AWS Command Line Interface (CLI) or AWS SDKs, this field is calculated automatically.)
So resource aws_s3_bucket_policy was trying to update the policy.

Related

Create parameterized resource policy on terraform

I want to create a resource policy for a Secrets Manager secret.
I am following the official example on the docs
resource "aws_secretsmanager_secret_policy" "this" {
count = var.create_resource_policy ? 1 : 0
secret_arn = aws_secretsmanager_secret.mysecret.arn
policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EnableAnotherAWSAccountToReadTheSecret",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789012:root"
},
"Action": "secretsmanager:GetSecretValue",
"Resource": "*"
}
]
}
POLICY
}
Is there a way to pass the following as variables in the policy document things as the principal(s), the action the resources etc?
I want to be able to pass those things as terraform variables.
Yes, you can use the built-in functions in terraform for that with interpolation syntax. For example, if you had a data source to get the account ID, you could do the following:
data "aws_caller_identity" "current" {}
resource "aws_secretsmanager_secret_policy" "this" {
count = var.create_resource_policy ? 1 : 0
secret_arn = aws_secretsmanager_secret.mysecret.arn
policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EnableAnotherAWSAccountToReadTheSecret",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${data.aws_caller_identity.current.id}:root"
},
"Action": "secretsmanager:GetSecretValue",
"Resource": "*"
}
]
}
POLICY
}
You could then do the same for any other property you want/need. However, I find it easier to use the built-in data source [1] for creating policies. So for example, the policy you have could be written in the following way:
data "aws_iam_policy_document" "secrets_manager_policy" {
statement {
sid = "EnableAnotherAWSAccountToReadTheSecret"
effect = "Allow"
principals {
identifiers = ["arn:aws:iam::${data.aws_caller_identity.current.id}:root"]
type = "AWS"
}
actions = [
"secretsmanager:GetSecretValue"
]
resources = ["*"]
}
}
You could then tell actions argument to use a variable which would most preferably be a list(string) which would list all the actions necessary:
data "aws_iam_policy_document" "secrets_manager_policy" {
statement {
sid = "EnableAnotherAWSAccountToReadTheSecret"
effect = "Allow"
principals {
identifiers = "arn:aws:iam::${data.aws_caller_identity.current.id}:root"
type = "AWS"
}
actions = var.secrets_actions
resources = [ "*" ]
}
}
Then, you would only have to reference the output of the data source in the original resource:
resource "aws_secretsmanager_secret_policy" "this" {
count = var.create_resource_policy ? 1 : 0
secret_arn = aws_secretsmanager_secret.mysecret.arn
policy = data.aws_iam_policy_document.secrets_manager_policy.json
}
[1] https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document

aws terraform policy returns 'malformed' although it looks right

I'm following these instructions from AWS to try to add access logs to my application load balancer.
Precisely the policies described at the Bucket permissions -> To prepare an Amazon S3 bucket for access logging -> point 5
Unfortunately when I add the policy it returns 'MalformedPolicy: Invalid policy syntax"
Can anyone point me in the right direction with this?
Policy data:
data "aws_iam_policy_document" "allow-lb-aws" {
statement {
principals {
type = "service"
identifiers = ["logdelivery.elb.amazonaws.com"]
}
actions = [
"s3:PutObject",
"s3:ListBucket",
]
resources = [
"arn:aws:s3:::${aws_s3_bucket.lb-logs.bucket}/main-lb/AWSLogs/652711504416/*"
]
}
}
In the console, this shows up as:
+ resource "aws_s3_bucket_policy" "allow-lb-aws" {
+ bucket = "<my-bucket-name>"
+ id = (known after apply)
+ policy = jsonencode(
{
+ Statement = [
+ {
+ Action = [
+ "s3:PutObject",
+ "s3:ListBucket",
]
+ Effect = "Allow"
+ Principal = {
+ service = "logdelivery.elb.amazonaws.com"
}
+ Resource = "arn:aws:s3:::<my bucket name>/main-lb/AWSLogs/<europe-west-2-listed-account>/*"
+ Sid = ""
},
]
+ Version = "2012-10-17"
}
)
}
You are trying to capture the logs from ALB to S3, the policy should be like this:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<aws_elb_account_id>:root"
},
"Action": "s3:PutObject",
"Resource": [
"arn:aws:s3:::${aws_s3_bucket.lb-logs.bucket}/main-lb/AWSLogs/<your_aws_account_id>/*"
]
}
]
}
From the docs you mentioned (https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html), for region eu-west-2, the aws_elb_account_id is 652711504416.
I believe that s3:PutObject is enough to put the logs.

Passing List to IAM Group Policy Template file

I am creating policy based on tags. The environment value differs for each resource, so I have to add them as multiple values. Unable to pass as list. Tried join,split and for loop. None of them works. Pls help. Below code just add the value as "beta,test" which will not work as expected
main.tf
locals{
workspaceValues = terraform.workspace == "dev" ? ["alpha", "dev"] : terraform.workspace == "test" ? ["beta", "test"] : ["prod", "staging"]
}
resource "aws_iam_group_policy" "inline_policy" {
name = "${terraform.workspace}_policy"
group = aws_iam_group.backend_admin.name
policy = templatefile("policy.tpl", { env = join(",", local.workspaceValues), region = "${data.aws_region.current.name}", account_id = "${data.aws_caller_identity.current.account_id}" })
}
policy.tpl:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "ec2:*",
"Resource": "*",
"Condition": {
"ForAllValues:StringLikeIfExists": {
"aws:ResourceTag/Environment": "${env}"
}
}
}
]
}
You can use jsonencode to properly format TF list as a list in json in your template:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "ec2:*",
"Resource": "*",
"Condition": {
"ForAllValues:StringLikeIfExists": {
"aws:ResourceTag/Environment": ${jsonencode(env)}
}
}
}
]
}
For that you would call the template as follows:
resource "aws_iam_group_policy" "inline_policy" {
name = "${terraform.workspace}_policy"
group = aws_iam_group.backend_admin.name
policy = templatefile("policy.tpl", {
env = local.workspaceValues
})
}
You are not using region nor account_id in your template, so there is no reason to pass them in.
An alternative solution to avoid this all together is to recreate the policy using the data component "aws_iam_policy_document" and then pass it to the aws_iam_policy resource like the following,
data "aws_iam_policy_document" "example" {
statement {
effect = "Allow"
actions = [
"ec2:*"
]
resources = [
"*"
]
condition {
test = "ForAllValues:StringLikeIfExists"
variable = "aws:ResourceTag/Environment"
values = local.workspaceValues
}
}
}
resource "aws_iam_policy" "example" {
name = "example_policy"
path = "/"
policy = data.aws_iam_policy_document.example.json
}

S3 Cross region replication using Terraform

I was using Terraform to setup S3 buckets (different region) and set up replication between them.
It was working properly until I added KMS in it.
I created 2 KMS keys one for source and one for destination.
Now while applying replication configuration, there is an option to pass destination key for destination bucket but I am not sure how to apply key at the source.
Any help would be appreciated.
provider "aws" {
alias = "east"
region = "us-east-1"
}
resource "aws_s3_bucket" "destination-bucket" {
bucket = ""destination-bucket"
provider = "aws.east"
acl = "private"
region = "us-east-1"
versioning {
enabled = true
}
server_side_encryption_configuration {
rule {
apply_server_side_encryption_by_default {
kms_master_key_id = "${var.kms_cmk_dest_arn}"
sse_algorithm = "aws:kms"
}
}
}
}
resource "aws_s3_bucket" "source-bucket" {
bucket = "source-bucket"
acl = "private"
versioning {
enabled = true
}
server_side_encryption_configuration {
rule {
apply_server_side_encryption_by_default {
kms_master_key_id = "${var.kms_cmk_arn}"
sse_algorithm = "aws:kms"
}
}
}
replication_configuration {
role = "${aws_iam_role.replication.arn}"
rules {
status = "Enabled"
destination {
bucket = "${aws_s3_bucket.source-bucket.arn}"
storage_class = "STANDARD"
replica_kms_key_id = "${var.kms_cmk_dest_arn}"
}
source_selection_criteria {
sse_kms_encrypted_objects {
enabled = true
}
}
}
}
}
resource "aws_iam_role" "replication" {
name = "cdd-iam-role-replication"
permissions_boundary = "arn:aws:iam::${var.account_id}:policy/ServiceRoleBoundary"
assume_role_policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
POLICY
}
resource "aws_iam_role_policy" "replication" {
name = "cdd-iam-role-policy-replication"
role = "${aws_iam_role.replication.id}"
policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:GetReplicationConfiguration",
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": [
"${aws_s3_bucket.source-bucket.arn}"
]
},
{
"Action": [
"s3:GetObjectVersion",
"s3:GetObjectVersionAcl"
],
"Effect": "Allow",
"Resource": [
"${aws_s3_bucket.source-bucket.arn}/*"
]
},
{
"Action": [
"s3:ReplicateObject",
"s3:ReplicateDelete"
],
"Effect": "Allow",
"Resource": "${aws_s3_bucket.destination-bucket.arn}/*"
}
]
}
POLICY
}
In case you're using a Customer Managed Key(CMK) for S3 encryption, you need extra configuration.
AWS S3 Documentation mentions that the CMK owner must grant the source bucket owner permission to use the CMK.
https://docs.aws.amazon.com/AmazonS3/latest/dev/replication-config-for-kms-objects.html#replication-kms-cross-acct-scenario
Also, a good article to summarize the S3 cross region replication configuration:
https://medium.com/#devopslearning/100-days-of-devops-day-44-s3-cross-region-replication-crr-8c58ae8c68d4
If I understand you correctly, you've got two S3 Buckets in two different regions within the same account.
One way I've done this in the past is to plan/apply the KMS keys to both regions first.
Then on a separate plan/apply, I used Terraform's data sources:
data "aws_kms_key" "source_credentials_encryption_key" {
key_id = "alias/source-encryption-key"
}
data "aws_kms_key" "destination_credentials_encryption_key" {
provider = aws.usEast
key_id = "alias/destination-encryption-key"
}
And used the data source for the replication configuration like so:
replication_configuration {
role = aws_iam_role.replication_role.arn
rules {
status = "Enabled"
destination {
bucket = aws_s3_bucket.source_bucket.arn
storage_class = "STANDARD"
replicate_kms_key_id = data.aws_kms_key.destination_bucket_encryption_key.arn
}
source_selection_criteria {
sse_kms_encrypted_objects {
enabled = true
}
}
}
}

Interpolate Terraform local variable from resource?

I am trying to create a default bucket policy if none is provided by terraform.tfvars via the var.override_policy variable.
In my below configuration I am attempting to use "${aws_s3_bucket.destination.arn}" in my local block and then use the computed local as my bucket policy. When I change this interpolation to a variable or hardcode it, I am able to get Terraform to plan successfully.
resource "aws_s3_bucket" "destination" {
provider = "aws.dest"
count = "${var.bucket_side == "dest" ? 1 : 0}"
acl = "private"
bucket = "${var.dest_bucket_name}"
website {
index_document = "index.html"
error_document = "error.html"
}
tags = {
Name = "${var.dest_bucket_name}"
}
}
resource "aws_s3_bucket_policy" "destination" {
provider = "aws.dest"
count = "${var.bucket_side == "dest" ? 1 : 0}"
bucket = "${aws_s3_bucket.destination.id}"
policy = "${var.override_policy != "" ? var.override_policy : "${local.default_bucket_policy}"}"
}
locals {
default_bucket_policy = <<EOF
{
"Version": "2008-10-17",
"Id": "${var.dest_bucket_name}-policy",
"Statement": [
{
"Sid": "AllowRead",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": [
"s3:List*",
"s3:Get*"
],
"Resource": [
"${aws_s3_bucket.destination.arn}",
"${aws_s3_bucket.destination.arn}/*"
]
}
]
}
EOF
}
When I try to plan the above version though, I get the following error.
Error: Error running plan: 1 error occurred:
* local.default_bucket_policy: local.default_bucket_policy: Resource 'aws_s3_bucket.dest
ination' not found for variable 'aws_s3_bucket.destination.arn'
Is there something I am missing or is attempting to use interpolation inside of a local just a limitation of Terraform?