I am trying to create a default bucket policy if none is provided by terraform.tfvars via the var.override_policy variable.
In my below configuration I am attempting to use "${aws_s3_bucket.destination.arn}" in my local block and then use the computed local as my bucket policy. When I change this interpolation to a variable or hardcode it, I am able to get Terraform to plan successfully.
resource "aws_s3_bucket" "destination" {
provider = "aws.dest"
count = "${var.bucket_side == "dest" ? 1 : 0}"
acl = "private"
bucket = "${var.dest_bucket_name}"
website {
index_document = "index.html"
error_document = "error.html"
}
tags = {
Name = "${var.dest_bucket_name}"
}
}
resource "aws_s3_bucket_policy" "destination" {
provider = "aws.dest"
count = "${var.bucket_side == "dest" ? 1 : 0}"
bucket = "${aws_s3_bucket.destination.id}"
policy = "${var.override_policy != "" ? var.override_policy : "${local.default_bucket_policy}"}"
}
locals {
default_bucket_policy = <<EOF
{
"Version": "2008-10-17",
"Id": "${var.dest_bucket_name}-policy",
"Statement": [
{
"Sid": "AllowRead",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": [
"s3:List*",
"s3:Get*"
],
"Resource": [
"${aws_s3_bucket.destination.arn}",
"${aws_s3_bucket.destination.arn}/*"
]
}
]
}
EOF
}
When I try to plan the above version though, I get the following error.
Error: Error running plan: 1 error occurred:
* local.default_bucket_policy: local.default_bucket_policy: Resource 'aws_s3_bucket.dest
ination' not found for variable 'aws_s3_bucket.destination.arn'
Is there something I am missing or is attempting to use interpolation inside of a local just a limitation of Terraform?
Related
I want to create a resource policy for a Secrets Manager secret.
I am following the official example on the docs
resource "aws_secretsmanager_secret_policy" "this" {
count = var.create_resource_policy ? 1 : 0
secret_arn = aws_secretsmanager_secret.mysecret.arn
policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EnableAnotherAWSAccountToReadTheSecret",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789012:root"
},
"Action": "secretsmanager:GetSecretValue",
"Resource": "*"
}
]
}
POLICY
}
Is there a way to pass the following as variables in the policy document things as the principal(s), the action the resources etc?
I want to be able to pass those things as terraform variables.
Yes, you can use the built-in functions in terraform for that with interpolation syntax. For example, if you had a data source to get the account ID, you could do the following:
data "aws_caller_identity" "current" {}
resource "aws_secretsmanager_secret_policy" "this" {
count = var.create_resource_policy ? 1 : 0
secret_arn = aws_secretsmanager_secret.mysecret.arn
policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EnableAnotherAWSAccountToReadTheSecret",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${data.aws_caller_identity.current.id}:root"
},
"Action": "secretsmanager:GetSecretValue",
"Resource": "*"
}
]
}
POLICY
}
You could then do the same for any other property you want/need. However, I find it easier to use the built-in data source [1] for creating policies. So for example, the policy you have could be written in the following way:
data "aws_iam_policy_document" "secrets_manager_policy" {
statement {
sid = "EnableAnotherAWSAccountToReadTheSecret"
effect = "Allow"
principals {
identifiers = ["arn:aws:iam::${data.aws_caller_identity.current.id}:root"]
type = "AWS"
}
actions = [
"secretsmanager:GetSecretValue"
]
resources = ["*"]
}
}
You could then tell actions argument to use a variable which would most preferably be a list(string) which would list all the actions necessary:
data "aws_iam_policy_document" "secrets_manager_policy" {
statement {
sid = "EnableAnotherAWSAccountToReadTheSecret"
effect = "Allow"
principals {
identifiers = "arn:aws:iam::${data.aws_caller_identity.current.id}:root"
type = "AWS"
}
actions = var.secrets_actions
resources = [ "*" ]
}
}
Then, you would only have to reference the output of the data source in the original resource:
resource "aws_secretsmanager_secret_policy" "this" {
count = var.create_resource_policy ? 1 : 0
secret_arn = aws_secretsmanager_secret.mysecret.arn
policy = data.aws_iam_policy_document.secrets_manager_policy.json
}
[1] https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document
I am creating policy based on tags. The environment value differs for each resource, so I have to add them as multiple values. Unable to pass as list. Tried join,split and for loop. None of them works. Pls help. Below code just add the value as "beta,test" which will not work as expected
main.tf
locals{
workspaceValues = terraform.workspace == "dev" ? ["alpha", "dev"] : terraform.workspace == "test" ? ["beta", "test"] : ["prod", "staging"]
}
resource "aws_iam_group_policy" "inline_policy" {
name = "${terraform.workspace}_policy"
group = aws_iam_group.backend_admin.name
policy = templatefile("policy.tpl", { env = join(",", local.workspaceValues), region = "${data.aws_region.current.name}", account_id = "${data.aws_caller_identity.current.account_id}" })
}
policy.tpl:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "ec2:*",
"Resource": "*",
"Condition": {
"ForAllValues:StringLikeIfExists": {
"aws:ResourceTag/Environment": "${env}"
}
}
}
]
}
You can use jsonencode to properly format TF list as a list in json in your template:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "ec2:*",
"Resource": "*",
"Condition": {
"ForAllValues:StringLikeIfExists": {
"aws:ResourceTag/Environment": ${jsonencode(env)}
}
}
}
]
}
For that you would call the template as follows:
resource "aws_iam_group_policy" "inline_policy" {
name = "${terraform.workspace}_policy"
group = aws_iam_group.backend_admin.name
policy = templatefile("policy.tpl", {
env = local.workspaceValues
})
}
You are not using region nor account_id in your template, so there is no reason to pass them in.
An alternative solution to avoid this all together is to recreate the policy using the data component "aws_iam_policy_document" and then pass it to the aws_iam_policy resource like the following,
data "aws_iam_policy_document" "example" {
statement {
effect = "Allow"
actions = [
"ec2:*"
]
resources = [
"*"
]
condition {
test = "ForAllValues:StringLikeIfExists"
variable = "aws:ResourceTag/Environment"
values = local.workspaceValues
}
}
}
resource "aws_iam_policy" "example" {
name = "example_policy"
path = "/"
policy = data.aws_iam_policy_document.example.json
}
I am having issues with my s3 bucket policy, it seems to add the policy correctly and even verified it in AWS and it shows the exact policy set in the policy.tpl but it keeps saying there are changes
I've tried changing the action and resource into arrays which I've heard may help.. tried removing the "Version" from the policy, the SID, keeps saying there are changes everytime i run it
policy.tf
resource "aws_s3_bucket_policy" "bucket" {
bucket = aws_s3_bucket.bucket.id
policy = local.policy
}
locals.tf
locals {
template_dir = "${path.module}/templates"
template_vars = {
encrypt = var.s3_require_encryption_enabled
bucket_arn = aws_s3_bucket.bucket.arn
extra_statements = var.s3_bucket_policy
}
policy = templatefile("${local.template_dir}/policy.tpl", local.template_vars)
}
templates/policy.tpl
{
"Version": "2008-10-17",
"Statement": [
{
"Sid" : "",
"Effect" : "Deny",
"Principal" : "*",
"Action" : "s3:*",
"Resource" : "${bucket_arn}/*",
"Condition": {
"Bool": {
"aws:SecureTransport": "false"
}
}
}
]
}
in AWS
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": "arn:aws:s3:::test-bucket-us-east-1/*",
"Condition": {
"Bool": {
"aws:SecureTransport": "false"
}
}
}
]
}
says
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
~ update in-place
Terraform will perform the following actions:
#aws_s3_bucket_policy.bucket will be updated in-place
~ resource "aws_s3_bucket_policy" "bucket" {
bucket = "test-bucket-us-east-1"
id = "test-bucket-us-east-1"
+ policy = jsonencode(
{
+ Statement = [
+ {
+ Action = "s3:*"
+ Condition = {
+ Bool = {
+ aws:SecureTransport = "false"
}
}
+ Effect = "Deny"
+ Principal = "*"
+ Resource = "arn:aws:s3:::test-bucket-us-east-1/*"
+ Sid = ""
},
]
+ Version = "2008-10-17"
}
)
}
Plan: 0 to add, 1 to change, 0 to destroy.
Based on the comments, the underlying bucket policy had issues.
PutBucketPolicy has
Content-MD5
The MD5 hash of the request body.
For requests made using the AWS Command Line Interface (CLI) or AWS SDKs, this field is calculated automatically.)
So resource aws_s3_bucket_policy was trying to update the policy.
I am creating Elasticsearch and I am using count variable with value of 0 or 1.
My code:
resource "aws_elasticsearch_domain" "es" {
count = "${var.enable_pipeline_1 ? 1 : 0}"
domain_name = "${var.name_prefix}"
elasticsearch_version = "6.8"
cluster_config {
instance_type = "t2.small.elasticsearch"
}
vpc_options {
subnet_ids = [
"${aws_subnet.selected.id}",
]
security_group_ids = ["${aws_security_group.security_group.id}"]
}
advanced_options = {
"rest.action.multi.allow_explicit_index" = "true"
}
access_policies = <<CONFIG
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "es:*",
"Principal": "*",
"Effect": "Allow",
"Resource": "arn:aws:es:${var.aws_region}:${var.aws_account_id}:domain/${var.name_prefix}/*"
}
]
}
CONFIG
snapshot_options {
automated_snapshot_start_hour = 23
}
tags = {
Domain = "ES-${var.name_prefix}"
}
ebs_options {
ebs_enabled = true
volume_type = "standard"
volume_size = "${var.elasticserch_disk_size}"
}
log_publishing_options {
cloudwatch_log_group_arn = "${aws_cloudwatch_log_group.es_log_resource_policy.arn}"
log_type = "INDEX_SLOW_LOGS"
}
depends_on = [
"aws_iam_service_linked_role.es",
]
}
resource "aws_cloudwatch_log_group" "es_log_resource_policy" {
count = "${var.enable_pipeline_1 ? 1 : 0}"
name = "${var.name_prefix}_es_log_group"
}
resource "aws_cloudwatch_log_resource_policy" "es_log_resource_policy" {
count = "${var.enable_pipeline_1 ? 1 : 0}"
policy_name = "${var.name_prefix}_es_log_resource_policy"
policy_document = <<CONFIG
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "es.amazonaws.com"
},
"Action": [
"logs:PutLogEvents",
"logs:PutLogEventsBatch",
"logs:CreateLogStream"
],
"Resource": "arn:aws:logs:*"
}
]
}
CONFIG
}
resource "aws_iam_service_linked_role" "es" {
count = "${var.enable_pipeline_1 ? 1 : 0}"
aws_service_name = "es.amazonaws.com"
}
So I have 2 problems now:
1st problem: If I set count value to zero then the elasticsearch domain should delete if exist but it is not deleting.
2nd problem: So I deleted domain manually on aws console and now I want to create domain but error said:
aws_iam_service_linked_role.es: Error creating service-linked role with name es.amazonaws.com: InvalidInput: Service role name AWSServiceRoleForAmazonElasticsearchService has been taken in this account, please try a different suffix.
I don't understand what is going wrong, any help please?
Thank you.
When creating AWS Elasticsearch domain AWS will automatically create a Service Role for you. It is needed so AWS systems can manage domains that are located in VPCs that are located in your account.
To solve this issue you need to manually delete Service Role as described in documentation. You also need to remember that this role can be created only once per AWS account so if you want to create another domain with this exact code you need to delete aws_iam_service_linked_role resource as this role will already exist it the account. This role is also automatically created when you spin a new domain with AWS Console.
We solved this by extracting aws_iam_service_linked_role into genric config directory that is deployed only once per account.
I was using Terraform to setup S3 buckets (different region) and set up replication between them.
It was working properly until I added KMS in it.
I created 2 KMS keys one for source and one for destination.
Now while applying replication configuration, there is an option to pass destination key for destination bucket but I am not sure how to apply key at the source.
Any help would be appreciated.
provider "aws" {
alias = "east"
region = "us-east-1"
}
resource "aws_s3_bucket" "destination-bucket" {
bucket = ""destination-bucket"
provider = "aws.east"
acl = "private"
region = "us-east-1"
versioning {
enabled = true
}
server_side_encryption_configuration {
rule {
apply_server_side_encryption_by_default {
kms_master_key_id = "${var.kms_cmk_dest_arn}"
sse_algorithm = "aws:kms"
}
}
}
}
resource "aws_s3_bucket" "source-bucket" {
bucket = "source-bucket"
acl = "private"
versioning {
enabled = true
}
server_side_encryption_configuration {
rule {
apply_server_side_encryption_by_default {
kms_master_key_id = "${var.kms_cmk_arn}"
sse_algorithm = "aws:kms"
}
}
}
replication_configuration {
role = "${aws_iam_role.replication.arn}"
rules {
status = "Enabled"
destination {
bucket = "${aws_s3_bucket.source-bucket.arn}"
storage_class = "STANDARD"
replica_kms_key_id = "${var.kms_cmk_dest_arn}"
}
source_selection_criteria {
sse_kms_encrypted_objects {
enabled = true
}
}
}
}
}
resource "aws_iam_role" "replication" {
name = "cdd-iam-role-replication"
permissions_boundary = "arn:aws:iam::${var.account_id}:policy/ServiceRoleBoundary"
assume_role_policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
POLICY
}
resource "aws_iam_role_policy" "replication" {
name = "cdd-iam-role-policy-replication"
role = "${aws_iam_role.replication.id}"
policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:GetReplicationConfiguration",
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": [
"${aws_s3_bucket.source-bucket.arn}"
]
},
{
"Action": [
"s3:GetObjectVersion",
"s3:GetObjectVersionAcl"
],
"Effect": "Allow",
"Resource": [
"${aws_s3_bucket.source-bucket.arn}/*"
]
},
{
"Action": [
"s3:ReplicateObject",
"s3:ReplicateDelete"
],
"Effect": "Allow",
"Resource": "${aws_s3_bucket.destination-bucket.arn}/*"
}
]
}
POLICY
}
In case you're using a Customer Managed Key(CMK) for S3 encryption, you need extra configuration.
AWS S3 Documentation mentions that the CMK owner must grant the source bucket owner permission to use the CMK.
https://docs.aws.amazon.com/AmazonS3/latest/dev/replication-config-for-kms-objects.html#replication-kms-cross-acct-scenario
Also, a good article to summarize the S3 cross region replication configuration:
https://medium.com/#devopslearning/100-days-of-devops-day-44-s3-cross-region-replication-crr-8c58ae8c68d4
If I understand you correctly, you've got two S3 Buckets in two different regions within the same account.
One way I've done this in the past is to plan/apply the KMS keys to both regions first.
Then on a separate plan/apply, I used Terraform's data sources:
data "aws_kms_key" "source_credentials_encryption_key" {
key_id = "alias/source-encryption-key"
}
data "aws_kms_key" "destination_credentials_encryption_key" {
provider = aws.usEast
key_id = "alias/destination-encryption-key"
}
And used the data source for the replication configuration like so:
replication_configuration {
role = aws_iam_role.replication_role.arn
rules {
status = "Enabled"
destination {
bucket = aws_s3_bucket.source_bucket.arn
storage_class = "STANDARD"
replicate_kms_key_id = data.aws_kms_key.destination_bucket_encryption_key.arn
}
source_selection_criteria {
sse_kms_encrypted_objects {
enabled = true
}
}
}
}