Granting cloudtrails access to kms keys in terraform - amazon-web-services

My first attempt at doing this resulted in a circular reference:
resource "aws_kms_key" "cloudtrails-key" {
description = "KMS Master Key for trails logs"
key_usage = "ENCRYPT_DECRYPT"
customer_master_key_spec = "SYMMETRIC_DEFAULT"
deletion_window_in_days = 30
is_enabled = true
enable_key_rotation = false
policy = data.aws_iam_policy_document.cloudtrails-key-policy.json
multi_region = true
tags = {
ManagedBy = "terraform"
Name = "${var.project_name}-${var.environment}-${var.aws_account_id}-cloudtrails-key"
Environment = var.environment
Usage = "monitoring"
}
}
data "aws_iam_policy_document" "cloudtrails-key-policy" {
statement {
principals {
type = "Service"
identifiers = [ "cloudtrail.amazonaws.com" ]
}
actions = [
"kms:CallerAccount",
"kms:EncryptionAlgorithm",
"kms:EncryptionContext:*",
"kms:EncryptionContextKeys",
"kms:RequestAlias",
"kms:ViaService"
]
resources = [ aws_kms_key.cloudtrails-key.arn ]
condition {
test = "ForAnyValue:StringLike"
variable = "aws:SourceArn"
values = toset([for k, v in var.all_account_ids : "arn:aws:cloudtrail:*:${v}:trail/full-cloudtrails"])
}
}
}
As I couldn't determine how to remove the circular reference, I decided to try using grants instead:
resource "aws_kms_grant" "cloudtrails-key-grant" {
depends_on = [ aws_kms_key.cloudtrails-key ]
for_each = var.all_account_ids
name = "${each.key}-cloudtrails-key-grant"
key_id = aws_kms_key.cloudtrails-key.key_id
grantee_principal = "arn:aws:cloudtrail::${each.value}:trail/full-cloudtrails"
operations = [ "Encrypt" ]
}
var.all_account_ids refers to a map of account names and account IDs.
I then updated the aws_kms_key block to remove the policy argument. However, this resulted instead in the following output on apply:
Terraform v1.3.8
on linux_amd64
aws_kms_grant.cloudtrails-key-grant["users"]: Creating...
aws_kms_grant.cloudtrails-key-grant["root"]: Creating...
aws_kms_grant.cloudtrails-key-grant["dev"]: Creating...
aws_kms_grant.cloudtrails-key-grant["shared"]: Creating...
aws_kms_grant.cloudtrails-key-grant["monitoring"]: Creating...
aws_cloudtrail.full-cloudtrails: Modifying... [id=full-cloudtrails]
...
aws_kms_grant.cloudtrails-key-grant["users"]: Still creating... [2m50s elapsed]
aws_kms_grant.cloudtrails-key-grant["monitoring"]: Still creating... [2m50s elapsed]
╷
│ Error: updating CloudTrail Trail (full-cloudtrails): InsufficientEncryptionPolicyException: Insufficient permissions to access S3 bucket full-cloudtrails or KMS key arn:aws:kms:us-east-1:703617067304:key/mrk-c0a4616ba4f74ae9b5ac4ecd1f96e95b.
│
│ with aws_cloudtrail.full-cloudtrails,
│ on monitoring.tf line 1, in resource "aws_cloudtrail" "full-cloudtrails":
│ 1: resource "aws_cloudtrail" "full-cloudtrails" {
│
╵
╷
│ Error: creating KMS Grant for Key (mrk-c0a4616ba4f74ae9b5ac4ecd1f96e95b): InvalidArnException: ARN is not valid: arn:aws:cloudtrail:us-east-1:323924548037:trail/full-cloudtrails
│
│ with aws_kms_grant.cloudtrails-key-grant["root"],
│ on security.tf line 19, in resource "aws_kms_grant" "cloudtrails-key-grant":
│ 19: resource "aws_kms_grant" "cloudtrails-key-grant" {
│
╵
╷
│ Error: creating KMS Grant for Key (mrk-c0a4616ba4f74ae9b5ac4ecd1f96e95b): InvalidArnException: ARN is not valid: arn:aws:cloudtrail:us-east-1:739800733271:trail/full-cloudtrails
│
│ with aws_kms_grant.cloudtrails-key-grant["shared"],
│ on security.tf line 19, in resource "aws_kms_grant" "cloudtrails-key-grant":
│ 19: resource "aws_kms_grant" "cloudtrails-key-grant" {
│
╵
╷
│ Error: creating KMS Grant for Key (mrk-c0a4616ba4f74ae9b5ac4ecd1f96e95b): InvalidArnException: ARN is not valid: arn:aws:cloudtrail:us-east-1:703617067304:trail/full-cloudtrails
│
│ with aws_kms_grant.cloudtrails-key-grant["monitoring"],
│ on security.tf line 19, in resource "aws_kms_grant" "cloudtrails-key-grant":
│ 19: resource "aws_kms_grant" "cloudtrails-key-grant" {
│
╵
╷
│ Error: creating KMS Grant for Key (mrk-c0a4616ba4f74ae9b5ac4ecd1f96e95b): InvalidArnException: ARN is not valid: arn:aws:cloudtrail:us-east-1:457407591303:trail/full-cloudtrails
│
│ with aws_kms_grant.cloudtrails-key-grant["dev"],
│ on security.tf line 19, in resource "aws_kms_grant" "cloudtrails-key-grant":
│ 19: resource "aws_kms_grant" "cloudtrails-key-grant" {
│
╵
╷
│ Error: creating KMS Grant for Key (mrk-c0a4616ba4f74ae9b5ac4ecd1f96e95b): InvalidArnException: ARN is not valid: arn:aws:cloudtrail:us-east-1:158889104371:trail/full-cloudtrails
│
│ with aws_kms_grant.cloudtrails-key-grant["users"],
│ on security.tf line 19, in resource "aws_kms_grant" "cloudtrails-key-grant":
│ 19: resource "aws_kms_grant" "cloudtrails-key-grant" {
│
╵
I tried adding the region (though I would prefer multi-region), taking off the trail name after the slash, using wildcards for region, but everything resulted in the above invalid ARN exception.
I was curious to know if anyone has any working recipes for granting cloudtrails encryption access to a KMS key, as I'm unsure what I'm doing wrong with either method (policy or grant)

The solution for the cyclical reference was to broaden the resource scope to all keys, and not just the one key. This was done on all examples in AWS documentation. I also had to add the default key policy offered by AWS otherwise I got a warning that I would be locking myself out.
The Default Key Policy can be found documented here. The specific key policy required for CloudTrail is found here. Using these two resources, the final working configuration is below:
data "aws_iam_policy_document" "cloudtrail-key-policy" {
statement {
principals {
type = "Service"
identifiers = [ "cloudtrail.amazonaws.com" ]
}
actions = [
"kms:DescribeKey",
"kms:GenerateDataKey*"
]
resources = [ "*" ]
condition {
test = "ForAnyValue:StringLike"
variable = "kms:EncryptionContext:aws:cloudtrail:arn"
values = toset([for k, v in var.all_account_ids : "arn:aws:cloudtrail:*:${v}:trail/${k}-full-cloudtrail"])
}
condition {
test = "ForAnyValue:StringLike"
variable = "aws:SourceArn"
values = toset([for k, v in var.all_account_ids : "arn:aws:cloudtrail:*:${v}:trail/${k}-full-cloudtrail"])
}
}
statement {
principals {
type = "AWS"
identifiers = [ "arn:aws:iam::${var.aws_account_id}:role/terraform-${var.environment}" ]
}
actions = [
"kms:Create*",
"kms:Describe*",
"kms:Enable*",
"kms:List*",
"kms:Put*",
"kms:Update*",
"kms:Revoke*",
"kms:Disable*",
"kms:Get*",
"kms:Delete*",
"kms:TagResource",
"kms:UntagResource",
"kms:ScheduleKeyDeletion",
"kms:CancelKeyDeletion"
]
resources = [ "*" ]
}
statement {
principals {
type = "AWS"
identifiers = [ "arn:aws:iam::${var.aws_account_id}:root" ]
}
actions = [ "kms:*" ]
resources = [ "*" ]
}
}
Additionally, no grants were needed as we're doing this for a service. According to the documentation, service ARNs are excluded:
The grantee principal can be any AWS principal, including an AWS
account (root), an IAM user, an IAM role, a federated role or user, or
an assumed role user. The grantee principal can be in the same account
as the KMS key or a different account. However, the grantee principal
cannot be a service principal, an IAM group, or an AWS organization
The above text can be found here
Hopefully this helps future readers with the same issue.
EDIT: Updated with final working config

Related

How to attach iam policy to Lambda function for SNS publish access?

I am trying to allow my Lambda function permissions so that it can publish an SNS message.
I am creating my Lambda in Terraform:
resource "aws_lambda_function" "DynamoDB_SNS_Lambda" {
s3_bucket = var.bucket_id
s3_key = "s3key.zip"
handler = "index.handler"
runtime = "nodejs16.x"
role = aws_iam_role.lambda_iam_role.arn
function_name = "lambda_name"
}
I am then creating a new IAM role:
resource "aws_iam_role" "lambda_iam_role" {
name = "sns_lambda_iam_role"
assume_role_policy = data.aws_iam_policy_document.sns_lambda_policy.json
}
data "aws_iam_policy_document" "sns_lambda_policy" {
statement {
sid = ""
effect = "Allow"
actions = [
"sns:Publish",
]
resources = [
var.order_update_sns_topic_arn
]
}
}
I am then trying to attach a policy to that IAM role to allow SNS permissions.
resource "aws_iam_role_policy_attachment" "lambda_basic_execution" {
role = aws_iam_role.lambda_iam_role.name
policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonSNSRole"
}
When I run terraform apply, I get the following error:
module.Lambda.aws_iam_role.lambda_iam_role: Modifying... [id=sns_lambda_iam_role]
╷
│ Error: error updating IAM Role (sns_lambda_iam_role) assume role policy: MalformedPolicyDocument: Has prohibited field Resource
│ status code: 400, request id: 98a57a4c-1839-4a04-b903-e7deb409c71e
│
│ with module.Lambda.aws_iam_role.lambda_iam_role,
│ on modules/lambda/main.tf line 17, in resource "aws_iam_role" "lambda_iam_role":
│ 17: resource "aws_iam_role" "lambda_iam_role" {
Am I creating my aws_iam_policy_document correctly?
Am I correct in thinking I need to attach the policy to the role with aws_iam_role_policy_attachment?
This is incorrect assume policy. Assume policy tells only who/what can assume the role, not what permissions the role has. In your case, only lambda can assume the role. And the actual permissions can be set using inline_policy. For example:
resource "aws_iam_role" "lambda_iam_role" {
name = "sns_lambda_iam_role"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = "sts:AssumeRole"
Effect = "Allow"
Sid = ""
Principal = {
Service = "lambda.amazonaws.com"
}
},
]
})
inline_policy {
name = "policy-8675309"
policy = data.aws_iam_policy_document.sns_lambda_policy.json
}
}

Error in Cloud9 for terraform - InvalidClientTokenId: The security token included in the request is invalid

I am trying to create a role in Cloud9 with Terraform.
resource "aws_iam_role" "firehose_role" {
name = "firehose_test_role"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "firehose.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
Error:
failed creating IAM Role (firehose_test_role): InvalidClientTokenId: The security token included in the request is invalid │ status code: 403, request id: 6ea2ca59-70ac-47b8-b83c-6547ad98aec1
And when I disable the "AWS managed temporary credentials", this is what I get:
Error: Invalid provider configuration
│
│ Provider "registry.terraform.io/hashicorp/aws" requires explicit configuration. Add a provider block to the root module and configure the provider's required arguments as described
│ in the provider documentation.
│
╵
╷
│ Error: error configuring Terraform AWS Provider: no valid credential sources for Terraform AWS Provider found.
│
│ Please see https://registry.terraform.io/providers/hashicorp/aws
│ for more information about providing credentials.
│
│ Error: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, http response error StatusCode: 404, request to EC2 IMDS failed
│
│
│ with provider["registry.terraform.io/hashicorp/aws"],
│ on <empty> line 0:
│ (source code not available)
So, what can I do here to make it work on Cloud9?

how to identify an element in a collection value

I am getting this error messages when i run terragrunt apply.I need to be able to specify each element when creating SQS and DeadLetterQueue with the policy.I am unsure how to fix this issue.I want to find a way to reference individual DLQ directly with corresponding SQS to create the required resources
resource "aws_sqs_queue_policy" "Cloudtrail_SQS_Policy" {
for_each = toset(var.sqs_queue_names)
queue_url = aws_sqs_queue.CloudTrail_SQS[each.key].id
policy = <<POLICY
{
"Version": "2012-10-17",
"Id": "sqspolicy",
"Statement": [
{
"Sid": "AllowSQSInvocation",
"Effect": "Allow",
"Principal": {"AWS":"*"},
"Action": "sqs:*",
"Resource": "${aws_sqs_queue.CloudTrail_SQS[each.key].arn}"
resource "aws_sqs_queue_policy" "CloudTrail_SQS_DLQ"{
for_each = toset(var.dead_queue_names)
queue_url = aws_sqs_queue.CloudTrail_SQS_DLQ[each.key].id
policy = <<POLICY
{
"Version": "2012-10-17",
"Id": "sqspolicy",
"Statement": [
{
"Sid": "DLQ Policy",
"Effect": "Allow",
"Principal": {"AWS":"*"},
"Action": "sqs:*",
"Resource": "${aws_sqs_queue.CloudTrail_SQS_DLQ[each.key].arn}
Error Messages:
Error: Invalid index
│
│ on iam.tf line 3, in resource "aws_sqs_queue_policy" "Cloudtrail_SQS_Policy":
│ 3: queue_url = aws_sqs_queue.CloudTrail_SQS[each.key].id
│ ├────────────────
│ │ aws_sqs_queue.CloudTrail_SQS is object with 2 attributes
│ │ each.key is "CloudTrail_SQS_Management_Event"
│
│ The given key does not identify an element in this collection value.
╵
╷
│ Error: Invalid index
│
│ on iam.tf line 15, in resource "aws_sqs_queue_policy" "Cloudtrail_SQS_Policy":
│ 15: "Resource": "${aws_sqs_queue.CloudTrail_SQS[each.key].arn}",
│ ├────────────────
│ │ aws_sqs_queue.CloudTrail_SQS is object with 2 attributes
│ │ each.key is "CloudTrail_SQS_Data_Event"
│
│ The given key does not identify an element in this collection value.
╵
╷
│ Error: Invalid index
│
│ on iam.tf line 15, in resource "aws_sqs_queue_policy" "Cloudtrail_SQS_Policy":
│ 15: "Resource": "${aws_sqs_queue.CloudTrail_SQS[each.key].arn}",
│ ├────────────────
│ │ aws_sqs_queue.CloudTrail_SQS is object with 2 attributes
│ │ each.key is "CloudTrail_SQS_Management_Event"
│
│ The given key does not identify an element in this collection value.
I refactored your entire code, as any changes will cascade to more errors. In your case, the best way to deal with your requirement is to use map, not list or set. You don't have to make your input variables a map if you don't want, though it would be easier. But keeping your input variables as they are, you can create local.sqs_dlq_map and use that:
locals {
sqs_dlq_map = zipmap(var.sqs_queue_names, var.dead_queue_names)
}
resource "aws_sqs_queue" "CloudTrail_SQS"{
for_each = local.sqs_dlq_map
name = each.key
redrive_policy = jsonencode({
deadLetterTargetArn = aws_sqs_queue.CloudTrail_SQS_DLQ[each.key].arn
maxReceiveCount = 2 #var.max_receive_count
})
#tags = var.default_tags
}
resource "aws_sqs_queue" "CloudTrail_SQS_DLQ"{
for_each = local.sqs_dlq_map
name = each.value
#tags = var.default_tags
}
resource "aws_sqs_queue_policy" "Cloudtrail_SQS_Policy" {
for_each = local.sqs_dlq_map
queue_url = aws_sqs_queue.CloudTrail_SQS[each.key].id
policy = <<POLICY
{
"Version": "2012-10-17",
"Id": "sqspolicy",
"Statement": [
{
"Sid": "AllowSQSInvocation",
"Effect": "Allow",
"Principal": {"AWS":"*"},
"Action": "sqs:*",
"Resource": "${aws_sqs_queue.CloudTrail_SQS[each.key].arn}"
}]
}
POLICY
}
resource "aws_sqs_queue_policy" "CloudTrail_SQS_DLQ"{
for_each = local.sqs_dlq_map
queue_url = aws_sqs_queue.CloudTrail_SQS_DLQ[each.key].id
policy = <<POLICY
{
"Version": "2012-10-17",
"Id": "sqspolicy",
"Statement": [
{
"Sid": "DLQ Policy",
"Effect": "Allow",
"Principal": {"AWS":"*"},
"Action": "sqs:*",
"Resource": "${aws_sqs_queue.CloudTrail_SQS_DLQ[each.key].arn}"
}]
}
POLICY
}

Terraform assume role despite correct IAM setup

Ive been able to test this role by assuming it using the CLI with the given the profile. However, I still get this error on apply:
│ Error: error configuring Terraform AWS Provider: IAM Role (arn:aws:iam::<omitted>:role/<omitted>) cannot be assumed.
│
│ There are a number of possible causes of this - the most common are:
│ * The credentials used in order to assume the role are invalid
│ * The credentials do not have appropriate permission to assume the role
│ * The role ARN is not valid
│
│ Error: NoCredentialProviders: no valid providers in chain. Deprecated.
│ For verbose messaging see aws.Config.CredentialsChainVerboseErrors
│
│
│ with provider["registry.terraform.io/hashicorp/aws"],
│ on providers.tf line 3, in provider "aws":
│ 3: provider "aws" {
│
The provider configuration looks like so:
provider "aws" {
region = var.aws_region
profile = var.aws_profile
assume_role {
role_arn = var.assume_role_arn
session_name = "assume-role-${timestamp()}"
}
}
Assuming all variables are correct - what am I doing wrong?
I was able to re-create this in a simple repositiory:
https://github.com/SparkPost/tf-recreate-assume-role-bug
You'll obviously have to create a correct assume role setup with permissions both in the assume-role policy of the role and the policy of the user/role doing the assuming.
Edit:
It's been requested, so here's the trust policy of the assumed role:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::<omitted>:user/<my username>"
]
},
"Action": "sts:AssumeRole"
}
]
}
Additionally, it should be noted that I am an admin user with permission to assume all roles.
Here's the command I used to test the assume role from the CLI:
aws sts --profile <profile used in module> assume-role --role-arn <role arn from error message> --role-session-name test
This returned successfully.
Your timestamp() will return illegal characters for session name. You have to format it to have only good characters, e.g.:
session_name = "assume-role-${formatdate("MMM-DD-YYYY", timestamp())}"

How to perform terraform provisioning for other accounts using assumed role?

I want to provision terraform instance in Account A and from account A to account B.
1)Created a role in account A (arn - arn:aws:iam::XXXXXXX:role/stg-tf-devops-ec2_role to assume role in account B and also have fullEC2 access to provision instance in account A
2)Created a role in account B (arn - arn:aws:iam::YYYYYYY:role/TF_Automation_DevOps) and added ec2 privilege in that role and edited the trust policy and added the role arn created in account A
In account A - I have an instance which i'm using for terraform execution is attached to that stg-tf-devops-ec2_role
Now from my understanding this instance should take care of provisioning instances in both account A and account B without access and secret key.But unfortunately i'm getting below error
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
}
}
}
provider "aws" {
region = var.region
}
provider "aws" {
region = var.region
assume_role {
role_arn = "arn:aws:iam::YYYYYYYYY:role/TF_Automation_DevOps"
}
alias = "dim"
}
Error message when executing terraform plan is as below
terraform plan
╷
│ Error: error configuring Terraform AWS Provider: IAM Role (arn:aws:iam::YYYYYYYYYY:role/TF_Automation_DevOps) cannot be assumed.
│
│ There are a number of possible causes of this - the most common are:
│ * The credentials used in order to assume the role are invalid
│ * The credentials do not have appropriate permission to assume the role
│ * The role ARN is not valid
│
│ Error: NoCredentialProviders: no valid providers in chain. Deprecated.
│ For verbose messaging see aws.Config.CredentialsChainVerboseErrors
│
│
│ with provider["registry.terraform.io/hashicorp/aws"].dim,
│ on main.tf line 17, in provider "aws":
│ 17: provider "aws" {
Could you please advice how to proceed further or what went wrong and how to debug further?
As requested in comments,here i'm able to use aws cli without access key and secret key and fetch details from account A
aws ec2 describe-instances --region eu-west-2|head -10
{
"Reservations": [
{
"Groups": [],
"Instances": [
{
"AmiLaunchIndex": 0,
"ImageId": "ami-0aac9d7fa83beb6d2",
"InstanceId": "i-0d13491bfd0af559e",
"InstanceType": "t3.medium",
Got the below error when checking whether able to assume the role on account b
$ aws sts assume-role --role-arn "arn:aws:iam::YYYYYYYYYYY:role/TF_Automation_DevOp" --role-session-name "access_chk"
An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:sts::YYYYYYYYYYY:assumed-role/stg-tf-devops-ec2_role/i-066506ec0993d0148 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::YYYYYYYYYYY:role/TF_Automation_DevOp
Trusted policy in account B as below
{​​​​​​​
"Version": "2012-10-17",
"Statement": [
{​​​​​​​
"Effect": "Allow",
"Principal": {​​​​​​​
"AWS": "arn:aws:iam::XXXXXXXX:role/stg-tf-devops-ec2_role"
}​​​​​​​,
"Action": "sts:AssumeRole"
}​​​​​​​
]
}​​​​​​​
Role on account A is having below privileges