Terraform Failure configuring LB attributes - amazon-web-services

I've followed the first answer on this post on StackOverflow but I obtain this error:
Failure configuring LB attributes: InvalidConfigurationRequest: Access Denied for bucket: myproject-log. Please check S3bucket permission status code: 400
This is my code:
s3_bucket
data "aws_elb_service_account" "main" {}
resource "aws_s3_bucket" "bucket_log" {
bucket = "${var.project}-log"
acl = "log-delivery-write"
policy = <<POLICY
{
"Id": "Policy",
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:PutObject"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::${var.project}-log/AWSLogs/*",
"Principal": {
"AWS": [
"${data.aws_elb_service_account.main.arn}"
]
}
}
]
}
POLICY
}
load balancer
resource "aws_lb" "vm_stage" {
name = "${var.project}-lb-stg"
internal = false
load_balancer_type = "application"
subnets = [aws_subnet.subnet_1.id, aws_subnet.subnet_2.id, aws_subnet.subnet_3.id]
security_groups = [aws_security_group.elb_project_stg.id]
access_logs {
bucket = aws_s3_bucket.bucket_log.id
prefix = "lb-stg"
enabled = true
}
tags = {
Name = "${var.project}-lb-stg"
}
}

Just going to drop this here since this cross applied to another question that was asked.
This took me awhile to figure out, but the S3 bucket has two requirements per the documentation:
The bucket must be located in the same Region as the load balancer.
Amazon S3-Managed Encryption Keys (SSE-S3) is required. No other encryption options are supported.
Source: https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/enable-access-logs.html
While it makes it seem like it's a permissions issue with the error message given it may actually be an issue with the bucket having the wrong encryption type. In my case the issue was that my bucket was unencrypted.
Updated the bucket to SSE-S3 encryption and I no longer received the error:
resource "aws_s3_bucket" "s3_access_logs_bucket" {
bucket = var.access_logs_bucket_name
acl = "private"
server_side_encryption_configuration {
rule {
apply_server_side_encryption_by_default {
sse_algorithm = "AES256"
}
}
}
versioning {
enabled = true
}
}
And just because, here's the policy I used:
data "aws_elb_service_account" "main" {}
data "aws_iam_policy_document" "s3_lb_write" {
statement {
principals {
identifiers = ["${data.aws_elb_service_account.main.arn}"]
type = "AWS"
}
actions = ["s3:PutObject"]
resources = [
"${aws_s3_bucket.s3_access_logs_bucket.arn}/*"
]
}
}
resource "aws_s3_bucket_policy" "load_balancer_access_logs_bucket_policy" {
bucket = aws_s3_bucket.s3_access_logs_bucket.id
policy = data.aws_iam_policy_document.s3_lb_write.json
}

Official AWS Docs
https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/enable-access-logs.html
Solution
Reference the docs above and change your bucket's iam policy to reflect what the documentation states. The logging is actually done by AWS and not your roles or IAM users. So you need to give ÅWS permission to do this. That's why the docs show statements in the policy that specify the delivery.logs.amazonaws.com principal. That principal is the AWS logging service. Even though your bucket is hosted on AWS, they don't give themselves access to your bucket by default. You have to explicitly grant access to AWS if you want their services to work.

As per this post, I was able to resolve this issue by disabling KMS and using SSE-S3 for bucket encryption. Also, there are additional permissions listed in the AWS docs.

I struggled with this as well the entire terraform bucket policy that worked for me is below.
data "aws_iam_policy_document" "elb_bucket_policy" {
statement {
effect = "Allow"
resources = [
"arn:aws:s3:::unique-bucket-name/${local.prefix}/AWSLogs/${local.application_account_id}/*",
]
actions = ["s3:PutObject"]
principals {
type = "AWS"
identifiers = ["arn:aws:iam::${local.elb_account_id}:root"]
}
}
statement {
effect = "Allow"
resources = [
"arn:aws:s3:::unique-bucket-name/${local.prefix}/AWSLogs/${local.application_account_id}/*",
]
actions = ["s3:PutObject"]
principals {
type = "Service"
identifiers = ["logdelivery.elb.amazonaws.com"]
}
}
statement {
effect = "Allow"
resources = [
"arn:aws:s3:::unique-bucket-name/${local.prefix}/AWSLogs/${local.application_account_id}/*",
]
actions = ["s3:PutObject"]
principals {
type = "Service"
identifiers = ["logdelivery.elb.amazonaws.com"]
}
condition {
test = "StringEquals"
variable = "s3:x-amz-acl"
values = ["bucket-owner-full-control"]
}
}
}

Related

Policy document should not specify a principal - terraform aws_iam_policy_document

I have a an IAM policy which I have created and it seems to keep complaining that the policy document should not specify a principal. Am really unsure of what is wrong with my policy. This policy will be attached to my S3 Bucket which specifies only a certain group is allowed to do the following actions GetObject and ListBucket.
Error : MalformedPolicyDocument: Policy document should not specify a principal
My IAM Policy is as follows :
data "aws_iam_policy_document" "s3_admin_access" {
statement {
sid = "AllowGroupAAccess"
effect = "Allow"
actions = [
"s3:GetObject",
"s3:ListBucket"
]
resources = local.s3_etl_bucket_array
principals {
type = "AWS"
identifiers = [aws_iam_group.iam_group_team["admin-team"].arn]
}
}
statement {
sid = "DenyAllOtherUsers"
effect = "Deny"
actions = [
"s3:*"
]
resources = local.s3_etl_bucket_array
principals {
type = "AWS"
identifiers = ["*"]
}
condition {
test = "StringNotEquals"
variable = "aws:PrincipalArn"
values = [aws_iam_group.iam_group_team["admin-team"].arn]
}
}
}
resource "aws_iam_policy" "s3_admin_access" {
name = "${local.csi}-s3_admin_access"
path = "/"
policy = data.aws_iam_policy_document.s3_admin_access.json
}
The short answer is that groups cannot be used as a principal in a resource policy and the bucket policy is a type of resource policy [1]:
You cannot identify a user group as a principal in a policy (such as a resource-based policy) because groups relate to permissions, not authentication, and principals are authenticated IAM entities.
[1] https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_principal.html#Principal_specifying

Create S3 bucket and lambda policies with terraform

I have some trouble accessing my AWS bucket from a lambda. I create and configure my bucket/lambdas using terraform (terraform newbie here).
Here is the module that creates the S3 bucket :
module "create-my-bucket" {
source = "terraform-aws-modules/s3-bucket/aws"
bucket = "my-bucket"
acl = "private"
versioning = {
enabled = true
}
block_public_acls = true
block_public_policy = true
restrict_public_buckets = true
ignore_public_acls = true
attach_deny_insecure_transport_policy = true
server_side_encryption_configuration = {
rule = {
apply_server_side_encryption_by_default = {
sse_algorithm = "AES256"
}
}
}
}
Here is the module that configure policies for the lambda :
module "my_lambda_policy" {
source = "terraform-aws-modules/iam/aws//modules/iam-policy"
name = "validate_lambda_policy"
path = "/"
description = "Validate Policy"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": [
"s3:Put*",
"s3:Get*",
"s3:List*",
"ses:SendEmail"
],
"Resource": [
"arn:aws:s3:::my-bucket/",
"arn:aws:s3:::my-bucket/*",
"arn:aws:ses:..."
]
}]
}
EOF
}
Terraform properly creates my bucket and configure my lambda, however, when my lambda tries to perform a "ListObjectsV2" or a "GetObject" operation, it get an "Access Denied".
I have set up with my policies some SES policy. These policies are properly applied (my lambda sends mails) so I expect that my S3 policies are also properly applied. Am I missing something with the bucket configuration ? What should I do to correct this (without setting my bucket full public of course)
This ARN is wrong for the S3 bucket:
"arn:aws:s3:::my-bucket/",
the / makes it not match the bucket ARN. This set of documentation is the best place I know of to determine exactly what an ARN looks like for a given resource.
So you should change it to
"arn:aws:s3:::my-bucket",
Without the slash. Leave "arn:aws:s3:::my-bucket/*" also, because that will match the objects' arns for Get/Put Object.

Malformed Policy Document: Has prohibited field Resource

I'm trying to create an IAM Role and IAM Policy using Terraform.
I'm getting this error:
│ Error: error creating IAM Role (asg-domain-join-policy): MalformedPolicyDocument: Has prohibited field Resource
status code: 400, request id: 53fa1ae0-f22f-4f2e-8aa6-1947421eae9b
with aws_iam_role.ad_join_role,
on iam.tf line 30, in resource "aws_iam_role" "ad_join_role":
30: resource "aws_iam_role" "ad_join_role" {
My current code for the IAM Role is the following:
resource "aws_iam_role" "ad_join_role" {
name = "asg-domain-join-policy"
assume_role_policy = data.aws_iam_policy_document.asg_domain_join_policy.json
permissions_boundary = "arn:aws:iam::${var.account_id}:policy/****"
}
The code for IAM Policy is the following:
data "aws_iam_policy_document" "asg_domain_join_policy" {
statement {
actions = [
"ssm:DescribeAssociation",
"ssm:GetDocument",
"ssm:ListAssociations",
"ssm:UpdateAssociationStatus",
"ssm:UpdateInstanceInformation",
"ssm:CreateAssociation",
]
effect = "Allow"
resources = ["ec2"]
}
}
I'm unsure why I'm getting that error.
The assume_role_policy can have a document which specifies only the AssumeRole action. What you have to do is split your policy to create separate ones for being able to assume a role, and being able to attach other permissions to the role.
For example:
# Allow EC2 instances to assume the role
data "aws_iam_policy_document" "asg_assume_role_policy" {
statement {
actions = [
"sts:AssumeRole"
]
effect = "Allow"
principals {
type = "Service"
identifiers = ["ec2.amazonaws.com"]
}
}
}
# Create the policy which allows other actions for the EC2 instance
data "aws_iam_policy_document" "asg_domain_join_policy" {
statement {
actions = [
"ssm:DescribeAssociation",
"ssm:GetDocument",
"ssm:ListAssociations",
"ssm:UpdateAssociationStatus",
"ssm:UpdateInstanceInformation",
"ssm:CreateAssociation"
]
effect = "Allow"
resources = ["*"]
}
}
resource "aws_iam_role" "ad_join_role" {
name = "asg-domain-join-policy"
assume_role_policy = data.aws_iam_policy_document.asg_assume_role_policy.json
# Attach the policy
inline_policy {
policy = data.aws_iam_policy_document.asg_domain_join_policy.json
}
}
Some things to note in this example:
The second policy is attached as an inline policy. This is fine, if the policy is shorter, otherwise you may want to use aws_iam_policy_attachment
The resource type for the actions in the second policy is a wildcard ["*"]. If you want more granularity for your actions on the policy, you may want to check out this page to see which action allows what kind of resource type. Obviously, ["ec2"] is not a valid resource type.
Your resource block is using an incorrect reference. ec2 is not a resource. If you are referencing an instance you need to use aws_instance.my_ec2_instance, or if you want to allow all resources you can put "*".

How to add the same IAM role to assume_role_policy with Terraform?

I'm creating a role for the Grafana EC2 instance to allow it to read metrics from CloudWatch. I've faced this issue: https://github.com/grafana/grafana/issues/19173, and it seems like I need to add these lines to the Trust relationship to fix it.
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::[id-removed]:role/grafana",
"arn:aws:sts::[id-removed]:assumed-role/grafana/GrafanaSession"
]
},
"Action": "sts:AssumeRole"
}
So I'm wondering how to do it with Terraform. At the moment I'm playing with this version of terraform script:
provider "aws" {
region = "eu-west-1"
version = "~> 2.0"
}
variable "aws_account_id" {
type = string
default = "account_id"
}
data "aws_iam_policy_document" "assume_role_policy" {
statement {
actions = ["sts:AssumeRole"]
principals {
type = "Service"
identifiers = ["ec2.amazonaws.com"]
}
}
statement {
actions = ["sts:AssumeRole"]
principals {
type = "AWS"
identifiers = [
"arn:aws:iam::${var.aws_account_id}:role/grafana",
"arn:aws:sts::${var.aws_account_id}:assumed-role/grafana/GrafanaSession"
]
}
}
}
resource "aws_iam_role" "grafana" {
name = "grafana"
assume_role_policy = data.aws_iam_policy_document.assume_role_policy.json
}
It fails with MalformedPolicyDocument: Invalid principal in policy: "AWS":"arn:aws:iam::[id-removed]:role/grafana"
To make it work I have to comment the second statement block of aws_iam_policy_document, run terraform apply, then uncomment it and run terraform apply again, which is not a very convenient way to work with, as we run our terraform scripts from GitLab, so it means we have to commit without assumed roles, tag, deploy, commit with assumed roles, tag and deploy again :explode

Terraform aws assume role

I have a problem with AWS assume role using terraform.
In AWS I have three accounts: root, staging and production (let's focus only on root & staging account) in single organization. The root account has one IAM user terraform (with AdministratorAccess policy) which is used by terraform to provisioning all stuff.
The image of organization structure
Root account ID: 111111111111
Staging account ID: 333333333333
A terraform script looks like that:
############## backend.tf
terraform {
required_version = "0.12.19"
}
############## providers.tf
provider "aws" {
region = "eu-west-1"
profile = "default"
}
provider "aws" {
version = ">= 2.44"
region = "eu-west-1"
profile = "default"
assume_role {
role_arn = "arn:aws:iam::333333333333:role/staging-admin"
}
allowed_account_ids = ["333333333333"]
alias = "staging"
}
############## organization.tf
resource "aws_organizations_account" "staging" {
email = "staging#domain.com"
name = "Staging"
parent_id = "ZZZZZ"
}
############## data.tf
data "aws_iam_policy_document" "assume_staging_role" {
statement {
effect = "Allow"
actions = ["sts:AssumeRole"]
resources = [
aws_iam_role.staging.arn
]
}
}
data "aws_iam_policy" "administrator_access" {
arn = "arn:aws:iam::aws:policy/AdministratorAccess"
}
data "template_file" "cross_admin_trust_policy" {
template = file("templates/cross_admin_trust_policy.json")
vars = {
staging_account_number = aws_organizations_account.staging.id
}
}
############## iam.tf
resource "aws_iam_role" "staging" {
name = "staging-admin"
description = "Assumable role granting administrator permissions to the staging account"
assume_role_policy = data.template_file.cross_admin_trust_policy.rendered
max_session_duration = 20000
provider = aws.staging
}
resource "aws_iam_role_policy_attachment" "staging_admin_access" {
role = aws_iam_role.staging.name
policy_arn = data.aws_iam_policy.administrator_access.arn
provider = aws.staging_ireland
}
resource "aws_iam_role_policy_attachment" "staging_attach_assume_any_admin" {
role = aws_iam_role.staging.name
policy_arn = data.aws_iam_policy.administrator_access.arn
provider = aws.staging_ireland
}
and my policy.json file:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${staging_account_number}:role/staging-admin"
},
"Action": "sts:AssumeRole"
}
]
}
when I execute terraform plan I'm getting this error:
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.aws_iam_policy.administrator_access: Refreshing state...
aws_organizations_account.staging: Refreshing state... [id=333333333333]
data.template_file.cross_admin_trust_policy: Refreshing state...
Error: The role "arn:aws:iam::333333333333:role/staging-admin" cannot be assumed.
There are a number of possible causes of this - the most common are:
* The credentials used in order to assume the role are invalid
* The credentials do not have appropriate permission to assume the role
* The role ARN is not valid
on providers.tf line 25, in provider "aws"
Someone has an idea how to fix?
According to https://aws.amazon.com/premiumsupport/knowledge-center/iam-assume-role-cli/
Run the aws sts get-caller-identity command to check your identity.
Run the aws sts assume-role --role-arn arn:aws:iam::333333333333:role/staging-admin command to see if this role can be assumed by your identity.
Check your IAM role's trust relationship. You should restrict it so that the IAM role should be only assumed by specific IAM users.
{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Principal": { "AWS": "arn:aws:iam:: 333333333333:root" },
"Action": "sts:AssumeRole"
}
}
According to the data you provided you don't have a role, and you trying to add roles with first apply. Correct me if I'm wrong.
You need to manually create an "account aws role" in IAM with "AdministratorAccess" policy attached.