I have a problem with AWS assume role using terraform.
In AWS I have three accounts: root, staging and production (let's focus only on root & staging account) in single organization. The root account has one IAM user terraform (with AdministratorAccess policy) which is used by terraform to provisioning all stuff.
The image of organization structure
Root account ID: 111111111111
Staging account ID: 333333333333
A terraform script looks like that:
############## backend.tf
terraform {
required_version = "0.12.19"
}
############## providers.tf
provider "aws" {
region = "eu-west-1"
profile = "default"
}
provider "aws" {
version = ">= 2.44"
region = "eu-west-1"
profile = "default"
assume_role {
role_arn = "arn:aws:iam::333333333333:role/staging-admin"
}
allowed_account_ids = ["333333333333"]
alias = "staging"
}
############## organization.tf
resource "aws_organizations_account" "staging" {
email = "staging#domain.com"
name = "Staging"
parent_id = "ZZZZZ"
}
############## data.tf
data "aws_iam_policy_document" "assume_staging_role" {
statement {
effect = "Allow"
actions = ["sts:AssumeRole"]
resources = [
aws_iam_role.staging.arn
]
}
}
data "aws_iam_policy" "administrator_access" {
arn = "arn:aws:iam::aws:policy/AdministratorAccess"
}
data "template_file" "cross_admin_trust_policy" {
template = file("templates/cross_admin_trust_policy.json")
vars = {
staging_account_number = aws_organizations_account.staging.id
}
}
############## iam.tf
resource "aws_iam_role" "staging" {
name = "staging-admin"
description = "Assumable role granting administrator permissions to the staging account"
assume_role_policy = data.template_file.cross_admin_trust_policy.rendered
max_session_duration = 20000
provider = aws.staging
}
resource "aws_iam_role_policy_attachment" "staging_admin_access" {
role = aws_iam_role.staging.name
policy_arn = data.aws_iam_policy.administrator_access.arn
provider = aws.staging_ireland
}
resource "aws_iam_role_policy_attachment" "staging_attach_assume_any_admin" {
role = aws_iam_role.staging.name
policy_arn = data.aws_iam_policy.administrator_access.arn
provider = aws.staging_ireland
}
and my policy.json file:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${staging_account_number}:role/staging-admin"
},
"Action": "sts:AssumeRole"
}
]
}
when I execute terraform plan I'm getting this error:
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.aws_iam_policy.administrator_access: Refreshing state...
aws_organizations_account.staging: Refreshing state... [id=333333333333]
data.template_file.cross_admin_trust_policy: Refreshing state...
Error: The role "arn:aws:iam::333333333333:role/staging-admin" cannot be assumed.
There are a number of possible causes of this - the most common are:
* The credentials used in order to assume the role are invalid
* The credentials do not have appropriate permission to assume the role
* The role ARN is not valid
on providers.tf line 25, in provider "aws"
Someone has an idea how to fix?
According to https://aws.amazon.com/premiumsupport/knowledge-center/iam-assume-role-cli/
Run the aws sts get-caller-identity command to check your identity.
Run the aws sts assume-role --role-arn arn:aws:iam::333333333333:role/staging-admin command to see if this role can be assumed by your identity.
Check your IAM role's trust relationship. You should restrict it so that the IAM role should be only assumed by specific IAM users.
{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Principal": { "AWS": "arn:aws:iam:: 333333333333:root" },
"Action": "sts:AssumeRole"
}
}
According to the data you provided you don't have a role, and you trying to add roles with first apply. Correct me if I'm wrong.
You need to manually create an "account aws role" in IAM with "AdministratorAccess" policy attached.
Related
I'm trying to run terraform plan locally with a non-default aws credentials profile, where my default profile will not work. I also need to use assume_role in terraform provider "aws". My code looks something like this:
provider "aws" {
version = "~> 2.45"
region = "us-east-1"
profile = <profile name>
allowed_account_ids = [<account_id>]
assume_role {
role_arn = "arn:aws:iam::<account id>:role/<role name>"
}
}
The error I'm getting is:
Error: The role "arn:aws:iam::<account_id>:role/<role_name>" cannot be assumed.
There are a number of possible causes of this - the most common are:
* The credentials used in order to assume the role are invalid
* The credentials do not have appropriate permission to assume the role
* The role ARN is not valid
Interestingly, when I put access_key and secret_key in the provider like this:
provider "aws" {
version = "~> 2.45"
region = "us-east-1"
access_key = <aws access key>
secret_key = <aws secret key>
assume_role {
role_arn = "arn:aws:iam::<account_id>:role/<role_name>"
}
}
terraform plan works fine. I've double checked my aws credentials file several time and it's setup correctly, but I'm not sure why terraform plan doesn't work.
I've also tried deleting the assume_role parameter in provider "aws" when i have access_key and secret_key in the file, and terraform plan works fine, which means i don't need the assume_role. however, if i use the profile from aws credentials without assume_role in terraform file, i'm getting:
Error: error using credentials to get account ID: error calling sts:GetCallerIdentity: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
status code: 403, request id:
You need amend the Trust Policy on the IAM Role like below
How to use trust policies with IAM roles
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::111122223333:user/<Your username>"
},
"Action": "sts:AssumeRole",
"Condition": {}
}
]
}
Once you update the Trust Policy on the IAM Role you can verify this via assume-role command
aws sts assume-role --role-arn arn:aws:iam::123456789012:role/xaccounts3access --role-session-name s3-access-example
You would receive something like:
{
"AssumedRoleUser": {
"AssumedRoleId": "AROA3XFRBF535PLBIFPI4:s3-access-example",
"Arn": "arn:aws:sts::123456789012:assumed-role/xaccounts3access/s3-access-example"
},
"Credentials": {
"SecretAccessKey": "9drTJvcXLB89EXAMPLELB8923FB892xMFI",
"SessionToken": "AQoXdzELDDY//////////wEaoAK1wvxJY12r2IrDFT2IvAzTCn3zHoZ7YNtpiQLF0MqZye/qwjzP2iEXAMPLEbw/m3hsj8VBTkPORGvr9jM5sgP+w9IZWZnU+LWhmg+a5fDi2oTGUYcdg9uexQ4mtCHIHfi4citgqZTgco40Yqr4lIlo4V2b2Dyauk0eYFNebHtYlFVgAUj+7Indz3LU0aTWk1WKIjHmmMCIoTkyYp/k7kUG7moeEYKSitwQIi6Gjn+nyzM+PtoA3685ixzv0R7i5rjQi0YE0lf1oeie3bDiNHncmzosRM6SFiPzSvp6h/32xQuZsjcypmwsPSDtTPYcs0+YN/8BRi2/IcrxSpnWEXAMPLEXSDFTAQAM6Dl9zR0tXoybnlrZIwMLlMi1Kcgo5OytwU=",
"Expiration": "2016-03-15T00:05:07Z",
"AccessKeyId": "ASIAJEXAMPLEXEG2JICEA"
}
}
You also have to specify the role arn on the backend.tf:
terraform {
backend "s3" {
bucket = "your_bucket"
key = "your_key"
region = "us-east-1" (or whataver is the one you're using)
role_arn = "arn:aws:iam::<account_id>:role/<role_name>"
}
}
I'm creating a role for the Grafana EC2 instance to allow it to read metrics from CloudWatch. I've faced this issue: https://github.com/grafana/grafana/issues/19173, and it seems like I need to add these lines to the Trust relationship to fix it.
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::[id-removed]:role/grafana",
"arn:aws:sts::[id-removed]:assumed-role/grafana/GrafanaSession"
]
},
"Action": "sts:AssumeRole"
}
So I'm wondering how to do it with Terraform. At the moment I'm playing with this version of terraform script:
provider "aws" {
region = "eu-west-1"
version = "~> 2.0"
}
variable "aws_account_id" {
type = string
default = "account_id"
}
data "aws_iam_policy_document" "assume_role_policy" {
statement {
actions = ["sts:AssumeRole"]
principals {
type = "Service"
identifiers = ["ec2.amazonaws.com"]
}
}
statement {
actions = ["sts:AssumeRole"]
principals {
type = "AWS"
identifiers = [
"arn:aws:iam::${var.aws_account_id}:role/grafana",
"arn:aws:sts::${var.aws_account_id}:assumed-role/grafana/GrafanaSession"
]
}
}
}
resource "aws_iam_role" "grafana" {
name = "grafana"
assume_role_policy = data.aws_iam_policy_document.assume_role_policy.json
}
It fails with MalformedPolicyDocument: Invalid principal in policy: "AWS":"arn:aws:iam::[id-removed]:role/grafana"
To make it work I have to comment the second statement block of aws_iam_policy_document, run terraform apply, then uncomment it and run terraform apply again, which is not a very convenient way to work with, as we run our terraform scripts from GitLab, so it means we have to commit without assumed roles, tag, deploy, commit with assumed roles, tag and deploy again :explode
I've followed the first answer on this post on StackOverflow but I obtain this error:
Failure configuring LB attributes: InvalidConfigurationRequest: Access Denied for bucket: myproject-log. Please check S3bucket permission status code: 400
This is my code:
s3_bucket
data "aws_elb_service_account" "main" {}
resource "aws_s3_bucket" "bucket_log" {
bucket = "${var.project}-log"
acl = "log-delivery-write"
policy = <<POLICY
{
"Id": "Policy",
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:PutObject"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::${var.project}-log/AWSLogs/*",
"Principal": {
"AWS": [
"${data.aws_elb_service_account.main.arn}"
]
}
}
]
}
POLICY
}
load balancer
resource "aws_lb" "vm_stage" {
name = "${var.project}-lb-stg"
internal = false
load_balancer_type = "application"
subnets = [aws_subnet.subnet_1.id, aws_subnet.subnet_2.id, aws_subnet.subnet_3.id]
security_groups = [aws_security_group.elb_project_stg.id]
access_logs {
bucket = aws_s3_bucket.bucket_log.id
prefix = "lb-stg"
enabled = true
}
tags = {
Name = "${var.project}-lb-stg"
}
}
Just going to drop this here since this cross applied to another question that was asked.
This took me awhile to figure out, but the S3 bucket has two requirements per the documentation:
The bucket must be located in the same Region as the load balancer.
Amazon S3-Managed Encryption Keys (SSE-S3) is required. No other encryption options are supported.
Source: https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/enable-access-logs.html
While it makes it seem like it's a permissions issue with the error message given it may actually be an issue with the bucket having the wrong encryption type. In my case the issue was that my bucket was unencrypted.
Updated the bucket to SSE-S3 encryption and I no longer received the error:
resource "aws_s3_bucket" "s3_access_logs_bucket" {
bucket = var.access_logs_bucket_name
acl = "private"
server_side_encryption_configuration {
rule {
apply_server_side_encryption_by_default {
sse_algorithm = "AES256"
}
}
}
versioning {
enabled = true
}
}
And just because, here's the policy I used:
data "aws_elb_service_account" "main" {}
data "aws_iam_policy_document" "s3_lb_write" {
statement {
principals {
identifiers = ["${data.aws_elb_service_account.main.arn}"]
type = "AWS"
}
actions = ["s3:PutObject"]
resources = [
"${aws_s3_bucket.s3_access_logs_bucket.arn}/*"
]
}
}
resource "aws_s3_bucket_policy" "load_balancer_access_logs_bucket_policy" {
bucket = aws_s3_bucket.s3_access_logs_bucket.id
policy = data.aws_iam_policy_document.s3_lb_write.json
}
Official AWS Docs
https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/enable-access-logs.html
Solution
Reference the docs above and change your bucket's iam policy to reflect what the documentation states. The logging is actually done by AWS and not your roles or IAM users. So you need to give ÅWS permission to do this. That's why the docs show statements in the policy that specify the delivery.logs.amazonaws.com principal. That principal is the AWS logging service. Even though your bucket is hosted on AWS, they don't give themselves access to your bucket by default. You have to explicitly grant access to AWS if you want their services to work.
As per this post, I was able to resolve this issue by disabling KMS and using SSE-S3 for bucket encryption. Also, there are additional permissions listed in the AWS docs.
I struggled with this as well the entire terraform bucket policy that worked for me is below.
data "aws_iam_policy_document" "elb_bucket_policy" {
statement {
effect = "Allow"
resources = [
"arn:aws:s3:::unique-bucket-name/${local.prefix}/AWSLogs/${local.application_account_id}/*",
]
actions = ["s3:PutObject"]
principals {
type = "AWS"
identifiers = ["arn:aws:iam::${local.elb_account_id}:root"]
}
}
statement {
effect = "Allow"
resources = [
"arn:aws:s3:::unique-bucket-name/${local.prefix}/AWSLogs/${local.application_account_id}/*",
]
actions = ["s3:PutObject"]
principals {
type = "Service"
identifiers = ["logdelivery.elb.amazonaws.com"]
}
}
statement {
effect = "Allow"
resources = [
"arn:aws:s3:::unique-bucket-name/${local.prefix}/AWSLogs/${local.application_account_id}/*",
]
actions = ["s3:PutObject"]
principals {
type = "Service"
identifiers = ["logdelivery.elb.amazonaws.com"]
}
condition {
test = "StringEquals"
variable = "s3:x-amz-acl"
values = ["bucket-owner-full-control"]
}
}
}
I'm using Terraform to create an EKS cluster. The worker nodes have an IAM role defined as follows:
resource "aws_iam_role" "eks-node" {
name = "${local.resource_prefix}-eks-node"
assume_role_policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
POLICY
}
resource "aws_iam_role_policy_attachment" "eks-node-AmazonEKSWorkerNodePolicy" {
policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
role = "${aws_iam_role.eks-node.name}"
}
resource "aws_iam_role_policy_attachment" "eks-node-AmazonEKS_CNI_Policy" {
policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
role = "${aws_iam_role.eks-node.name}"
}
resource "aws_iam_role_policy_attachment" "eks-node-AmazonEC2ContainerRegistryReadOnly" {
policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
role = "${aws_iam_role.eks-node.name}"
}
resource "aws_iam_instance_profile" "eks-node" {
name = "${local.resource_prefix}-eks-node"
role = "${aws_iam_role.eks-node.name}"
}
This all spins up fine, and I can launch containers into the cluster. Now I want to try and play with allowing the containers to access AWS SSM in order to retrieve sensitive information (like a DB password). I did a quick test by launching a simple alpine instance into the cluster, installing the AWS CLI and running
aws ssm get-parameters --name /tmp/test-001 --region ...
(the /tmp/test-001 parameter does exist in the same region). This spits back a predictable error of:
An error occurred (AccessDeniedException) when calling the GetParameters operation: User: arn:aws:sts::(account-id):assumed-role/(resource-prefix)-eks-node/i-(EC2-instance-id) is not authorized to perform: ssm:GetParameters on resource: arn:aws:ssm:(region):(account-id):parameter/tmp/test-001
That's good actually - I want this to be rejected until I give that user permission. However note that the user is assumed-role/(resource-prefix)-eks-node/i-(EC2-instance-id) and not simply assumed-role/(resource-prefix)-eks-node - I suspect that has something to do with my actual problem...
So I head back to Terraform and add in the following Policy to the worker nodes:
resource "aws_iam_role_policy" "eks_node_allow_ssm_access" {
name = "${local.resource_prefix}-eks-node-ssm-access"
role = "${aws_iam_role.eks-node.name}"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ssm:GetParameters"
],
"Effect": "Allow",
"Resource": "arn:aws:ssm:${local.aws_region}:${data.aws_caller_identity.current.account_id}:parameter/tmp/test-001"
}
]
}
EOF
}
This generates what I would expect - an inline Policy on arn:aws:iam::(account-id):role/(resource-prefix)-eks-node that allows acccess to ssm:GetParameters. However when I re-launch the container and try again, I still get the AccessDeniedException error.
I suspect it has something to do with the assumed-role being for (prefix)-eks-node/i-0...............e instead of what the role Terraform creates actually is (prefix)-eks-node. But since that EC2 instance is dynamically created with auto-scaling, I have no idea how I would target that with a policy...
All that brings me back to the question... How do I allow the assumed user role of the EC2 instance in the EKS worker nodes access to another resource (in this case SSM)?
I want to perform MFA for Terraform so it's expected to ask the 6-digit token from my virtual MFA device for every terraform [command]. After reading the documentation:
cli-roles
terraform mfa
I created a role:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::[ACCOUNT_ID]:user/testuser"
},
"Action": "sts:AssumeRole",
"Condition": {
"Bool": {
"aws:MultiFactorAuthPresent": "true"
}
}
}
]
}
This user is forced to use MFA by default and I have a configured virtual MFA device for him.
~/.aws/credentials:
[default]
...
[terraform_role]
role_arn = arn:aws:iam::[ACCOUNT_ID]:role/terraform-test-role
source_profile = default
mfa_serial = arn:aws:iam::[ACCOUNT_ID]:mfa/testuser
in my Terraform environment I placed the following:
provider "aws" {
profile = "terraform_role"
}
But when i run terraform plan it throws me an error:
Error refreshing state: 1 error(s) occurred:
* provider.aws: No valid credential sources found for AWS Provider.
Please see https://terraform.io/docs/providers/aws/index.html for more information on
providing credentials for the AWS Provider
The solution is to specify an assume_role statement:
provider "aws" {
profile = "default"
assume_role {
role_arn = "arn:aws:iam::[ACCOUNT_ID]:role/terraform-test-role"
}
}
Unfortunately, the assume_role statement by itself is not a working solution. You need to use a MFA authentication wrapper aws-runas that eases the process not only of assuming the role but providing support for the mfa_serial clause on the .aws/credentials file.
In short, there are 3 steps:
You'll need to have your .aws/credentials file as you have.
Install aws-runas
Run the apply: aws-runas [your_profile] - terraform apply