Cross account IAM roles for Kubernetes service account - s3 bucket - amazon-web-services

Question

Hey im trying to cross account access for a role. i have 2 accounts: prod and non-prod.
and bucket in prod account, which im trying to write files to there from a non-prod role which is used as a service account in k8s cluster.
in prod account i configured:
a role with the following policy(read write access to the bucket):
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ListObjectsInBucket",
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::test2"
]
},
{
"Sid": "AllObjectActions",
"Effect": "Allow",
"Action": "s3:*Object",
"Resource": [
"arn:aws:s3:::test2/*"
]
}
]
}
and the following trust:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::non-prod-AccountID:role/name-of-the-non-prod-role"
},
"Action": "sts:AssumeRole",
"Condition": {}
}
]
}
in non prod i configured:
a role with the following policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::prod-Account-ID:role/prod-role-name"
},
"Action": "sts:AssumeRole",
"Condition": {}
}
]
}
and trust as follows:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::non-prod-accountID:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/1111111111111111111"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.us-east-1.amazonaws.com/id/1111111111111111111:sub":
"system:serviceaccount:name-space:name-of-the-service-account"
}
}
}
]
}
serviceAccount annotation is:
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::non-prod-AccountID:role/non-prod-role-name
when running the command from inside the pod with the service account of the role in non-prod:
aws s3 cp hello.txt s3://test2/hello.txt
im having:
upload failed: ./hello.txt to s3://test2/hello.txt An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
by the way the cluster is in another account (devops account) if its related, surely added OIDC provider identity to both non-prod and prod accounts as identity provider.

Answer 1

If you're getting the error An error occurred (InvalidIdentityToken) when calling the AssumeRoleWithWebIdentity operation: No OpenIDConnect provider found in your account for $oidc_url when trying to cross-account assume roles, but you can assume roles in your cluster account normally, here's some points:
EKS ACCOUNT
Create a ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
name: $sa_name
namespace: $eks_ns
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::$resource_account_id:role/$role_name
Annotate your deployment
spec.template.spec:
serviceAccountName: $sa_name
Get info about your cluster OIDC Provider
aws iam get-open-id-connect-provider --open-id-connect-provider-arn arn:aws:iam::$eks_cluster_account_id:oidc-provider/$oidc_provider
3.1. The output will be like:
{
"Url": "...",
"ClientIDList": ["..."],
"ThumbprintList": ["..."],
"CreateDate": "...",
"Tags": [...]
}
3.2. Take note of the outputs (Url and ThumbprintList specially)
RESOURCE ACCOUNT
Add the provider (if you don`t have it already), using the output from your cluster account
aws iam create-open-id-connect-provider --url $oidc_url --client-id-list sts.amazonaws.com --thumbprint-list $oidc_thumbprint
This should be enought to the mentioned error stop. If you now get An error occurred (AccessDenied) when calling the AssumeRoleWithWebIdentity operation: Not authorized to perform sts:AssumeRoleWithWebIdentity, you're problably using the $eks_cluster_account_id on Principal.Federated, instead of $resource_account_id created on the previous step. So, make sure you're using the ARN from the IP that is assigned to the resource account, not the cluster account.
Create a role and a policy to access your resources with following trusted entities policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::$resource_account_id:oidc-provider/$oidc_provider"
},
"Action": "sts:AssumeRoleWithWebIdentity"
}
]
}
Also, there's no need to have two roles. One is enough.