Why can't I access my bucket from an assumed role? - amazon-web-services

Question

I have an S3 bucket with no attached ACLs or policies. It was created by terraform like so:
resource "aws_s3_bucket" "runners_cache" {
bucket = var.runners_cache.bucket
}
I created a role and attached a policy to it; see the following console log for details
$ aws iam get-role --role-name bootstrap-test-bootstrapper
{
"Role": {
{
"Role": {
"Path": "/bootstrap-test/",
"RoleName": "bootstrap-test-bootstrapper",
"RoleId": "#SNIP",
"Arn": "arn:aws:iam::#SNIP:role/bootstrap-test/bootstrap-test-bootstrapper",
... #SNIP
$ aws iam list-attached-role-policies --role-name bootstrap-test-bootstrapper
{
"AttachedPolicies": [
{
"PolicyName": "bootstrap-test-bootstrapper",
"PolicyArn": "arn:aws:iam::#SNIP:policy/bootstrap-test/bootstrap-test-bootstrapper"
},
... #SNIP
$ aws iam get-policy --policy-arn arn:aws:iam::#SNIP:policy/bootstrap-test/bootstrap-test-runner
{
"Policy": {
"PolicyName": "bootstrap-test-runner",
"PolicyId": "#SNIP",
"Arn": "arn:aws:iam::#SNIP:policy/bootstrap-test/bootstrap-test-runner",
"Path": "/bootstrap-test/",
"DefaultVersionId": "v7",
... #SNIP
$ aws iam get-policy-version --policy-arn arn:aws:iam::#SNIP:policy/bootstrap-test/bootstrap-test-runner --version-id v7
{
"PolicyVersion": {
"Document": {
"Statement": [
{
"Action": [
"s3:AbortMultipartUpload",
"s3:CompleteMultipartUpload",
"s3:ListBucket",
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:PutObjectAcl"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::#SNIP-runners-cache/*",
"arn:aws:s3:::#SNIP-cloud-infrastructure-terraform-states/*"
]
},
{
"Action": [
"s3:*"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::*"
]
}
],
"Version": "2012-10-17"
},
"VersionId": "v7",
"IsDefaultVersion": true,
"CreateDate": "2022-08-18T14:16:33+00:00"
}
}
tl;dr this role has an attached policy that allows full access to s3 within the account.
I can successfully assume this role:
$ aws sts assume-role --role-arn arn:aws:iam::#SNIP:role/bootstrap-test/bootstrap-test-bootstrapper --role-session-name test123
{ ... #REDACTED }
$ export AWS_ACCESS_KEY_ID=ASIA2 #REDACTED
$ export AWS_SECRET_ACCESS_KEY=8 #REDACTED
$ export AWS_SESSION_TOKEN=IQoJb #REDACTED
$ aws sts get-caller-identity
{
"UserId": "#SNIP",
"Account": "#SNIP",
"Arn": "arn:aws:sts::#SNIP:assumed-role/bootstrap-test-bootstrapper/test123"
}
However, once I do this, I no longer have access to S3:
$ aws s3 ls #SNIP-runners-cache
An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied
$ aws s3 ls
An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied
What am I missing? Is there some default behavior that prevents access to S3? How should I go about debugging these 403 errors?

Answer 1

It is easy to get over-obsessed with the details of the policy and forget about the role itself. In this case the permissions boundary went unnoticed in the CLI, but it is quite easy to see in the web console:
Indeed, #luk2302 was right, the limiting factor was a permissions boundary. After removing it from the role, access to S3 was restored.