I am trying to simulate an IAM policy I want to attach to a user so I can restrict their access to two buckets, one for file upload and one for file download.
The policy simulator tells me that the following policy does not work and I cannot figure out why, but it seems to be to do with the wildcards.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "GetObject",
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::mybucket-*-report-output/*.csv"
]
},
{
"Sid": "PutObjects",
"Effect": "Allow",
"Action": "s3:PutObject",
"Resource": [
"arn:aws:s3:::mybucket-*-report-input/*.csv"
]
}
]
}
The policy simulator says the following policy does work however:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "GetObject",
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::mybucket-*-report-output"
]
},
{
"Sid": "PutObjects",
"Effect": "Allow",
"Action": "s3:PutObject",
"Resource": [
"arn:aws:s3:::mybucket-*-report-input"
]
}
]
}
There must be something I am missing about how to structure the policy, but I want to restrict access to the buckets in the policy, for the operations mentioned, but I also want to ensure that the user can only add and retrieve files with .csv extension.
Below is a screenshot of the simulator:
Your policy is 100% correct - the IAM Policy Simulator is showing wrong results for some absurd reason.
I also can reproduce your problem using the above policy, and the results are all over the place - sometimes both allowed, both denied, only one allowed etc.
It seems to be having an issue with the double wildcard, and sometimes it is coming back with the wrong resource ARN being evaluated in the HTTP response being returned (I'm sometimes seeing both ARNs set to output instead of only 1 set to output in the network tab for the HTTP response - caching?).
It's not limited to PutObject either only and it's giving me loads of conflicting results with the double wildcard, even for other actions like s3:RestoreObject.
Regardless, I'm not sure what the issue is but your policy is correct - ignore IAM Policy Simulator in this case.
If you have access to AWS Support, I would create a support ticket there or post this same question as a potential bug on the AWS forums.
Evidence of a conflicting result, even though I have exactly recreated your scenario:
I am trying to import a disk image into AWS, for EC2 instance launching. I follow the guide as stated and fulfill all the prerequisites as stated. However I am faced with an error that I've been trying (unsuccessfully) to debug. The error is as follows. An error occurred (InvalidParameter) when calling the ImportImage operation: The service role vmimport provided does not exist or does not have sufficient permissions However when I check the permissions of the vmimport role it has all necessary permissions for EC2 and S3! My aws cli user also has full privileges to EC2 and S3. I've tried many different solutions to this problem, including, 1. Making the S3 bucket public, 2. Adding an access policy so that my AWS cli user had permissions to access the bucket. Everything I have tried still returns this exact same error message... I'm thinking there might be a region problem? I'm using us-east-2 in my AWS cli user configuration, and in the S3 buckets region. Is there something else I have not considered?P.S. I'm trying to import an OVA 1 format vm image.
Here is my trust policy
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": { "Service": "vmie.amazonaws.com" },
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals":{
"sts:Externaloid": "vmimport"
}
}
}
]
}
and my roles policy
"Version":"2012-10-17",
"Statement":[
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::MY-IMPORT-BUCKET",
"arn:aws:s3:::MY-IMPORT-BUCKET/*"
]
},
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:GetObject",
"s3:ListBucket",
"s3:PutObject",
"s3:GetBucketAcl"
],
"Resource": [
"arn:aws:s3:::MY-EXPORT-BUCKET",
"arn:aws:s3:::MY-EXPORT-BUCKET/*"
]
},
{
"Effect": "Allow",
"Action": [
"ec2:ModifySnapshotAttribute",
"ec2:CopySnapshot",
"ec2:RegisterImage",
"ec2:Describe*"
],
"Resource": "*"
}
]
}
And finally the containers.json
[
{
"Description": "My Special OVA",
"Format": "ova",
"Url": "s3://MY-IMPORT-BUCKET/VM.ova"
}
]
UPDATE: After investigating the problem further, I found that the role vmimport last access was "Not accessed", i.e. never, meaning that the role is not even being attempted to be used! So this error is clearly saying that it does not exist (it can't find the service role). In the final command there is nothing in the command that suggests that vmimport is going to be used, neither in the containers.json . I thought this was the purpose of allowing vmie.amazonaws.com to take control. Clearly it isn't assuming the role, so I need to investigate into this and sts.
The problem is in your(my) Trust Policy.json file. If you notice the conditions for it to assume the role, is that the Externaloid must be equal to vmimport. There is an added o in the attribute that it is checking, this will always be false and so vmie can never assume the role. Remove the o from the trust policy and try again and your policy works.
I had exactly the same scenario, you need to create vmimport role as described here (AWS docs):
https://docs.aws.amazon.com/vm-import/latest/userguide/required-permissions.html
I am getting:
An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied
When I try to get folder from my S3 bucket.
Using this command:
aws s3 cp s3://bucket-name/data/all-data/ . --recursive
The IAM permissions for the bucket look like this:
{
"Version": "version_id",
"Statement": [
{
"Sid": "some_id",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::bucketname/*"
]
}
] }
What do I need to change to be able to copy and ls successfully?
You have given permission to perform commands on objects inside the S3 bucket, but you have not given permission to perform any actions on the bucket itself.
Slightly modifying your policy would look like this:
{
"Version": "version_id",
"Statement": [
{
"Sid": "some_id",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::bucketname",
"arn:aws:s3:::bucketname/*"
]
}
]
}
However, that probably gives more permission than is needed. Following the AWS IAM best practice of Granting Least Privilege would look something like this:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::bucketname"
]
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::bucketname/*"
]
}
]
}
If you wanted to copy all s3 bucket objects using the command "aws s3 cp s3://bucket-name/data/all-data/ . --recursive" as you mentioned, here is a safe and minimal policy to do that:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::bucket-name"
],
"Condition": {
"StringLike": {
"s3:prefix": "data/all-data/*"
}
}
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::bucket-name/data/all-data/*"
]
}
]
}
The first statement in this policy allows for listing objects inside a specific bucket's sub directory. The resource needs to be the arn of the S3 bucket, and to limit listing to only a sub-directory in that bucket you can edit the "s3:prefix" value.
The second statement in this policy allows for getting objects inside of the bucket at a specific sub-directory. This means that anything inside the "s3://bucket-name/data/all-data/" path you will be able to copy. Be aware that this doesn't allow you to copy from parent paths such as "s3://bucket-name/data/".
This solution is specific to limiting use for AWS CLI commands; if you need to limit S3 access through the AWS console or API, then more policies will be needed. I suggest taking a look here: https://aws.amazon.com/blogs/security/writing-iam-policies-grant-access-to-user-specific-folders-in-an-amazon-s3-bucket/.
A similar issue to this can be found here which led me to the solution I am giving.
https://github.com/aws/aws-cli/issues/2408
Hope this helps!
I got the same error when using policy as below, although i have "s3:ListBucket" for s3:ListObjects operation.
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:ListBucket",
"s3:GetObject",
"s3:GetObjectAcl"
],
"Resource": [
"arn:aws:s3:::<bucketname>/*",
"arn:aws:s3:::*-bucket/*"
],
"Effect": "Allow"
}
]
}
Then i fixed it by adding one line
"arn:aws:s3:::bucketname"
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:ListBucket",
"s3:GetObject",
"s3:GetObjectAcl"
],
"Resource": [
"arn:aws:s3:::<bucketname>",
"arn:aws:s3:::<bucketname>/*",
"arn:aws:s3:::*-bucket/*"
],
"Effect": "Allow"
}
]
}
I tried the following:
aws s3 ls s3.console.aws.amazon.com/s3/buckets/{bucket name}
This gave me the error:
An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied
Using this form worked:
aws s3 ls {bucket name}
I was unable to access to S3 because
first I configured key access on the instance (it was impossible to attach role after the launch then)
forgot about it for a few months
attached role to instance
tried to access.
The configured key had higher priority than role, and access was denied because the user wasn't granted with necessary S3 permissions.
Solution: rm -rf .aws/credentials, then aws uses role.
I faced with the same issue. I just added credentials config:
aws_access_key_id = your_aws_access_key_id
aws_secret_access_key = your_aws_secret_access_key
into "~/.aws/credentials" + restart terminal for default profile.
In the case of multi profiles --profile arg needs to be added:
aws s3 sync ./localDir s3://bucketName --profile=${PROFILE_NAME}
where PROFILE_NAME:
.bash_profile ( or .bashrc) -> export PROFILE_NAME="yourProfileName"
More info about how to config credentials and multi profiles can be found here
For Amazon users who have enabled MFA, please use this:
aws s3 ls s3://bucket-name --profile mfa.
And prepare the profile mfa first by running
aws sts get-session-token --serial-number arn:aws:iam::123456789012:mfa/user-name --token-code 928371 --duration 129600. (replace 123456789012, user-name and 928371).
You have to specify Resource for the bucket via "arn:aws:s3:::bucketname" or "arn:aws:3:::bucketname*". The latter is preferred since it allows manipulations on the bucket's objects too. Notice there is no slash!
Listing objects is an operation on Bucket. Therefore, action "s3:ListBucket" is required.
Adding an object to the Bucket is an operation on Object. Therefore, action "s3:PutObject" is needed.
Certainly, you may want to add other actions as you require.
{
"Version": "version_id",
"Statement": [
{
"Sid": "some_id",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::bucketname*"
]
}
]
}
Okay for those who have done all the above and still getting this issue, try this:
Bucket Policy should look like this:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowBucketSync",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:PutObjectAcl",
"s3:DeleteObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::BUCKET_NAME",
"arn:aws:s3:::BUCKET_NAME/*"
]
}
]
}
Then save and ensure your Instance or Lightsail is connected to the right profile on AWS Configure.
First:
try adding --recursive at the end, any luck? No okay try the one below.
Second:
Okay now try this instead: --no-sign-request
so it should look like this:
sudo aws s3 sync s3://BUCKET_NAME /yourpath/path/folder --no-sign-request
You're welcome 😂
I was thinking the error is due to "s3:ListObjects" action but I had to add the action "s3:ListBucket" to solve the issue "AccessDenied for ListObjects for S3 bucket"
I'm adding an answer with the same direction as the accepted answer but with small (important) differences and adding more details.
Consider the configuration below:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:ListBucket"],
"Resource": ["arn:aws:s3:::<Bucket-Name>"]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:DeleteObject"
],
"Resource": ["arn:aws:s3:::<Bucket-Name>/*"]
}
]
}
The policy grants programmatic write-delete access and is separated into two parts:
The ListBucket action provides permissions on the bucket level and the other PutObject/DeleteObject actions require permissions on the objects inside the bucket.
The first Resource element specifies arn:aws:s3:::<Bucket-Name> for the ListBucket action so that applications can list all objects in the bucket.
The second Resource element specifies arn:aws:s3:::<Bucket-Name>/* for the PutObject, and DeletObject actions so that applications can write or delete any objects in the bucket.
The separation into two different 'arns' is important from security reasons in order to specify bucket-level and object-level fine grained permissions.
Notice that if I would have specified just GetObject in the 2nd block what would happen is that in cases of programmatic access I would receive an error like:
Upload failed: <file-name> to <bucket-name>:<path-in-bucket> An error occurred (AccessDenied) when calling the PutObject operation: Access Denied.
To allow permissions in s3 bucket go to the permissions tab in s3 bucket and in bucket policy change the action to this which will allow all actions to be performed:
"Action":"*"
Here's the policy that worked for me.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::bucket-name"
]
},
{
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::bucket-name/*"
]
}
]
}
I had a similar problem while trying to sync an entire s3 bucket locally. For me MFA (Multi-factor authentication) was enforced on my account, which is required while making commands via AWS CLI.
So the solution for me was - provide mfa credentials using a profile (mfa documentation) while using any AWS CLI commands.
Ran into a similar issues, for me the problem was that I had different AWS keys set in my bash_profile.
I answered a similar question here: https://stackoverflow.com/a/57317494/11871462
If you have conflicting AWS keys in your bash_profile, AWS CLI defaults to these instead.
I had this issue
my requirement i wanted to allow user to write to specific path
{
"Sid": "raspiiotallowspecificBucket",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::<bucketname>/scripts",
"arn:aws:s3:::<bucketname>/scripts/*"
]
},
and problem was solved with this change
{
"Sid": "raspiiotallowspecificBucket",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::<bucketname>",
"arn:aws:s3:::<bucketname>/*"
]
},
I like this better than any of the previous answers. It shows how to use the YAML format and lets you use a variable to specify the bucket.
- PolicyName: "AllowIncomingBucket"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action: "s3:*"
Resource:
- !Ref S3BucketArn
- !Join ["/", [!Ref S3BucketArn, '*']]
My issue was having set
env:
AWS_ACCESS_KEY_ID: {{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: {{ secrets.AWS_SECRET_ACCESS_KEY }}
again, under the aws-sync GitHub Action as environment variables. They were coming from my GitHub settings. Though in my case I had assumed a role in the previous step which would set me some new keys into those same name environment variables. So i was overwriting the good assumed keys with the bad GitHub basic keys.
Please take care of this if you're assuming roles.
I had the same issue. I had to provide the right resource and action, resource is your bucket's arn and action in your desired permission. Also please ensure you have your right user arn. Below is my solution.
{
"Version": "2012-10-17",
"Id": "Policy1546414123454",
"Statement": [
{
"Sid": "Stmt1546414471931",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789101:root"
},
"Action": ["s3:ListBucket", "s3:ListBucketVersions"],
"Resource": "arn:aws:s3:::bucket-name"
}
]
}
If you are suddenly getting this error on a new version of minio on buckets that used to work, the reason is that bucket access policy defaults were changed from version 2021 to 2022. Now in version 2022 by default all buckets (both newly created and existing ones) have Access Policy set to Private - it is not sufficient to provide server credentials to access them - you will still get errors such as these (here: returned to the python minio client):
S3Error: S3 operation failed; code: AccessDenied, message: Access Denied., resource: /dicts, request_id: 16FCBE6EC0E70439, host_id: 61486e5a-20be-42fc-bd5b-7f2093494367, bucket_name: dicts
To roll back to the previous security settings in version 2022, the quickest method is to change the bucket access Access Policy back to Public in the MinIO console (or via mc client).
This is not the best practice but this will unblock you.
Make sure for the user that's executing the command, it has the following policy attached to it under it's permission.
A. PowerUserAccess
B. AmazonS3FullAccess
I had faced same error "An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied"
Note:
Bucket policy not a good solution.
In IAM service create new custom policy attached with respective user would be safer.
Solved by below procedure:
IAM Service > Policies > Create Policy > select JSON >
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetObject",
"s3:GetObjectAcl",
"s3:ListBucketVersions"
],
"Resource": [
"arn:aws:s3:::<bucket name>"
]
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetObject",
"s3:GetObjectAcl",
"s3:PutObject",
"s3:DeleteObject",
"s3:ListBucketMultipartUploads",
"s3:ListMultipartUploadParts",
"s3:AbortMultipartUpload",
"s3:DeleteObjectVersion",
"s3:GetObjectVersion",
"s3:PutObjectACL",
"s3:ListBucketVersions"
],
"Resource": [
"arn:aws:s3:::<bucketname>/*"
]
}
]
}
Select Next Tag > Review Policy enter and create policy.
Select the newly created policy
Select the tab 'Policy Usage' in edit window of newly created policy window.
Select "Attach" select the user from the list and Save.
Now try in console with bucket name to list the objects, without bucket name it throws same error.
$aws s3 ls
A little late but might be helpful for someone. First thing first I am managing all access to s3 buckets using bucket policy.
My bucket policy to allow access to folder1 to IAM user: user1
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789012:user/user1"
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::s3-bucket/folder1",
"arn:aws:s3:::s3-bucket/folder1/*"
]
}
]
}
Now when user1 tries to perform list operation they get an error. It may look weird as the user has s3 full access from the bucket policy.
aws s3 ls s3://s3-bucket/folder1
aws s3 ls s3://s3-bucket/folder1/
aws s3 ls s3://s3-bucket/folder1/*
An error occurred (AccessDenied) when calling the ListObjectsV2
operation: Access Denied
Now let's take a look at the AWS documentation for ListBucket
Grants permission to list some or all of the objects in an Amazon S3
bucket (up to 1000)
To test that try to create a bucket policy and only provide the ListBucket permission for folder1 like this. Observe that you will get an error.
Conclusion
The ListBucket operation is only permitted for buckets and not for prefixes and hence if we want to provide list operation then it must be at the bucket level. Of course this will allow the user to list objects inside all other folders present in the bucket.
I have a client who I want to be able to upload files, but not navigate freely around my S3 bucket. I’ve created them an IAM user account, and applied the following policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1416387009000",
"Effect": "Allow",
"Action": [
"s3:ListAllMyBuckets"
],
"Resource": [
"arn:aws:s3:::*"
]
},
{
"Sid": "Stmt1416387127000",
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::progress"
]
},
{
"Sid": "Stmt1416387056000",
"Effect": "Allow",
"Action": [
"s3:AbortMultipartUpload",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::progress/*"
]
}
]
}
There are three statements:
Ability to list all buckets (otherwise they can’t see anything in the S3 console when they log in)
Ability to list the contents of the progress bucket
Ability to put objects in the progress bucket
The user can log in to the AWS console with their username and password (and my custom account URL, i.e. https://account.signin.aws.amazon.com/console). They can go to the S3 section of the console, and see a list of all my buckets. However, if they click progress then they just get the following error message:
Sorry! You were denied access to do that.
I’ve checked with the IAM Policy Simulator whether the user has the ListBucket permission on the bucket’s ARN (arn:aws:s3:::progress) and the Policy Simulator says the user should be allowed.
I’ve logged out and in again as the target user in case policies are only refreshed on log out, but still no joy.
What have I done wrong? Have I missed something?
My guess is that when using the AWS console another call is made to get the bucket location before it can list the objects in that bucket, and the user doesn't have permission to make that call. You need to also give he account access to GetBucketLocation. Relevant text from the documentation
When you use the Amazon S3 console, note that when you click a bucket,
the console first sends the GET Bucket location request to find the
AWS region where the bucket is deployed. Then the console uses the
region-specific endpoint for the bucket to send the GET Bucket (List
Objects) request. As a result, if users are going to use the console,
you must grant permission for the s3:GetBucketLocation action as shown
in the following policy statement:
{
"Sid": "RequiredByS3Console",
"Action": ["s3:GetBucketLocation"],
"Effect": "Allow",
"Resource": ["arn:aws:s3:::*"]
}