I have access to one of two AWS environments and I've created a protected S3 bucket in it to upload files to from an account in the one that I do not. The environment and the account that I don't have access to are what a project's CI uses.
environment I have access to: env1
environment I do not have access to: env2
account I do not have access to: user/ci
bucket name: content
S3 bucket policy:
{
"Version": "2008-10-17",
"Id": "PolicyForCloudFrontPrivateContent",
"Statement": [
{
...
},
{
"Sid": "Allow access to bucket from profile in env1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::111122223333:user/ci"
},
"Action": [
"s3:GetBucketLocation",
"s3:ListBucket*"
],
"Resource": "arn:aws:s3:::content"
},
{
"Sid": "Allow access to bucket items from profile in env1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::111122223333:user/ci"
},
"Action": [
"s3:Get*",
"s3:PutObject",
"s3:ListMultipartUploadParts"
],
"Resource": [
"arn:aws:s3:::content",
"arn:aws:s3:::content/*"
]
}
]
}
From inside a container that's configured for env1 and user/ci I'm testing with the command
aws s3 sync content/ s3://content/
and I get the error:
fatal error: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied
I have two questions:
Am I even using the correct aws command to upload the data to the bucket?
Am I missing something from my bucket policy?
For the latter, I've basically followed what a load of examples and answers online have suggested.
To test your policy, I did the following:
Created an IAM User with no policies
Created an Amazon S3 bucket
Attached your Bucket Policy to the bucket, and updated the ARN and bucket name
Tested access to the bucket with:
aws s3 ls s3://bucketname
aws s3 sync folder/ s3://bucketname/folder/
It worked fine.
Therefore, the policy you display appears to be giving all necessary permissions. It is possible that you have something else that is Denying access on the bucket.
The solution was to given the ACL
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::content",
"arn:aws:s3:::content/*"
]
}
]
}
to user/ci in env1.
Related
I am stuck with provisioning end-user access into a cross account shared bucket, and need help figuring out if there are specific policy requirements for using clients to access the bucket, vs straight CLI.
IAM User Accounts are managed in our "Core" AWS Account.
S3 Bucket is provisioned in our "Dev" AWS Account.
S3 Bucket in Dev account is encrypted with KMS key in Dev Account.
We have configured our Bucket Policy to permit the user access.
We have configured user policies to permit access to the S3 bucket.
We have configured user policies to permit use of the KMS key.
When using the CLI our user account can succesfully access and use the S3 bucket. When attempting to connect with a GUI Client (Win-SCP, CyberDuck, MAC ForkLift) we receive permission denied errors.
BUCKET POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ListObjectsInBucket",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::[DEVACCOUNT#]:role/EC2-ROLE-FOR-APP-ACCESS",
"arn:aws:iam::[COREACCOUNT#]:user/end.user"
]
},
"Action": "s3:List*",
"Resource": [
"arn:aws:s3:::dev-mybucket",
"arn:aws:s3:::dev-mybucket/*"
]
},
{
"Sid": "AllObjectActions",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::[DEVACCOUNT#]:role/EC2-ROLE-FOR-APP-ACCESS",
"arn:aws:iam::[COREACCOUNT#]:user/end.user"
]
},
"Action": [
"s3:GetObject",
"s3:Put*"
],
"Resource": "arn:aws:s3:::dev-mybucket/*"
}
]
}
User Policy - access KMS
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowUseOfDevAPPSKey",
"Effect": "Allow",
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:Describe*"
],
"Resource": [
"arn:aws:kms:ca-central-1:[DEVACCOUNT#]:key/[redacted-key-number]"
]
},
{
"Sid": "AllowAttachmentOfPersistentResources",
"Effect": "Allow",
"Action": [
"kms:CreateGrant",
"kms:List*",
"kms:RevokeGrant"
],
"Resource": [
"arn:aws:kms:ca-central-1:[DEVACCOUNT#]:key/[redacted-key-number]"
],
"Condition": {
"Bool": {
"kms:GrantIsForAWSResource": true
}
}
}
]
}
User policy - Access S3 Bucket
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowAccessToMyBucket",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::dev-mybucket/",
"arn:aws:s3:::dev-mybucket/*"
]
}
]
}
From aws s3 commands we can 'ls' content and 'cp' content from local to remote and from remote to local.
When configuring access with the GUI Clients we always receive somewhat generic 'permission denied' or 'access denied' type errors.
The GUI client is probably making a call that is not List*, Put* or GetObject.
For example, it might be calling GetObjectVersion, GetObjectAcl or GetBucketAcl.
Try adding Get* permissions in addition to List*.
You might also be able to look at the events in your AWS CloudTrail trail to see what specific API calls were denied.
For details, see: Specifying Permissions in a Policy - Amazon Simple Storage Service
Access to an S3 bucket via a GUI such as the AWS web console or SFTP clients with s3 functionality(FileZilla, Cyberduck, ForkLift, etc.) requires the s3:ListAllMyBuckets action in a policy attached to that IAM user. This is very unfortunate as the user will now have access to see ALL your bucket names in that account even if they just have read, write, and or List access to just one bucket in that account.
https://docs.aws.amazon.com/AmazonS3/latest/API/API_Operations.html
https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListBuckets.html
One other option is to go to the bucket URL directly. The user/role will require access via that bucket's Bucket policy.
https://s3.console.aws.amazon.com/s3/buckets/dev-mybucket
I have an AWS account with read/write permissions as shown below:
I'd like to make it so that an IAM user can download files from an S3 bucket but I'm getting access denied when executing aws s3 sync s3://<bucket_name> . I have tried various things, but not to avail. Some steps that I did:
Created a user called s3-full-access
Executed aws configure in my CLI and entered the generated access key id and secret access key for the above user
Created a bucket policy (shown below) that I'd hoped grants access for my user created in first step.
My bucket has a folder name AffectivaLogs in which files were being added anonymously by various users, and it seems like though the bucket is public, the folder inside it is not and I am not even able to make it public, and it leads to following error.
Following are the public access settings:
Update: I updated the bucket policy as follows, but it doesn't work.
To test the situation, I did the following:
Created an IAM User with no attached policies
Created an Amazon S3 bucket
Turned off S3 block public access settings:
Block new public bucket policies
Block public and cross-account access if bucket has public policies
Added a Bucket Policy granting s3:* access to the contents of the bucket for the IAM User
I then ran aws s3 sync and got Access Denied.
I then modified the policy to also permit access to the bucket itself:
{
"Id": "Policy",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "statement",
"Action": "s3:*",
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::my-bucket/*",
"arn:aws:s3:::my-bucket"
],
"Principal": {
"AWS": [
"arn:aws:iam::123456789012:user/stack-user"
]
}
}
]
}
This worked.
Bottom line: Also add permissions to access the bucket, in addition to the contents of the bucket. (I suspect it is because aws s3 sync requires listing of bucket contents, in addition to accessing the objects themselves.)
If you use KMS encryption enabled on bucket you should also add policy that allows you to decrypt data using KMS key.
You can configure the S3 policy with the required principal
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ListBucket",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::accountId:user/*
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::bucket"
},
{
"Sid": "GetObjects",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::accountId:user/*
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::bucket/*"
}
]
}
Or you can create IAM policy and attached it to the role
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ListBucket",
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::bucket"
},
{
"Sid": "GetObject",
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": "arn:aws:s3:::bucket/*"
}
]
}
In CodeBuild, I have 2 projects. One is for a staging site, and another one is for a production site. When I compile my site, and run it through the staging project, it works fine. It sync's successfully to my s3 bucket for the staging site. However, when tried to compile it and run it through the production project, when running the sync command, it returns an error :
fatal error: An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied
[Container] 2018/09/11 08:40:33 Command did not exit successfully aws s3 sync public/ s3://$S3_BUCKET exit status 1
I did some digging around, and I think the problem is with my bucket policy. I am using CloudFront as a CDN on top of my S3 bucket. I don't want to modify the bucket policy of the production bucket right until I'm absolutely sure that I must. I'm worried it might have some affect on the live site.
Here is my bucket policy for the production bucket:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadGetObject",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::[bucket_name]/*"
},
{
"Sid": "2",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity [access_code]"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::[bucket_name]/*"
}
]
}
As per the error description, the list permission is missing.
Add the below permission at your bucket policy:
"Action": [
"s3:Get*",
"s3:List*"
]
This should solve your issue. Also check the IAM service role created on codebuild to access S3 buckets. The S3 bucket policy and IAM role both control the access to the S3 bucket in this kind of setup.
Your service role should have list permission for S3.
{
"Sid": "S3ObjectPolicy",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:List*"
],
"Resource": ["arn:aws:s3:::my_bucket",
"arn:aws:s3:::my_bucket/*"]
}
I have two accounts, account ACCOUNTAAAA, and ACCOUNTBBBB. A bucket (BUCKETAAAA) exists in account ACCOUNTAAAA and an instance with a role (ROLEBBBB) in ACCOUNTBBBB needs to be able to read from it.
I have adding the following permissions to the bucket:
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ACCOUNTBBBB:role/ROLEBBBB"
},
"Action": [
"s3:*"
],
"Resource": "arn:aws:s3:::BUCKETAAAA/*"
}
I have the following permissions on the role in ACOUNTBBBB:
"Sid": "",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::BUCKETAAAA/",
"arn:aws:s3:::BUCKETAAAA/*"
]
}
My assumption is I should be able to run aws s3 ls s3://BUCKETAAAA on the EC2 instance with the IAM role attached, and see the contants of BUCKETAAAA. When I try this, I get An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied.
What am I missing here?
To reproduce your situation, I did the following:
Created Role-B in Account-B with EC2 as the Trusted Entity ("Allows EC2 instances to call AWS services on your behalf") and a policy granting access to Bucket-A
Created Bucket-A in Account-A
Added a Bucket Policy to Bucket-A, which grants access to Role-B
Assigned Role-B to an Amazon EC2 instance
The Bucket Policy on Bucket-A was:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AddPerm",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<account>:role/role-b"
},
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::bucket-a",
"arn:aws:s3:::bucket-a/*"
]
}
]
}
The permissions on Role-B were:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BucketA",
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::bucket-a",
"arn:aws:s3:::bucket-a/*"
]
}
]
}
I was able to successfully use aws s3 ls s3://bucket-a and was able to copy files to Bucket-A.
The main difference I see with your attempt was that your Bucket Policy only granted permissions for:
"Resource": "arn:aws:s3:::BUCKETAAAA/*"
This means "anything within Bucket-A" but does not include Bucket-A itself. The command aws s3 ls s3://BUCKETAAAA operates on the bucket, for which the role has no permissions.
Thus, you should also add permissions for the bucket itself:
"Resource": [
"arn:aws:s3:::BUCKETAAAA",
"arn:aws:s3:::BUCKETAAAA/*"
I want to migrate s3 bucket from one account to another account here is my bucket policy
{
"Version": "2008-10-17",
"Id": "Policy1335892530063",
"Statement": [
{
"Sid": "DelegateS3Access",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::xxxxxxxx:root"
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::test123",
"arn:aws:s3:::test123/*"
]
},
{
"Sid": "Stmt1335892150622",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::xxxxxxx:root"
},
"Action": [
"s3:GetBucketAcl",
"s3:GetBucketPolicy"
],
"Resource": "arn:aws:s3:::test123"
},
{
"Sid": "Stmt1335892526596",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::xxxxxxxxx:root"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::test123/*"
}
]
}
here is my IAM user policy
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": ["arn:aws:s3:::*"]
}
]
}
When I run command
aws s3 sync s3://test123 s3://abc-test123
I get Error
A client error (AccessDenied) occurred when calling the CopyObject operation: Access Denied
Your bucket policy seems to be correct.
Please verify that you are using root account, just as specified in your bucket policy.
Also you may need to check if there is not any denied bucket policies on your destination bucket.
If nothing helps, you can enable temporary public access to your bucket as a workaround. Yes, it's not secure but it should probably work in all cases.
Make sure you are providing adequate permissions on both the source bucket (to read) and the destination bucket (to write).
If you are using Root credentials (not generally recommended) for an Account that owns the bucket, you probably don't even need the bucket policy -- the root account should, by default, have the necessary access.
If you are assigning permissions to an IAM user, then instead of creating a Bucket Policy, assign permissions on the IAM user themselves. No need to supply a Principal in this situation.
Start by checking that you have permissions to list both buckets:
aws s3 ls s3://test123
aws s3 ls s3://abc-test123
Then check that you have permissions to copy a file from the source and to the destination:
aws s3 cp s3://test123/foo.txt .
aws s3 cp foo.txt s3://abc-test123/foo.txt
If they work, then the sync command should work, too.