Migrate S3 bucket error - amazon-web-services

I want to migrate s3 bucket from one account to another account here is my bucket policy
{
"Version": "2008-10-17",
"Id": "Policy1335892530063",
"Statement": [
{
"Sid": "DelegateS3Access",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::xxxxxxxx:root"
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::test123",
"arn:aws:s3:::test123/*"
]
},
{
"Sid": "Stmt1335892150622",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::xxxxxxx:root"
},
"Action": [
"s3:GetBucketAcl",
"s3:GetBucketPolicy"
],
"Resource": "arn:aws:s3:::test123"
},
{
"Sid": "Stmt1335892526596",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::xxxxxxxxx:root"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::test123/*"
}
]
}
here is my IAM user policy
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": ["arn:aws:s3:::*"]
}
]
}
When I run command
aws s3 sync s3://test123 s3://abc-test123
I get Error
A client error (AccessDenied) occurred when calling the CopyObject operation: Access Denied

Your bucket policy seems to be correct.
Please verify that you are using root account, just as specified in your bucket policy.
Also you may need to check if there is not any denied bucket policies on your destination bucket.
If nothing helps, you can enable temporary public access to your bucket as a workaround. Yes, it's not secure but it should probably work in all cases.

Make sure you are providing adequate permissions on both the source bucket (to read) and the destination bucket (to write).
If you are using Root credentials (not generally recommended) for an Account that owns the bucket, you probably don't even need the bucket policy -- the root account should, by default, have the necessary access.
If you are assigning permissions to an IAM user, then instead of creating a Bucket Policy, assign permissions on the IAM user themselves. No need to supply a Principal in this situation.
Start by checking that you have permissions to list both buckets:
aws s3 ls s3://test123
aws s3 ls s3://abc-test123
Then check that you have permissions to copy a file from the source and to the destination:
aws s3 cp s3://test123/foo.txt .
aws s3 cp foo.txt s3://abc-test123/foo.txt
If they work, then the sync command should work, too.

Related

(AWS S3) Show File Ownership

We are trying to copy/move s3 bucket files that were originally transferred to our bucket from another AWS account.
However, when we try to move these file contents with aws s3 cp command we get: fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden
I think the problem is that someone copied this data over from another account without using --acl bucket-owner-full-control. Do you know if there is a way for us to go through and list file owners via the CLI or boto3? Maybe a recursive call to the bucket showing all object owners to each file? Or find anything that isn't owned by our account?
Current permissions of our bucket:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::account-user-id:root"
]
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::customers"
},
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::account-user-id:root"
]
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::customers/*"
}
]
}
A recent feature in Amazon S3 allows you to override the ownership settings:
If you configure the bucket with ACLs disabled, then you should immediately regain access to all of the objects.

Uploading to AWS S3 bucket from a profile in a different environment

I have access to one of two AWS environments and I've created a protected S3 bucket in it to upload files to from an account in the one that I do not. The environment and the account that I don't have access to are what a project's CI uses.
environment I have access to: env1
environment I do not have access to: env2
account I do not have access to: user/ci
bucket name: content
S3 bucket policy:
{
"Version": "2008-10-17",
"Id": "PolicyForCloudFrontPrivateContent",
"Statement": [
{
...
},
{
"Sid": "Allow access to bucket from profile in env1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::111122223333:user/ci"
},
"Action": [
"s3:GetBucketLocation",
"s3:ListBucket*"
],
"Resource": "arn:aws:s3:::content"
},
{
"Sid": "Allow access to bucket items from profile in env1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::111122223333:user/ci"
},
"Action": [
"s3:Get*",
"s3:PutObject",
"s3:ListMultipartUploadParts"
],
"Resource": [
"arn:aws:s3:::content",
"arn:aws:s3:::content/*"
]
}
]
}
From inside a container that's configured for env1 and user/ci I'm testing with the command
aws s3 sync content/ s3://content/
and I get the error:
fatal error: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied
I have two questions:
Am I even using the correct aws command to upload the data to the bucket?
Am I missing something from my bucket policy?
For the latter, I've basically followed what a load of examples and answers online have suggested.
To test your policy, I did the following:
Created an IAM User with no policies
Created an Amazon S3 bucket
Attached your Bucket Policy to the bucket, and updated the ARN and bucket name
Tested access to the bucket with:
aws s3 ls s3://bucketname
aws s3 sync folder/ s3://bucketname/folder/
It worked fine.
Therefore, the policy you display appears to be giving all necessary permissions. It is possible that you have something else that is Denying access on the bucket.
The solution was to given the ACL
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::content",
"arn:aws:s3:::content/*"
]
}
]
}
to user/ci in env1.

Messed up bucket policy and not able to access the bucket anymore

I created an s3 bucket and gave the following bucket policy to it
{
"Version": "2012-10-17",
"Id": "Policy1586942250763",
"Statement": [
{
"Sid": "Stmt1586942249918",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::accID:user/username1”
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::bucket-name"
}
]
}
Obviously this would give access only to the user username1. Now username1 was deleted from IAM. Is there any way to recover access to this bucket. I have an IAM account with admin privileges but aws s3api delete-bucket-policy111 seems to be not working.
Only root user can do anything on that bucket now.

AWS S3: An error occurred (AccessDenied) when calling the GetObject operation: Access Denied

I have an AWS account with read/write permissions as shown below:
I'd like to make it so that an IAM user can download files from an S3 bucket but I'm getting access denied when executing aws s3 sync s3://<bucket_name> . I have tried various things, but not to avail. Some steps that I did:
Created a user called s3-full-access
Executed aws configure in my CLI and entered the generated access key id and secret access key for the above user
Created a bucket policy (shown below) that I'd hoped grants access for my user created in first step.
My bucket has a folder name AffectivaLogs in which files were being added anonymously by various users, and it seems like though the bucket is public, the folder inside it is not and I am not even able to make it public, and it leads to following error.
Following are the public access settings:
Update: I updated the bucket policy as follows, but it doesn't work.
To test the situation, I did the following:
Created an IAM User with no attached policies
Created an Amazon S3 bucket
Turned off S3 block public access settings:
Block new public bucket policies
Block public and cross-account access if bucket has public policies
Added a Bucket Policy granting s3:* access to the contents of the bucket for the IAM User
I then ran aws s3 sync and got Access Denied.
I then modified the policy to also permit access to the bucket itself:
{
"Id": "Policy",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "statement",
"Action": "s3:*",
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::my-bucket/*",
"arn:aws:s3:::my-bucket"
],
"Principal": {
"AWS": [
"arn:aws:iam::123456789012:user/stack-user"
]
}
}
]
}
This worked.
Bottom line: Also add permissions to access the bucket, in addition to the contents of the bucket. (I suspect it is because aws s3 sync requires listing of bucket contents, in addition to accessing the objects themselves.)
If you use KMS encryption enabled on bucket you should also add policy that allows you to decrypt data using KMS key.
You can configure the S3 policy with the required principal
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ListBucket",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::accountId:user/*
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::bucket"
},
{
"Sid": "GetObjects",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::accountId:user/*
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::bucket/*"
}
]
}
Or you can create IAM policy and attached it to the role
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ListBucket",
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::bucket"
},
{
"Sid": "GetObject",
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": "arn:aws:s3:::bucket/*"
}
]
}

Cross account role granting S3 bucket access - Permission Denied

I have two accounts, account ACCOUNTAAAA, and ACCOUNTBBBB. A bucket (BUCKETAAAA) exists in account ACCOUNTAAAA and an instance with a role (ROLEBBBB) in ACCOUNTBBBB needs to be able to read from it.
I have adding the following permissions to the bucket:
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ACCOUNTBBBB:role/ROLEBBBB"
},
"Action": [
"s3:*"
],
"Resource": "arn:aws:s3:::BUCKETAAAA/*"
}
I have the following permissions on the role in ACOUNTBBBB:
"Sid": "",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::BUCKETAAAA/",
"arn:aws:s3:::BUCKETAAAA/*"
]
}
My assumption is I should be able to run aws s3 ls s3://BUCKETAAAA on the EC2 instance with the IAM role attached, and see the contants of BUCKETAAAA. When I try this, I get An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied.
What am I missing here?
To reproduce your situation, I did the following:
Created Role-B in Account-B with EC2 as the Trusted Entity ("Allows EC2 instances to call AWS services on your behalf") and a policy granting access to Bucket-A
Created Bucket-A in Account-A
Added a Bucket Policy to Bucket-A, which grants access to Role-B
Assigned Role-B to an Amazon EC2 instance
The Bucket Policy on Bucket-A was:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AddPerm",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<account>:role/role-b"
},
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::bucket-a",
"arn:aws:s3:::bucket-a/*"
]
}
]
}
The permissions on Role-B were:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BucketA",
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::bucket-a",
"arn:aws:s3:::bucket-a/*"
]
}
]
}
I was able to successfully use aws s3 ls s3://bucket-a and was able to copy files to Bucket-A.
The main difference I see with your attempt was that your Bucket Policy only granted permissions for:
"Resource": "arn:aws:s3:::BUCKETAAAA/*"
This means "anything within Bucket-A" but does not include Bucket-A itself. The command aws s3 ls s3://BUCKETAAAA operates on the bucket, for which the role has no permissions.
Thus, you should also add permissions for the bucket itself:
"Resource": [
"arn:aws:s3:::BUCKETAAAA",
"arn:aws:s3:::BUCKETAAAA/*"