AWS S3 GetObject error on cross account access - amazon-web-services

I am the owner of AWS AccountC and need List and Get Permissions to BucketName owned by another person/team.
The bucket policy created is attached below. Policy for AccountA and AccountB were already existing and I added the policy for AccountC as given below
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AccessA",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::AccountA:root",
"arn:aws:iam::AccountA:user/ABC-Prod"
]
},
"Action": [
"s3:GetObject",
"s3:List*"
],
"Resource": [
"arn:aws:s3:::BucketName/*",
"arn:aws:s3:::BucketName"
]
},
{
"Sid": "AccessB",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::AccountB:user/service-user",
"arn:aws:iam::AccountB:role/BatchUserRole"
]
},
"Action": "*",
"Resource": [
"arn:aws:s3:::BucketName/*",
"arn:aws:s3:::BucketName"
]
},
{
"Sid": "AccessC",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountC:root"
},
"Action": "s3:List*",
"Resource": "arn:aws:s3:::BucketName"
},
{
"Sid": "AccessD",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountC:root"
},
"Action": "s3:Get*",
"Resource": "arn:aws:s3:::BucketName/*"
}
]
}
I am able to list contents of BucketName using
aws s3 ls BucketName.
However, when I try
aws s3 cp --recursive BucketName/folderName/ ., it gives me an Access Denied error
An error occurred (AccessDenied) when calling the GetObject operation: Access Denied
Block public access is enabled on the bucket, however I believe it should not affect since the Bucket policy is added
Tried multiple way to write the policy but the error persists. Can someone please help me understand what I might be missing here? Would be really grateful

Related

S3 Permission denied when using Athena

I'm trying to query an S3 bucket using Athena but I am getting the following error:
Permission denied on S3 path: s3://BUCKET_NAME/LOGS_LOCATION
This query ran against the "default" database, unless qualified by the query. Please post the error message on our forum or contact customer support with Query Id: f72e7dbf-929c-4096-bd29-b55c6c41f582
This bucket is created through an organizational level CloudTrail (initiated in Root account) that deployed the bucket in our Logging account.
Here is the bucket policy in the logging account:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowSSLRequestsOnly",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::<our bucket name>",
"arn:aws:s3:::<our bucket name>/*"
],
"Condition": {
"Bool": {
"aws:SecureTransport": "false"
}
}
},
{
"Sid": "AWSBucketPermissionsCheck",
"Effect": "Allow",
"Principal": {
"Service": [
"cloudtrail.amazonaws.com",
"config.amazonaws.com"
]
},
"Action": "s3:GetBucketAcl",
"Resource": "arn:aws:s3:::<our bucket name>"
},
{
"Sid": "AWSConfigBucketExistenceCheck",
"Effect": "Allow",
"Principal": {
"Service": [
"cloudtrail.amazonaws.com",
"config.amazonaws.com"
]
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::<our bucket name>"
},
{
"Sid": "AWSBucketDeliveryForConfig",
"Effect": "Allow",
"Principal": {
"Service": "config.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::<our bucket name>/<prefix>/AWSLogs/*/*"
},
{
"Sid": "AWSBucketDeliveryForOrganizationTrail",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": [
"arn:aws:s3:::<our bucket name>/<prefix>/AWSLogs/<root account>/*",
"arn:aws:s3:::<our bucket name>/<prefix>/AWSLogs/<prefix>/*"
]
}
]
}
Here are the steps I followed:
First I created an Athena Table in our logging account (same account where the bucket is located), the Table is built from the same logging bucket. I then created a destination bucket for Athena.I ran a simple "preview table" query to test if everything works. This is when I got the above mentioned error. The S3 bucket mentioned in the error is the logging bucket, not the Athena destination bucket.
I am thinking it might be an issue with the logging bucket's policy, please advise.
Thanks in advance!

fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden

I am trying to publish AWS VPC Flow Logs from account A to S3 bucket in another account B. I am able to do so but when i try to download the logs from account A, i am getting the error
fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden
My account A has IAM userA that i am using.
I am using the below command to download:
aws s3 cp s3://vpcflowlogscross/VPCLogs/{subscribe_account_id}/vpcflowlogs/{region_code}/2021/06/14/xyz.log.gz .
I have tried adding the region as well in the above command but no luck. Also while investigating this i came across documentation to add ListBucket permission which i already have.
My bucket policy in account B:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSLogDeliveryWrite",
"Effect": "Allow",
"Principal": {
"Service": "delivery.logs.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::vpcflowlogs-cross/*",
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
},
{
"Sid": "AWSLogDeliveryCheck",
"Effect": "Allow",
"Principal": {
"Service": "delivery.logs.amazonaws.com"
},
"Action": [
"s3:GetBucketAcl",
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::vpcflowlogs-cross"
},
{
"Sid": "DownloadObjects",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountA:user/userA"
},
"Action": [
"s3:GetObject",
"s3:GetObjectAcl",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::vpcflowlogs-cross",
"arn:aws:s3:::vpcflowlogs-cross/*"
]
]
}
I am using the below IAM user policy in account A to download the objects that are in account B S3 bucket.
My IAM user policy in account A:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "IAMAllowDownload",
"Effect": "Allow",
"Action": [
"s3:GetObjectAcl",
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::vpcflowlogs-cross",
"arn:aws:s3:::vpcflowlogs-cross/*"
]
}
]
}
I have tried changing the bucket and IAM policy but still not luck. I understand that the log objects getting published to the account B are owned by it but i am using the s3 "getobject" permission in bucket policy for the account A Principal userA shouldn't this allow me to download the objects.

AWS Lambda S3 Access Denied

I have a lambda function using a role with the following policy excerpt
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::ipwl-lambda-config/*",
"arn:aws:s3:::ipwl-lambda-config"
]
}
My bucket policy looks like the following
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "DenyUnEncryptedObjectUploads",
"Effect": "Deny",
"Principal": {
"AWS": "*"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::ipwl-lambda-config/*",
"Condition": {
"StringNotEquals": {
"s3:x-amz-server-side-encryption": "aws:kms"
}
}
},
{
"Sid": "AllowLambda",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::accountid:role/iam_for_lambda"
},
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::ipwl-lambda-config/*",
"arn:aws:s3:::ipwl-lambda-config"
]
}
]
}
I've allowed GetObject and ListBucket on both the role and the bucket policy. However when my function runs
s3_obj = s3_res.Object(s3_bucket, s3_object)
I get
[ERROR] ClientError: An error occurred (AccessDenied) when calling the
GetObject operation: Access Denied
What more permissions do I have to add? The object is there, I can get it when I run the code locally using an admin role.
Update
I've checked to make sure the bucket and object names are correct dozens of times. The exception is actually coming from the second line here according to the stacktrace
s3_res = boto3.resource('s3')
s3_obj = s3_res.Object(s3_bucket, s3_object)
data = s3_obj.get()['Body'].read()
KMS should only be a factor for PutObject. We have a support account so I may check with them and update with their findings.
To download a KMS-encrypted object from S3, you not only need to be able to get the object. You also need to be able to decrypt the AWS KMS key.
Here's an example of an IAM policy that your Lambda function should have:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "s3get",
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::ipwl-lambda-config/*"
},
{
"Sid": "kmsdecrypt",
"Effect": "Allow",
"Action": "kms:Decrypt",
"Resource": "arn:aws:kms:example-region-1:123456789012:key/example-key-id"
}
]
}
The key policy also needs to allow the IAM role to decrypt the key, something like this:
{
"Sid": "kmsdecrypt",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789012:role/xyz"
},
"Action": "kms:Decrypt",
"Resource": "*"
}

Lambda function to write into S3 - IAM policy to access S3

Here is my policy which grants read/write access still not able to write into S3 bucket
Problem
Still getting below error:
Failed to upload /tmp/test.txt to bucketname/Automation_Result_2019-07-09 04:20:32_.csv: An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
Policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ConsoleAccess",
"Effect": "Allow",
"Action": [
"s3:GetAccountPublicAccessBlock",
"s3:GetBucketAcl",
"s3:GetBucketLocation",
"s3:GetBucketPolicyStatus",
"s3:GetBucketPublicAccessBlock",
"s3:ListAllMyBuckets"
],
"Resource": "*"
},
{
"Sid": "ListObjectsInBucket",
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": [
"arn:aws:s3:::bucketname"
]
},
{
"Sid": "AllObjectActions",
"Effect": "Allow",
"Action": "s3:*Object",
"Resource": [
"arn:aws:s3:::bucketname/*"
]
}
]
}
Bucket policy
{
"Version": "2012-10-17",
"Id": "MYBUCKETPOLICY",
"Statement": [
{
"Sid": "DenyIncorrectEncryptionHeader",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::bucket-name/*",
"Condition": {
"StringNotEquals": {
"s3:x-amz-server-side-encryption": "aws:kms"
}
}
},
{
"Sid": "DenyUnEncryptedObjectUploads",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::bucket-name/*",
"Condition": {
"Null": {
"s3:x-amz-server-side-encryption": "true"
}
}
}
]
}
Python code (within Lambda function)
Relevant part of code
s3 = boto3.resource('s3', config=Config(signature_version='s3v4'))
target_bucket = 'bucket-name'
target_file = "Output/Automation_Result_"+EST+"_.txt"
s3.meta.client.upload_file('/tmp/test.txt', target_bucket, target_file, ExtraArgs={"ServerSideEncryption": "aws:kms", "SSEKMSKeyId":"XXXXXXX-XXXX-XXXX" })
This is how my bucket public access looks like!
It works fine for me!
I took your policy, renamed the bucket and attached it to a user as their only policy.
I was then able to successfully copy an object to and from the bucket.
If it is not working for you, then either you are not using the credentials that are associated with this policy, or there is another policy that is preventing the access, such as a Deny policy or a scope-limiting policy.

Amazon S3 file 'Access Denied' exception in Cross-Account

I've 2 AWS accounts. Account A has S3 bucket 'BUCKET' in which I've put file using Java api. I've configured my 'BUCKET' policy to allow cross-account file publishing.
But, when I try to open this file from Account A, it says AccessDeniedAccess Denied with hostId and requestId.
This file is published through Account B using java api, and this file has same size as that published through api. I tried to change file sizes and the new sizes were shown on AWS S3 console.
Here is my bucket policy:
{
"Version": "2008-10-17",
"Id": "Policy1357935677554",
"Statement": [
{
"Sid": "Stmt1357935647218",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ACCOUNTB:user/accountb-user"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::bucket-name"
},
{
"Sid": "Stmt1357935676138",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ACCOUNTB:user/accountb-user"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::bucket-name/*"
},
{
"Sid": "Stmt1357935676138",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ACCOUNTB:user/accountb-user"
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::bucket-name/*"
},
{
"Sid": "Stmt1357935647218",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ACCOUNTA:user/accounta-user"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::bucket-name"
},
{
"Sid": "Stmt1357935676138",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ACCOUNTA:user/accounta-user"
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::bucket-name/*"
},
{
"Sid": "Stmt1357935676138",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ACCOUNTA:root"
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::bucket-name/*"
}
]
}
The problem is when I try to download/open this file from Account A, I'm not able to open it.
The problem is that by default, when AWS (cli or SDK) upload a file it grants access to the uploader only through s3 ACLs.
In that case, to allow the owner to read the uploaded file, the uploader has to explicitly grant access to the owner of the bucket during the upload. Ex:
with the aws CLI (documentation here): aws s3api put-object --bucket <bucketname> --key <filename> --acl bucket-owner-full-control
with the nodejs API (documentation here): you have to set the params.ACL property of the AWS.S3.upload method to "bucket-owner-full-control"
In parallel, you can also ensure that the Bucket Owner Has Full Control with the bucket policy, (additional documentation here):
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Grant Owner Full control dev",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::ACCOUNTB:root"
},
"Action": [
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::bucket-name/*"
],
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
}
]
}