AWS S3 upload fails with ACL public-true - amazon-web-services

I have this simple script in Ruby to upload a file on S3 I got from AWS documentation:
def object_uploaded?(s3_resource, bucket_name, object_key, file_path)
object = s3_resource.bucket(bucket_name).object(object_key)
File.open(file_path, 'rb') do |file|
object.put(body: file, acl: 'public-read')
end
return true
rescue StandardError => e
puts "Error uploading object: #{e.message}"
return false
end
The script gives me Access Denied when i add acl: 'public-read'. Works fine if I remove that.
Only way for this to work is to make my S3 bucket public.
This is my bucket policy:
{
"Version": "2012-10-17",
"Id": "Policy1610635552932",
"Statement": [
{
"Sid": "Stmt1610635551842",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::[mybucket]/*"
}
]
}
And this is my IAM policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"s3:ListStorageLensConfigurations",
"s3:GetAccessPoint",
"s3:PutAccountPublicAccessBlock",
"s3:GetAccountPublicAccessBlock",
"s3:ListAllMyBuckets",
"s3:ListAccessPoints",
"s3:ListJobs",
"s3:PutStorageLensConfiguration",
"s3:CreateJob"
],
"Resource": "*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::[mybucket]",
"arn:aws:s3:::[mybucket]/*"
]
}
]
}
Is there any way I can turn on Block public access as recommended by AWS and have public accessible files by setting the acl? What is wrong with my policy?

You cannot upload a file with ACL public-read if the bucket's block public access setting is on. Per the documentation: "Amazon S3 block public access prevents the application of any settings that allow public access to data within S3 buckets." (https://docs.aws.amazon.com/AmazonS3/latest/user-guide/block-public-access.html)

Related

cross account S3 bucket replication via replication rules

I have two buckets:
"source-bucket" (in AWS account 88888888).
"destination-bucket" (in AWS account 99999999)
Both buckets have versioning enabled and are located in the same region (eu-west-1).
In the source bucket, I've created a Replication-rule with the following settings:
I opted for automatic role creation, which created a role with the following policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:ListBucket",
"s3:GetReplicationConfiguration",
"s3:GetObjectVersionForReplication",
"s3:GetObjectVersionAcl",
"s3:GetObjectVersionTagging",
"s3:GetObjectRetention",
"s3:GetObjectLegalHold"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::source-bucket",
"arn:aws:s3:::source-bucket/*",
"arn:aws:s3:::destination-bucket",
"arn:aws:s3:::destination-bucket/*"
]
},
{
"Action": [
"s3:ReplicateObject",
"s3:ReplicateDelete",
"s3:ReplicateTags",
"s3:ObjectOwnerOverrideToBucketOwner"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::source-bucket/*",
"arn:aws:s3:::destination-bucket/*"
]
}
]
}
According to the documentation found here https://docs.aws.amazon.com/AmazonS3/latest/userguide/replication-walkthrough-2.html
, I've added a bucket policy to "destination-bucket", which looks as follows:
{
"Version": "2012-10-17",
"Id": "",
"Statement": [
{
"Sid": "Set permissions for objects",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::88888888:role/service-role/auto-created-role"
},
"Action": [
"s3:ReplicateObject",
"s3:ReplicateDelete"
],
"Resource": "arn:aws:s3:::destination-bucket/*"
},
{
"Sid": "Set permissions on bucket",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::88888888:role/service-role/auto-created-role"
},
"Action": [
"s3:GetBucketVersioning",
"s3:PutBucketVersioning"
],
"Resource": "arn:aws:s3:::destination-bucket"
}
]
}
But, when I add a file to the source bucket, nothing seems happens.
Does anyone have any idea what could be wrong here?
The AWS docs aren't the best here. From your pictures I see you have enabled the setting "change object ownership to destination bucket owner" (as most people would).
However, this requires an extra permission on the destination side give them ownership. s3:ObjectOwnerOverrideToBucketOwner
The following policy should work for you
{
"Version": "2012-10-17",
"Id": "",
"Statement": [
{
"Sid": "Set permissions for objects",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::88888888:role/service-role/auto-created-role"
},
"Action": [
"s3:ReplicateObject",
"s3:ReplicateDelete",
"s3:ObjectOwnerOverrideToBucketOwner"
],
"Resource": "arn:aws:s3:::destination-bucket/*"
},
{
"Sid": "Set permissions on bucket",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::88888888:role/service-role/auto-created-role"
},
"Action": [
"s3:GetBucketVersioning",
"s3:PutBucketVersioning"
],
"Resource": "arn:aws:s3:::destination-bucket"
}
]
}
To debug this issue I used aws s3api head-object --bucket <bucket> --key <prefix> --query ReplicationStatus to see the replication failed and then I added s3:* permission on the destination side to see if it was a permission issue. Which in this case it was.
Check if this helps.
By default, Amazon S3 doesn't replicate objects that are stored at rest using server-side encryption with AWS Key Management Service (AWS KMS) customer master keys (CMKs). To replicate encrypted objects, you modify the bucket replication configuration to tell Amazon S3 to replicate these objects.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/replication-walkthrough-4.html

AWS CrossAccount S3 PutObject from Lambda in another account

I am trying to put a text file from Lambda which is in Account A to S3 bucket in account B. S3 bucket(test-bucket) is having AWS-KMS encryption enabled. I added below permissions :
Added below bucket policy to S3 bucket in Account B:
{"Version": "2012-10-17",
"Id": "ExamplePolicy",
"Statement": [
{
"Sid": "ExampleStmt",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountA:role/Lambda-Role"
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::test-bucket/*"
}
]
}
Added below policy in KMS key:
"Version": "2012-10-17",
"Id": "key-default-1",
"Statement": [
{
"Sid": "Enable IAM User Permissions",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountB:root"
},
"Action": "kms:*",
"Resource": "*"
},
{
"Sid": "Allow use of the key",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::AccountA:role/Lambda-Role"
},
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey"
],
"Resource": "*"
}
]
}
Added below Inline policy in Account A - Lambda Role and gave access to KMS key:
{"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"kms:Decrypt",
"kms:Encrypt",
"kms:GenerateDataKey",
"kms:DescribeKey",
"kms:ReEncrypt*"
],
"Resource": [
"arn:aws:kms:us-west-2:AccountB:key/KMS-ID"
]
}
]
}
Files are also uploading in Account B S3 Bucket but not able to view/download any of those files. Gets this error:
<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>5H3KEXCJ7YSCJS</RequestId>
<HostId>hqwavZZo6D0asdddcvfff+prEtoBCwTFH0AYtzzzzzztqAaPflzs85aaaaa=</HostId>
</Error>
When I checks the file properties it has : Server-side encryption- Access denied.
Don't know what am I missing here. Someone please guide.
One thing missing in Account A - Lambda Role is - it should have permission to access the bucket in account B even though the bucket policy in Account-B allows it.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::AccountABucketName"
"arn:aws:s3:::AccountABucketName/*"
]
}
]
}
And to List the files in the bucket you should also add "Resource": "arn:aws:s3:::test-bucket as well
I found the solution. I only needed to add ACL='bucket-owner-full-control' in the put_object. Below is the complete boto3 cmd.
s3.put_object(
ACL='bucket-owner-full-control'
Body=processed_content,
Bucket=processed_bucket,
Key=processed_key)

Amazon CLI won't allow me to list objects

I'm trying to copy contents of an S3 bucket to another bucket on another account, and I wanted to use the CLI to do this. So I set up a bucket policy on the source bucket, allowing a IAM user in the destination account to perform all S3 actions, but it keeps complaining that the ListObjects operation is denied.
I've tried Google, but I can't tell what would be the problem with my policy compared to the solutions I find. Even if I make the source bucket public (and can list it in a browser), it still gives me access denied.
What to do, what to do? Here's my bucket policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowAll",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123123123123:user/USER"
},
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::SOURCE",
"arn:aws:s3:::SOURCE/*"
]
}
]
}
Please try using below policy,
{
"Version": "2008-10-17",
"Id": "Policy1357935677554",
"Statement": [
{
"Sid": "CrossAccountList",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::111111111111:root"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::examplebucket"
},
{
"Sid": "CrossAccountS3",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::111111111111:root"
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::examplebucket/*"
}
]
}
You can read the full steps here
Another read here

Can't use sync to move objects in a folder to another folder within the same bucket using s3cmd

I have been trying to move all the objects in a folder bucketA/product/pic/ up one level within the same bucket bucketA/pic/
I can sync files between local host and the s3 server with
s3cmd sync /script/ s3://bucketA/
as well as put an object:
s3cmd put zip.sh s3://bucketA/
But I'm getting Access Denied error when syncing files within the same bucket:
[root]s3cmd sync s3://bucketA/product/pic s3://bucketA/pic/
WARNING: Empty object name on S3 found, ignoring.
Summary: 441 source files to copy, 0 files at destination to delete
ERROR: S3 error: Access Denied
Is it possible to change the locations of the objects in a folder within the same bucket?
Here's my IAM policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt123456",
"Effect": "Allow",
"Action": [
"s3:ListAllMyBuckets"
],
"Resource": [
"arn:aws:s3:::*"
]
},
{
"Sid": "Stmt123457",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:PutObject",
"s3:PutObjectAcl",
"s3:GetBucketLocation"
],
"Resource": [
"arn:aws:s3:::bucketA",
"arn:aws:s3:::bucketA/*"
]
}
]
}
Here's my bucket policy which is set to prevent hotlinking:
{
"Version": "2012-10-17",
"Id": "HTTP referrer policy",
"Statement": [
{
"Sid": "Allow in my domains",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::bucketA/*",
"Condition": {
"StringLike": {
"aws:Referer": [
"https://mylocalhostip/*",
"http://mylocalhostip/*"
]
}
}
},
{
"Sid": "Deny access if referer is not my sites",
"Effect": "Deny",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::bucketA/*",
"Condition": {
"StringNotLike": {
"aws:Referer": [
"http://mylocalhostip/*",
"https://mylocalhostip/*"
]
}
}
}
]
}
These days, it is recommended to use the AWS Command-Line Interface (CLI).
The AWS CLI includes a sync command. See: https://docs.aws.amazon.com/cli/latest/reference/s3/sync.html

On Amazon S3 I'm able to give an individual object download rights but cannot for the entire bucket

I've got images on S3 that I'd like to share with my Rails app. I can get access to the images by setting the permission: Everybody - Open/Download on the S3 UI. I want to set this permission for all objects in the bucket. The bucket permission UI doesn't have the option of Open/Download for users. What is the action I need to assign in the bucket config command to allow this? Here is my current command:
{
"Version": "2008-10-17",
"Id": "anid",
"Statement": [
{
"Sid": "ansid",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::haggard/akey"
}
]
}
Your policy is probably wrong, this one is working fine for Everyone (Anonymous) users:
{
"Id": "Policy1416915063227",
"Statement": [
{
"Sid": "AddCannedAcl",
"Action": [
"s3:GetObject"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::examplebucket/*",
"Principal": {
"AWS": [
"*"
]
}
}
]
}
I recommend to use the policy generator at the following link:
http://awspolicygen.s3.amazonaws.com/policygen.html