S3 Bucket Policy - GET Implicitly Allowed - amazon-web-services

Question

When using the following bucket policy, I see that it restricts PUT access as expected - however GET is allowed on the created object, even though there is nothing which should allow this operation.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowPut",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::<BUCKET>/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"<IP ADDRESS>"
]
}
}
}
]
}
I am able to PUT files to <BUCKET> from <IP ADDRESS> using curl as follows:
curl https://<BUCKET>.s3-<REGION>.amazonaws.com/ --upload-file test.txt
The file uploads successfully, and appears in the S3 console. I am now for some reason able to GET the file from anywhere on the internet.
curl https://<BUCKET>.s3-<REGION>.amazonaws.com/test.txt -XGET
This only applies for files uploaded using the above method. When uploading a file in the S3 web console, I am not able to use curl to GET it (access denied). So I assume that it is an object level permission issue. Though I don't understand why the bucket policy would not implicitly deny this access.
When looking at the object level permissions in the console, the only differences between a file uploaded through the console (method 1), and one uploaded from the allowed <IP ADDRESS> (method 2) are that the file in method 2 does not have an 'Owner', Permissions, or Metadata - while the method 1 file has all of these.
Furthermore - when attempting to GET the objects using a Lambda script (boto3 download_file()) which assumes a role with full access to the bucket, it fails for objects uploaded with method 2. Though it succeeds for objects uploaded with method 1.

Answer 1

Issue Summary
To summarise the issue:
you have a policy that permits anonymous upload of objects from a given source IP address
those objects are then not readable by your authenticated users (specifically an Iam Role adopted by your lambda function)
those objects ARE readable from ANY IP by unauthenticated users
Other observations
unauthenticated user is unable to delete the object
The desired outcome is:
objects can be uploaded by an unauthenticated user from a known IP address
objects are not then downloadable by unauthenticated users from any IP address
objects are retrievable by an authenticated Iam user
Root Cause
Here is what's happening:
Anonymous user uploads the object
The Anonymous user becomes the object owner
Verifiable by retrieving the object acl (do a GET request for the object with query string ?acl) - you will receive:
<?xml version="1.0" encoding="UTF-8"?>
<AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Owner>
<ID>65a011a29cdf8ec533ec3d1ccaae921c</ID>
</Owner>
<AccessControlList>
<Grant>
<Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"><ID>65a011a29cdf8ec533ec3d1ccaae921c</ID></Grantee>
<Permission>FULL_CONTROL</Permission>
</Grant>
</AccessControlList>
</AccessControlPolicy>
The Owner ID is the universal id of the anonymous user - I have seen the same id referenced in some AWS forum discussions.
Being the object owner has the following impact:
Anonymous user has FULL_CONTROL (see acl above)
Anonymous user is unable to Delete - this appears to be an AWS blanket rule that cannot be changed - the anonymous user is never allowed to delete anything, even if they have FULL_CONTROL
Anonymous user is, however, able to PUT an empty object over the top of the existing object, as a result of FULL_CONTROL
When a bucket contains a object owned by a user who is not part of the bucket's account:
Bucket owner has no permission on the object (not referenced in acl)
Bucket owner is not able to read the object
Bucket owner is able to see the object in a bucket list operation due to bucket acl
Bucket owner is able to delete the object - this is a blanket rule that cannot be changed - as the person paying the bill, you always reserve the right to delete the object - even if you can't read it
Resolution
There is a way to achieve your desired outcome - unfortunately you have to reference the arn of the specific Iam entity (user, role, group) you want to be able to read the object in the bucket acl.
The key elements of the solution are:
Require the anonymous user to grant the bucket owner full access
This ensures the bucket owner and owner account Iam users aren't denied access by the object acl
Explicitly deny all non-PUT access to all users who aren't your nominated user/role
This ensure anonymous users can't read the object
Sample policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "allow-anonymous-put",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::<BUCKETNAME>/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": "<IPADDRESS>"
},
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
},
{
"Sid": "deny-not-my-user-everything-else",
"Effect": "Deny",
"NotPrincipal": {
"AWS": "arn:aws:iam::<ACCOUNTNUMBER>:role/<ROLENAME>"
},
"NotAction": [
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": "arn:aws:s3:::<BUCKETNAME>/*"
}
]
}
The key to the second statement is the use of NotPrincipal and NotAction.
I've tested this locally, but only with a regular Iam user granted access, not with a Lamba function assuming a role - but the principal should hold. Good luck!
The following articles were helpful in understanding what was going on - they each present a scenario similar, but not quite the same as yours, but the methods they used to tackle their scenarios led the way:
http://jayendrapatil.com/aws-s3-permisions/
http://prettyplease.me/anonymous-s3-upload-with-full-owner-control/
https://gist.github.com/jareware/d7a817a08e9eae51a7ea