Amazon S3 Policy Prevent Creation of Root-level Folders - amazon-web-services

Question

We're looking to prevent our S3 users from creating new folders in our bucket's root. In other words, they must use existing folders within the bucket to upload or modify files. They may choose to create subfolders in these existing folders if they'd like.
Note: Using S3 policies. Users choose any existing folder. They do not have assigned folders.
I know S3 treats both files and folders as objects so I'm not sure this can even be done, but I believe in the community's potential.
Here's what I want:
Bucket-name: test-bucket
Action: Create folder in test-bucket's root.
Desired Result: Denied
Action: Upload random file in test-bucket's root.
Desired Result: Denied
Action: Upload file "file1" in test-bucket's existing "folder1" folder (test-bucket/folder1/file1).
Desired Result: Success
Action: Create folder "sub-folder1" in test-bucket's existing "folder1" folder (test-bucket/folder1/sub-folder1/).
Desired Result: Success

Answer 1

There is a flaw in your conceptual model.
I know S3 treats both files and folders as objects
That isn't correct.
Here's the correct version:
The S3 service and API have no concept of folders.
S3 objects are not in any real sense hierarchical.
The S3 console is the only entity with a concept of folders.
The S3 service and API support a concept of prefix, delimiter, and common prefixes.
When a List Objects request to the API is accompanied by a specified prefix, only objects with keys beginning with that prefix are returned, regardless of any / in their object key after the prefix.
When a List Objects request is accompanied a prefix and also by a delimiter (usually /), the API only returns objects whose keys match the given prefix and which have no subsequent / (after the prefix specified in the request) in the key. These are analogous to the "files in the folder."
The prefixes of the keys of any objects matching the given prefix but which do have a subsequent (after the specified prefix) / are coalesced down to a unique list of their prefixes, truncated to their next /. These are the common prefixes, analogous to "folders in the folder."
But, in fact, nothing is really "in" anything else.
The console creates an illusion of folders by reading the common prefixes from a list objects request to the API, and showing these as folders.
The console furthers the illusion by allowing you to "create a folder" -- but it is not in fact a folder, and it isn't even needed. It is simply an empty object with a key whose last character is /. This object is not needed for normal operation of S3, but is created as a convenience, so that you can navigate "into" an "empty folder" and upload a file "into" the empty folder.
However, what's really happening is this:
Console: "create folder foo in the root of the bucket"
API: PUT /foo/
Content-Length: 0
Console: "click folder foo"
API: GET /?prefix=foo/&delimiter=/
Console: "upload file bar.txt inside folder foo"
API: PUT /foo/bar.txt
Now... if you take an empty bucket, and use the API (not the console), you can simply PUT /foo/bar.txt and you get the exact same net result in the console -- you see a folder named "foo" containing "bar.txt." The folder is displayed because there's an object with the prefix foo/. Delete the object and the folder vanishes.
Conversely, if you did it from the top with the console, once you deleted "bar.txt" there would still be folder "foo" because that's really just an empty object whose sole purpose is to cause a folder to appear in the console navigation when there are no other objects with that common prefix.
So, no... S3 does not treat both files and folders as objects. The S3 console creates objects that spoof folders, strictly as an aid to navigation, and the magic here is that the object's key ends with /. On the other hand, if those empty objects aren't there, the console still displays objects as though they were in folders.
You then see the problem that develops. The S3 service can't be asked to test against something of which it is unaware and which in fact need not exist.
So, it is not technically possible to do exactly what you asked; however, there appears to be a limited workaround. The primary limitation is that you can't specify that "the folder must exist," but you can specify "the object key prefix must match a predefined set of patterns."
The relevant part of the bucket or user policy might look something like this...
"Action": "s3:PutObject",
"Resource": [
"arn:aws:s3:::examplebucket/taxdocuments/*",
"arn:aws:s3:::examplebucket/personnel/*",
"arn:aws:s3:::examplebucket/unicorns/*"
...
],
A user impacted by this policy would be able to create any object beginning with taxdocuments/ or personnel/ or unicorns/ in the "examplebucket" bucket, and would not be able to create an object without one of those prefixes. Beyond that, the can create console folders "in" folders "in" folders all day long, as long as one of these prefixes is at the beginning of every fake folder's object key.
The limitation of course is that making another folder eligible for access requires modifying the policy.
This might work also, but proceed with caution:
"Resource": "arn:aws:s3:::examplebucket/?*/?*",
Intuitively it seems like this might work, but the flaw here -- assuming the ?*/?* is valid (it seems to be) and that ? does not match 0 characters the way * does -- is that this allows a user to create a new (pseudo-)folder in the root as long as they simultaneously create something inside it with a name at least one character long, using the API -- that is, creating an object with key pics/cat.jpg "creates" the "pics" folder if it's not already there, as explained above. From the console, this should prevent creation of new folders in the root, but from the API it would impose no such restriction.

Answer 2

Thank you for your elaborate response #Michael. You are absolutely correct in saying that API and CLI calls can continue to create root-level folders when non-empty. Console and S3-browser access works as intended. It is a compromise, but it is the closest we can get to what we wanted. Here's is the bucket policy I'm using:
{
"Version": "2012-10-17",
"Id": "Policy1486492608325",
"Statement": [
{
"Sid": "Stmt1486492495770",
"Effect": "Allow",
"Principal": {
"*"
},
"Action": [
"s3:DeleteObject",
"s3:Get*",
"s3:List*",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::test-storage",
"arn:aws:s3:::test-storage/*"
]
},
{
"Sid": "Stmt1486492534643",
"Effect": "Deny",
"Principal": {
"*"
},
"Action": [
"s3:DeleteObject",
"s3:PutObject"
],
"NotResource": "arn:aws:s3:::test-storage/?*/?*"
}
]
}