This question is in the same line of thought than Is it possible to give token access to link to amazon s3 storage?.
Basically, we are building an app where groups of users can save pictures, that should be visible only to their own group.
We are thinking of using either a folder per user group, or it could even be an independent S3 bucket per user group.
The rules are very simple:
Any member of Group A should be able to add a picture to the Group A folder (or bucket)
Any member of Group A should be able to read all pictures of the Group A folder (or bucket)
No member of Group A should not have access to any of the pictures
However, the solution used by the post mentioned above (temporary pre-signed URLs) is not usable, as we need the client to be able to write files on his bucket as well as read the files on his bucket, without having any access to any other bucket. The file write part is the difficulty here and the reason why we cannot use pre-signed URLs.
Additionally, the solution from various AWS security posts that we read (for example https://aws.amazon.com/blogs/security/writing-iam-policies-grant-access-to-user-specific-folders-in-an-amazon-s3-bucket/) do not apply because they show how to control accesses for IAM groups of for other AWS accounts. In our case, a group of users does not have an IAM account...
The only solutions that we see so far are either insecure or wasteful
Open buckets to everybody and rely on obfuscating the folder / bucket names (lots of security issues, including the ability to brute force and read / overwrite anybody's files)
Have a back-end that acts as a facade between the app and S3, validating the accesses. S3 has no public access, the bucket is only opened to an IAM role that the back-end has. However this is a big waste of bandwidth, since all the data would transit on the EC2 instance(s) of that back-end
Any better solution?
Is this kind of customized access doable with S3?
The correct way to achieve your goal is to use Amazon S3 pre-signed URLs, which are time-limited URLs that provides temporary access to a private object.
You can also Upload objects using presigned URLs - Amazon Simple Storage Service.
The flow is basically:
Users authenticate to your back-end app
When a user wants to access a private object, the back-end verifies that they are permitted to access the object (using your own business logic, such as the Groups you mention). If they are allowed to access the object, the back-end generates a pre-signed URL.
The pre-signed URL is returned to the user's browser, such as putting it in a <img src="..."> tag.
When the user's browser requests the object, S3 verifies the signature in the pre-signed URL. If it is valid and the time period has not expired, S3 provides the requested object. (Otherwise, it returns Access Denied.)
A similar process is used when users upload objects:
Users authenticate to your back-end app
They request the opportunity to upload a file
Your back-end app generates an S3 Pre-signed URL that is included in the HTML page for upload
Your back-end should track the object in a database so it knows who performed the upload and keeps track of who is permitted to access the object (eg particular users or groups)
Your back-end app is fully responsible for deciding whether particular users can upload/download objects. It then hands-off the actual upload/download process to S3 via the pre-signed URLs. This reduces load on your server because all uploads/downloads go direct to/from S3.
Related
Given application with teams(or teams inside of company). Each company can have any amount of teams. Each team can have any amount of users.
How do I provide private S3 storage for each group of users of my application? So only team members and managers could see attachments inside current group?
Since users are accessing Amazon S3 via your application, they should not be given any AWS credentials (eg IAM User credentials). Instead, the application would be responsible for all security of objects in Amazon S3.
It would work as follows:
All objects in Amazon S3 are kept private
The credentials used by your application, however, has full access to the objects in S3
When a user requests access to an object, or an object needs to be referenced within an HTML page (eg <img> tag), the application creates an Amazon S3 pre-signed URL, which provides time-limited access to a private object
When the user follows this link, or the browser references the link, Amazon S3 verifies the pre-signed URL and grants access to the object. (If, however, the time period has expired, access would be denied.)
If a user wishes to upload a file, then the application can generate a pre-signed URL for uploads, which will grant permission to upload an object directly to a private S3 bucket
At all times, your application will be responsible for determining whether a user is permitted to download or upload an object. The application 'signs' the request, and S3 will permit the operation.
As an example, imagine a photo-sharing application. The flow would be:
Users authenticate to the application
The application shows them thumbnails of their photos (using pre-signed URLs in the <img> tags)
Users choose to share photos with other users, which the application records in its database
When another user requests to see a shared photo, the application checks the sharing information within the database and, if they are permitted to view the photo, generates a pre-signed URL granting temporary access to the photo
I'm not sure if this is the appropriate use case, so please tell me what to look for if I'm incorrect in my assumption of how to do this.
What I'm trying to do:
I have an s3 bucket with different 'packs' that users can download. Upon their purchase, they are given a user role in Wordpress. I have an S3 browser set up via php that makes requests to the bucket for info.
Based on their 'role', it will only show files that match prefix (whole pack users see all, single product people only see single product prefix).
In that way, the server will be sending the files on behalf of the user, and changing IAM roles based on the user's permission level. Do I have to have it set that way? Can I just analyze the WP role and specify and endpoint or query that notes the prefixes allowed?
Pack users see /
Individual users see /--prefix/
If that makes sense
Thanks in advance! I've never used AWS, so this is all new to me. :)
This sounds too complex. It's possible to do with AWS STS but it would be extremely fragile.
I presume you're hiding the actual S3 bucket from end users and are streaming through your php application? If so, it makes more sense to do any role-based filtering in the php application as you have far more logic available to you there - IAM is granular, but restrictions to resources in S3 is going to be funky and there's always a chance you'll get something wrong and expose the incorrect downloads.
Rather do this inside your app:
establish the role you've granted
issue the S3 ls command filtered by the role - i.e. if the role permits only --prefix, issue the ls command so that it only lists files matching --prefix
don't expose files in the bucket globally - only your app should have access to the S3 bucket - that way people also can't share links once they've downloaded a pack.
this has the added benefit of not encoding your S3 bucket structure in IAM, and keeps your decision logic isolated to code.
There are basically three ways you can grant access to private content in Amazon S3.
Option 1: IAM credentials
You can add a policy to an IAM User, so that they can access private content. However, such credentials should only be used by staff in your own organization. it should not be used to grant access to application users.
Option 2: Temporary credentials via STS
Your application can generate temporary credentials via the AWS Security Token Service. These credentials can be given specific permissions and are valid for a limited time period. This is ideal for granting mobile apps access to Amazon S3 because they can communicate directly with S3 without having to go via the back-end app. The credentials would only be granted access to resources they are permitted to use.
These types of credentials can also be used by web applications, where the web apps make calls directly to AWS services (eg from Node/JavaScript in the browser). However, this doesn't seem suitable for your WordPress situation.
Option 3: Pre-Signed URLs
Imagine a photo-sharing application where users can access their private photos, and users can also share photos with other users. When a user requests access to a particular photo (or when the back-end app is creating an HTML page that uses a photo), the app can generate a pre-signed URL that grants temporary access to an Amazon S3 object.
Each pre-signed URL gives access only to a single S3 object and only for a selected time period (eg 5 minutes). This means that all the permission logic for whether a user is entitled to access a file can be performed in the back-end application. When the back-end application provides a pre-signed URL to the user's browser, the user can access the content directly from Amazon S3 without going via the back-end.
See: Amazon S3 pre-signed URLs
Your situation sounds suitable for Option #3. Once you have determined that a user is permitted to access a particular file in S3, it can generate the pre-signed URL and include it as a link (or even in <img src=...> tags). The user can then download the file. There is no need to use IAM Roles in this process.
I have an application where users are part of a 'group' of users. Each group can 'upload' documents to the application. Behind the scenes I am using S3 to store these documents.
I've spent a ton of time reading the AWS documentation but still don't understand the simplest/correct way to do the following:
User 1 in group A can upload documents to application
User 2 in group A can see and access all group A documents in application
User 3 in group B can upload documents to application
User 3 in group B cannot see any documents that belong to group A (and vice-versa)
Should I be using the API to create a new bucket for each 'group'?
Or can all of this be done in a single bucket with subdirectories for each group & then set access limitations?
Should I be setting up an IAM group policy and applying it to each web app user?
I'm not sure of the best architecture for this scenario so would really appreciate a point in the right direction.
AWS credentials should be assigned to your application and to your IT staff who need to maintain the application.
Users of your application should not be given AWS credentials.
Users should interact directly with your application and your application will make calls to the AWS API from the back-end. This way, your application has full control of what data they can see and what operations they can perform.
Think of it like a database -- you never want to give users direct access to a database. Instead, they should always interact via an application, which will store and update information in a database.
There are some common exceptions to the above:
If you want users to access/download a file stored in S3, your application can generate a pre-signed URL, which is a time-limited URL that permits access to an Amazon S3 object. Your application is responsible for generating the URL when it wants to grant access and the URl can be included in an HTML page (eg show a private picture on a web page).
If you want to allow users to upload files directly to S3, you could again use a pre-signed URL or you could grant public Write access to an Amazon S3 bucket. Think of it like a modern FTP server.
Bottom line: Your application is in charge! Also, consider using pre-signed URLs to provide direct access to objects when the application permits it.
I'm storing user images on S3 which are readable by default.
I need to access the images directly from the web as well.
However, I'd like to prevent hackers from brute forcing the URL and downloading my images.
For example, my S3 image url is at http://s3.aws.com/test.png
They can brute force test and download all the contents?
I cannot set the items inside my buckets to be private because I need to access directly from the web.
Any idea how to prevent it?
Using good security does not impact your ability to "access directly from the web". All content in Amazon S3 can be accessed from the web if appropriate permissions are used.
By default, all content in Amazon S3 is private.
Permissions to access content can then be assigned in several ways:
Directly on the object (eg make an object 'public')
Via a Bucket Policy (eg permit access to a subdirectory if accessed from a specific range of IP addresses, during a particular time of day, but only via HTTPS)
Via a policy assigned to an IAM User (which requires the user to authenticate when accessing Amazon S3)
Via a time-limited Pre-signed URL
The most interesting is the Pre-Signed URL. This is a calculated URL that permits access to an Amazon S3 object for a limited period of time. Applications can generate a Pre-signed URL and include the link in a web page (eg as part of a <img> tag). That way, your application determines whether a user is permitted to access an object and can limit the time duration that the link will work.
You should keep your content secure, and use Pre-signed URLs to allow access only for authorized visitors to your web site. You do have to write some code to make it work, but it's secure.
I work on a SaaS application where Creators can create Groups and invite others to their Group to share files, chat and so on. Only people within specific group should have access to this group's files.
People from other group must not have access to not their group's files.
And of course all files permission should be set to 'Private', i.e. they should not be searchable/visible/accessable by anonymous users of Internet since information in those files is for personal use only.
I am new to Amazon S3 and don't know how to achieve it... Should I create only 1 main bucket? Or create for each group a new Amazon Bucket?
It is not recommended to use AWS Identity and Access Management (IAM) for storing application users. Application users should be maintained in a separate database (or LDAP, Active directory, etc).
Therefore, creating "one bucket per group" is not feasible, since it is not possible to assign your applications users to permissions within Amazon S3.
The better method would be to manage permissions within your application. When a user requests access to a file, the application can determine whether they should be permitted access. If they are permitted, then the application can generate a Pre-Signed URL.
A Pre-Signed URL permits access to private objects stored on Amazon S3. It is a means of keeping objects secure, yet granting temporary access to a specific object.
When listing available files, your application would generate links that include the pre-signed URL. Then, when a user clicks the link, they can access the file. Then, after a certain time has expired (eg 10 minutes), the link will no longer function. So, if a user shares a link with somebody else, it will probably have timed-out.
See: Creating a pre-signed URL in Ruby