Given application with teams(or teams inside of company). Each company can have any amount of teams. Each team can have any amount of users.
How do I provide private S3 storage for each group of users of my application? So only team members and managers could see attachments inside current group?
Since users are accessing Amazon S3 via your application, they should not be given any AWS credentials (eg IAM User credentials). Instead, the application would be responsible for all security of objects in Amazon S3.
It would work as follows:
All objects in Amazon S3 are kept private
The credentials used by your application, however, has full access to the objects in S3
When a user requests access to an object, or an object needs to be referenced within an HTML page (eg <img> tag), the application creates an Amazon S3 pre-signed URL, which provides time-limited access to a private object
When the user follows this link, or the browser references the link, Amazon S3 verifies the pre-signed URL and grants access to the object. (If, however, the time period has expired, access would be denied.)
If a user wishes to upload a file, then the application can generate a pre-signed URL for uploads, which will grant permission to upload an object directly to a private S3 bucket
At all times, your application will be responsible for determining whether a user is permitted to download or upload an object. The application 'signs' the request, and S3 will permit the operation.
As an example, imagine a photo-sharing application. The flow would be:
Users authenticate to the application
The application shows them thumbnails of their photos (using pre-signed URLs in the <img> tags)
Users choose to share photos with other users, which the application records in its database
When another user requests to see a shared photo, the application checks the sharing information within the database and, if they are permitted to view the photo, generates a pre-signed URL granting temporary access to the photo
Related
This question is in the same line of thought than Is it possible to give token access to link to amazon s3 storage?.
Basically, we are building an app where groups of users can save pictures, that should be visible only to their own group.
We are thinking of using either a folder per user group, or it could even be an independent S3 bucket per user group.
The rules are very simple:
Any member of Group A should be able to add a picture to the Group A folder (or bucket)
Any member of Group A should be able to read all pictures of the Group A folder (or bucket)
No member of Group A should not have access to any of the pictures
However, the solution used by the post mentioned above (temporary pre-signed URLs) is not usable, as we need the client to be able to write files on his bucket as well as read the files on his bucket, without having any access to any other bucket. The file write part is the difficulty here and the reason why we cannot use pre-signed URLs.
Additionally, the solution from various AWS security posts that we read (for example https://aws.amazon.com/blogs/security/writing-iam-policies-grant-access-to-user-specific-folders-in-an-amazon-s3-bucket/) do not apply because they show how to control accesses for IAM groups of for other AWS accounts. In our case, a group of users does not have an IAM account...
The only solutions that we see so far are either insecure or wasteful
Open buckets to everybody and rely on obfuscating the folder / bucket names (lots of security issues, including the ability to brute force and read / overwrite anybody's files)
Have a back-end that acts as a facade between the app and S3, validating the accesses. S3 has no public access, the bucket is only opened to an IAM role that the back-end has. However this is a big waste of bandwidth, since all the data would transit on the EC2 instance(s) of that back-end
Any better solution?
Is this kind of customized access doable with S3?
The correct way to achieve your goal is to use Amazon S3 pre-signed URLs, which are time-limited URLs that provides temporary access to a private object.
You can also Upload objects using presigned URLs - Amazon Simple Storage Service.
The flow is basically:
Users authenticate to your back-end app
When a user wants to access a private object, the back-end verifies that they are permitted to access the object (using your own business logic, such as the Groups you mention). If they are allowed to access the object, the back-end generates a pre-signed URL.
The pre-signed URL is returned to the user's browser, such as putting it in a <img src="..."> tag.
When the user's browser requests the object, S3 verifies the signature in the pre-signed URL. If it is valid and the time period has not expired, S3 provides the requested object. (Otherwise, it returns Access Denied.)
A similar process is used when users upload objects:
Users authenticate to your back-end app
They request the opportunity to upload a file
Your back-end app generates an S3 Pre-signed URL that is included in the HTML page for upload
Your back-end should track the object in a database so it knows who performed the upload and keeps track of who is permitted to access the object (eg particular users or groups)
Your back-end app is fully responsible for deciding whether particular users can upload/download objects. It then hands-off the actual upload/download process to S3 via the pre-signed URLs. This reduces load on your server because all uploads/downloads go direct to/from S3.
My scenario is as follows:
Every user on my web application has uuid as primary key.
My web application users can upload images to their own folder(folder name is their uuid,
ex:
clients/0d75db15-07ad-4800-a5de-1fe82a7bf52e/
clients/0faf6315-6ba9-478d-aaa5-3cd2bd3b3b6e/
Every user can only access their own folder and files under the folder.
I know that IAM policies has permissions but not folder level or object level.
Is there any solutions for the scenario?
Thanks for any suggestions and answers!
Given that your application has 1000+ users, you should not be relying on IAM policies or S3 Bucket policies to manage the security.
Instead, all objects should be kept as private and the application should generate Amazon S3 pre-signed URLs, which grant time-limited access to private objects stored in Amazon S3.
It would work like this:
A user logs-in to your application
When user requests access to an image, or if application wants to send them an HTML page that includes images using <img src='xxx'> tags, the application should:
Verify that the user is entitled to access the object
If so, generate a pre-signed URL that grants temporary access to the object
The user's browser then accesses the provided URL
Amazon S3 will verify that the pre-signed URL is valid and then provides access to the object
What I am trying to achieve is the following:
Create users dynamicly through API(users might grow alot - 50-100k+ eventually)
Give those users access to a specific prefix of an AWS S3 bucket(IAM policy)
Currently my idea is to create AWS IAM Users and generate credentials for those users(The credentials should not be temporary). This works fine, but the problem is that AWS is limited to 5000 IAM users. Is there another way to avoid that limit. One way that I found out is via cognito users -> https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_examples_s3_cognito-bucket.html
However I do not think that there is a way to create long-term access keys(as the IAM user access keys) for those cognito users ?
Is there another way to achieve this ?
Thanks in advance!
You should not use IAM for application users. IAM is for staff within your organisation to operate your AWS infrastructure.
Your application should operate its own authentication method separate from IAM (as suggested in the above comments). An example of using AWS for this task would be to use Amazon Cognito.
Once a user has authenticated, you have a couple of options:
Option 1: Using AWS credentials
If you want to allow the authenticated users to access AWS resources (eg Amazon S3) via AWS API calls, then you can create temporary credentials that have limited permissions (eg can access any object within a given path of a given bucket). These credentials can then be provided to the users. This method is commonly used for mobile applications that are capable of making API calls directly to AWS. It requires that the users have software that can use the AWS credentials.
Option 2: Amazon S3 pre-signed URLS
If you are running a web application and you want users to be able to access private objects in Amazon S3, you can generate pre-signed URLs. For example, let's say you are running a photo-sharing website. The process would be:
Photos are kept in private S3 buckets.
Users authenticate to the application.
The application can then show them their private photos: When the application generates any links to this private content, or embeds content in the page (eg via <img> tags), it generates a pre-signed URL, which provides time-limited access to private content.
The user then accesses the URL, or their browser requests data (eg images) from that URL.
Amazon S3 verifies the signature on the URL and check the validity time. If it is correct, then S3 returns the private object.
The application uses a set of IAM credentials to sign the pre-signed URL. This can be done in a couple of lines of code and does not require an API call to AWS.
The benefit of this method is that the application is responsible for determining which objects the user may access. For example, let's say a user wants to share their photos with another user. This sharing information can be stored in a database and the application can consult the database when sharing photos. If a user is entitled to view another user's photos, the application can generate a pre-signed URL without caring in which directory the photos are stored. This is a much more flexible approach than using storage location to grant access. However, it does require additional logic within the application.
See: Amazon S3 pre-signed URLs
I have upload all my images in s3 bucket and allowing it to show in my website using s3 url, but when I access the s3 url directly in browser it showing the image, I want to block those access
Can anyone help me on how to Block s3 public url access for my image and only show the image in mobile app or website.
All objects in Amazon S3 are private by default.
Access to objects can be granted in several ways:
A Bucket Policy can make a whole bucket (or a part of a bucket) public to everyone. It is also possible to specify restrictions, such as IP address and referer.
An Access Control List on an object can make the object public (for everyone)
An IAM Policy can grant access to objects for specific IAM Users
A pre-signed URL can provide temporary access to a private object
Based upon your question, I would recommend:
Keep the objects private (remove Bucket Policies and ACLs)
When a user wishes to access an image or other object, your application determines whether the user is permitted to access the object (this logic is totally up to you to write in your application)
If they are permitted, your application can create a pre-signed URL in a few lines of code, which will allow the mobile app or website to access the object for a limited time period that your app specifies (eg 5 minutes). After this time period, the URL will no longer provide access.
Thus, your application has full control over whether somebody is permitted to access the image, while still serving the content directly from Amazon S3 (eg in a web page via a <img> tag).
See: Share an Object with Others - Amazon Simple Storage Service
How to allow read/write/delete etc, permissions to users in a particular IAM group for a specific Amazon S3 object/file.
If you wish to control access to "millions" of individual files where access is not based upon the path (directory/folder) of the files, then you will need to create your own authentication method.
This can be done by using an Amazon S3 Pre-signed URL. Basically:
Users access your application
When they request access to a secure file (or, for example, when the application generates an HTML page that includes a link to such a file, or even a reference in an Image tag), the application generates a time-limited pre-signed URL
Users can use this link/URL to access the object in Amazon S3
After the expiry period, the URL no longer works
This gives your application full control over whether a user can access an object.
The only alternative if you were to use IAM would be to grant access based upon the path of the object. It is not a good method to assign access to individual objects.