I have created a bucket in Amazon S3 and have uploaded 2 files in it and made them public. I have the links through which I can access them from anywhere on the Internet. I now want to put some restriction on who can download the files. Can someone please help me with that. I did try the documentation, but got confused.
I want that at the time of download using the public link it should ask for some credentials or something to authenticate the user at that time. Is this possible?
By default, all objects in Amazon S3 are private. You can then add permissions so that people can access your objects. This can be done via:
Access Control List permissions on individual objects
A Bucket Policy
IAM Users and Groups
A Pre-Signed URL
As long as at least one of these methods is granting access, your users will be able to access the objects from Amazon S3.
1. Access Control List on individual objects
The Make Public option in the Amazon S3 management console will grant Open/Download permissions to all Internet users. This can be used to grant public access to specific objects.
2. Bucket Policy
A Bucket Policy can be used to grant access to a whole bucket or a portion of a bucket. It can also be used to specify limits to access. For example, a policy could make a specific directory within a bucket public to users from a specific range of IP addresses, during particular times of the day, and only when accessing the bucket via SSL.
A bucket policy is a good way to grant public access to many objects (eg a particular directory) without having to specify permissions on each individual object. This is commonly used for static websites served out of an S3 bucket.
3. IAM Users and Groups
This is similar to defining a Bucket Policy, but permissions are assigned to specific Users or Groups of users. Thus, only those users have permission to access the objects. Users must authenticate themselves when accessing the objects, so this is most commonly used when accessing objects via the AWS API, such as using the aws s3 commands from the AWS Command-Line Interface (CLI).
Rather than being prompted to authenticate, users must provide the authentication when making the API call. A simple way of doing this is to store user credentials in a local configuration file, which the CLI will automatically use when calling the S3 API.
4. Pre-Signed URL
A Pre-Signed URL can be used to grant access to S3 objects as a way of "overriding" access controls. A normally private object can be accessed via a URL by appending an expiry time and signature. This is a great way to serve private content without requiring a web server.
Typically, an application constructs a Pre-Signed URL when it wishes to grant access to an object. For example, let's say you have a photo-sharing website and a user has authenticated to your website. You now wish to display their pictures in a web page. The pictures are normally private, but your application can generate Pre-Signed URLs that grant them temporary access to the pictures. The Pre-Signed URL will expire after a particular date/time.
Regarding the pre-signed URL, the signature is in the request headers, hence it should be within HTTPS/TLS encryption. But do check for yourself.
Related
What I am trying to achieve is the following:
Create users dynamicly through API(users might grow alot - 50-100k+ eventually)
Give those users access to a specific prefix of an AWS S3 bucket(IAM policy)
Currently my idea is to create AWS IAM Users and generate credentials for those users(The credentials should not be temporary). This works fine, but the problem is that AWS is limited to 5000 IAM users. Is there another way to avoid that limit. One way that I found out is via cognito users -> https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_examples_s3_cognito-bucket.html
However I do not think that there is a way to create long-term access keys(as the IAM user access keys) for those cognito users ?
Is there another way to achieve this ?
Thanks in advance!
You should not use IAM for application users. IAM is for staff within your organisation to operate your AWS infrastructure.
Your application should operate its own authentication method separate from IAM (as suggested in the above comments). An example of using AWS for this task would be to use Amazon Cognito.
Once a user has authenticated, you have a couple of options:
Option 1: Using AWS credentials
If you want to allow the authenticated users to access AWS resources (eg Amazon S3) via AWS API calls, then you can create temporary credentials that have limited permissions (eg can access any object within a given path of a given bucket). These credentials can then be provided to the users. This method is commonly used for mobile applications that are capable of making API calls directly to AWS. It requires that the users have software that can use the AWS credentials.
Option 2: Amazon S3 pre-signed URLS
If you are running a web application and you want users to be able to access private objects in Amazon S3, you can generate pre-signed URLs. For example, let's say you are running a photo-sharing website. The process would be:
Photos are kept in private S3 buckets.
Users authenticate to the application.
The application can then show them their private photos: When the application generates any links to this private content, or embeds content in the page (eg via <img> tags), it generates a pre-signed URL, which provides time-limited access to private content.
The user then accesses the URL, or their browser requests data (eg images) from that URL.
Amazon S3 verifies the signature on the URL and check the validity time. If it is correct, then S3 returns the private object.
The application uses a set of IAM credentials to sign the pre-signed URL. This can be done in a couple of lines of code and does not require an API call to AWS.
The benefit of this method is that the application is responsible for determining which objects the user may access. For example, let's say a user wants to share their photos with another user. This sharing information can be stored in a database and the application can consult the database when sharing photos. If a user is entitled to view another user's photos, the application can generate a pre-signed URL without caring in which directory the photos are stored. This is a much more flexible approach than using storage location to grant access. However, it does require additional logic within the application.
See: Amazon S3 pre-signed URLs
How can I let another user access to my AWS S3 bucket without having to create an IAM role for it and sending the key/secret.
I want the third-party to decide the credentials for himself.
Is this even possible?
Basically I'm searching for something similar to OAuth for S3
By default, Amazon S3 buckets are private.
Access to objects can be granted in several ways:
A Bucket Policy can make a bucket, or part of a bucket, publicly accessible (not applicable for your use-case)
The Access Control List (ACL) on an object can make it publicly accessible (not applicable for your use-case)
IAM Users can be granted permissions on an Amazon S3 bucket (but IAM Users should only be used for your staff, not for application users)
IAM Roles can be temporarily assumed, but first require authentication (more on this below)
Pre-Signed URLs can be generated to provide time-limited access to Amazon S3 objects
For your use-case, the most applicable approach would be:
Users authenticate to your application. This could use Amazon Cognito, or whatever authentication method you wish to use.
When a user wishes to access a private object, your application determines whether they are entitled to such access (done via your own code).
If they are permitted access, the application should generate a pre-signed URL. This URL can be included in tags like <a> and <img>.
When the user accesses the URL, they will be able to access the object directly from Amazon S3.
Once the expiry time passes, the pre-signed URL no longer works.
So, you are welcome to use OAuth or any other authentication method, but it is the responsibility of your application to determine whether they are entitled to access individual objects and, if so, generate and return the pre-signed URL. (It's just a couple of lines of code, no API calls required to generate it.)
See: Share an Object with Others - Amazon Simple Storage Service
Amazon Cognito has the ability to issue credentials associated with an IAM Role, and users could then use these credentials to make API calls to AWS. However, it is generic role that would be shared by many users and is not a way to grant user-specific permissions.
How to allow read/write/delete etc, permissions to users in a particular IAM group for a specific Amazon S3 object/file.
If you wish to control access to "millions" of individual files where access is not based upon the path (directory/folder) of the files, then you will need to create your own authentication method.
This can be done by using an Amazon S3 Pre-signed URL. Basically:
Users access your application
When they request access to a secure file (or, for example, when the application generates an HTML page that includes a link to such a file, or even a reference in an Image tag), the application generates a time-limited pre-signed URL
Users can use this link/URL to access the object in Amazon S3
After the expiry period, the URL no longer works
This gives your application full control over whether a user can access an object.
The only alternative if you were to use IAM would be to grant access based upon the path of the object. It is not a good method to assign access to individual objects.
Using Amazon's S3 storage, is it possible to set an image to only be viewable to specific users of an application?
I've looked at the policy generator, but I can't seem to find what I'm looking for.
Some of the information on this documentation page sounds relevant. Could I achieve this with signed URLs or IAM users? Ideally, the users of the app do not have to create an AWS account.
By default, all objects in Amazon S3 are private. You can then add permissions so that people can access your objects. This can be done via:
Access Control List (ACL) permissions on individual objects
A Bucket Policy
IAM Users and Groups
Pre-Signed URLs
Given that you wish to grant access to application users, the recommended method is a Pre-Signed URL.
A Pre-Signed URL can be used to grant access to S3 objects as a way of "overriding" access controls. A normally private object can be accessed via a URL by appending an expiry time and signature. This is a great way to serve private content to users without having to define every user within IAM. (It is recommended to only use IAM for staff, not application users.)
A pre-signed URL can be generated from a few lines of code. A quick way to experiment is to use the AWS Command-Line Interface (CLI), which has a aws s3 presign command.
See: AWS CLI aws s3 presign documentation
There are equivalent commands for all AWS SDKs in various programming languages.
I am storing files in a S3 bucket. I want the access to the files be restricted.
Currently, anyone with the URL to the file is able to access the file.
I want a behavior where file is accessed only when it is accessed through my application. The application is hosted on EC2.
Following are 2 possible ways I could find.
Use "referer" key in bucket policy.
Change "allowed origin" in CORS configuration
Which of the above 2 should be used, given the fact that 'referer' could be spoofed in the request header.
Also can cloudfront play a role over here?
I would recommend using a Pre-Signed URL that permits access to private objects stored on Amazon S3. It is a means of keeping objects secure, yet grant temporary access to a specific object.
It is created via a hash calculation based on the object path, expiry time and a shared Secret Access Key belonging to an account that has permission to access the Amazon S3 object. The result is a time-limited URL that grants access to the object. Once the expiry time passes, the URL does not return the object.
Start by removing existing permissions that grant access to these objects. Then generate Pre-Signed URLs to grant access to private content on a per-object basis, calculated every time you reference an S3 object. (Don't worry, it's fast to do!)
See documentation: Sample code in Java
When dealing with a private S3 bucket, you'll want to use an AWS SDK appropriate for your use case.
Here lies SDKs for many different languages: http://aws.amazon.com/tools/
Within each SDK, you can find sample calls to S3.
If you are trying to make private calls via browser-side JavaScript, you can use CORS.