I need to allow users to access files on S3 with a specific prefix -- ie username1 -- that I can't change, so I can't just use the ${cognito-identity.amazonaws.com:sub} as the prefix. My S3 bucket is already populated with content under specific prefixes - so a user logs into my app, and is then allowed to access one of those specific prefixes
Currently set up on AWS to allows users to authenticate in a Cognito role using Firebase and developer authenticated identities. Each user account (email/password) has an associated S3 prefix that they must be able to access. Some users will share this prefix (two users accessing S3-bucket/username1 for example). A user should not be able to list or access any other prefix except their associated prefix.
I'm not quite sure what the best way to go about accomplishing this -- just with Cognito roles or using a database and lambda function + api endpoint, bucket/user policies or ACLs. Is there a simple way that I'm missing?
Pretty new to AWS, any help will be greatly appreciated!
tl;dr: How to only allow a user to access files on S3 with a specific prefix, that is NOT the ${cognito-identity.amazonaws.com:sub} variable?
As mentioned in comments, Amazon Cognito does not directly support your use case today. You can implement something like the following to achieve your goals:
Use Cognito to authenticate your users as normal. The Cognito identities would have permissions to invoke an API Gateway endpoint.
Your API (running in a Lambda) uses the Cognito identity id (provided in the context) to lookup the mapping of Cognito identity to your custom S3 prefix.
Your API uses STS to generate temporary credentials to access that prefix, returning them to the client.
The client uses those credentials to make a request directly to S3.
Related
AWS has a large number of buckets that different users have access to. And there is a lambda function that selects data from s3 and gives it to the client via the Api Gateway. The client has the opportunity to specify in the api request from which bucket lambda should make a selection. But how to check that he is requesting exactly the bucket to which he has permission?
In the iam policies, I can only indicate that it can access a specific api resource, but the resource is shared by everyone. In lambda authorizer, I can't get information about the user's rights and permissions (or can I?).
Please tell me how you can solve this issue. Which way to move?
P.S. This should be the authorization of users in amazon, I can't give them my JWT with my data.
It would be your responsibility to code the authentication and permission requirements in your own code. The person making the request via API Gateway is not an IAM User, so AWS does not recognise them and cannot grant access based on the normal AWS permission model.
Your code would need to:
Recognise and authenticate the user
Determine what resources (buckets) that user is permitted to access
Only provide access to permitted resources
How to do this is your decision. You should start with a way of identifying and authenticating the user.
I made a dashboard where users can upload files. I want to make it so that users can only have access to S3 urls that are files that they uploaded. How can I achieve this?
The users are application based, meaning they are, in my case, Django users.
Thanks! Anything helps!!
As this is entirely application based there are a few steps you could take to try and mitigate against accidental exposure.
Firstly organise your S3 folder structure in a way that uses prefixes for usernames, this way from a hierarchical point of view you can limit the scope of where users objects are stored. By prefix I mean the key might look like this users/$USERNAME/file.txt where $USERNAME is actually the users username.
You could enhance this in in your application by expanding to use Cognito, with a seperate user for every user you have created. When the user logs into your application you could also have the login occur via the Cognito user (this can be done programmatically).
With a successful login you'll be provided temporary credentials for IAM, all users can be assigned to a Cognito group which can have an IAM role attached. When you login it will assume this role, which allows some special properties to be supported in IAM.
By using ${cognito-identity.amazonaws.com:sub} you can actually limit the IAM permissions to only access that prefix of the S3 bucket. This moves responsibility from your application to Cognito and IAM.
More information about this is available in: Amazon S3: Allows Amazon Cognito Users to Access Objects in Their Bucket.
Let's say I want to store some files for each user which is using my website on s3. Later I want authenticate each request to s3 to make sure that user has access to the files she is requesting. I guess this can't be done using presigned URLs or signed cookies(using cloud front). So which Amazon service should I use for that? What is the simplest way to achieve this?
Lets saying I'm authenticating users using jwt and its possible to recognize wheather a user has access to a file or not by the filename and content of the jwt.
I'm sorry that I don't have enough reputation to comment so I'll post an answer here.
One solution is:
AWS Cognito (Federated Identities)
S3 (one bucket)
S3 bucket policies allow you to restrict access to "user folders" equivalent here to "identity" by the prefix like yourbucket/<cognito_identity_id>/* Each user on your webpage will have its own federated identity.
When you create and configure the identity pool in AWS define a custom authentication provider and authenticate users "by the developer" in your backend.
Also, associate the authenticated identities to one IAM Role with access to the S3 bucket where you will keep the data. The bucket policy will take care of only allowing each user to their files and not to others. (See referenced links for policy example and more)
Amazon S3: Allows Amazon Cognito Users to Access Objects in Their Bucket
Access to User level folders using Amazon S3 and Cognito
Developer Authenticated Identities (Identity Pools)
What I am trying to achieve is the following:
Create users dynamicly through API(users might grow alot - 50-100k+ eventually)
Give those users access to a specific prefix of an AWS S3 bucket(IAM policy)
Currently my idea is to create AWS IAM Users and generate credentials for those users(The credentials should not be temporary). This works fine, but the problem is that AWS is limited to 5000 IAM users. Is there another way to avoid that limit. One way that I found out is via cognito users -> https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_examples_s3_cognito-bucket.html
However I do not think that there is a way to create long-term access keys(as the IAM user access keys) for those cognito users ?
Is there another way to achieve this ?
Thanks in advance!
You should not use IAM for application users. IAM is for staff within your organisation to operate your AWS infrastructure.
Your application should operate its own authentication method separate from IAM (as suggested in the above comments). An example of using AWS for this task would be to use Amazon Cognito.
Once a user has authenticated, you have a couple of options:
Option 1: Using AWS credentials
If you want to allow the authenticated users to access AWS resources (eg Amazon S3) via AWS API calls, then you can create temporary credentials that have limited permissions (eg can access any object within a given path of a given bucket). These credentials can then be provided to the users. This method is commonly used for mobile applications that are capable of making API calls directly to AWS. It requires that the users have software that can use the AWS credentials.
Option 2: Amazon S3 pre-signed URLS
If you are running a web application and you want users to be able to access private objects in Amazon S3, you can generate pre-signed URLs. For example, let's say you are running a photo-sharing website. The process would be:
Photos are kept in private S3 buckets.
Users authenticate to the application.
The application can then show them their private photos: When the application generates any links to this private content, or embeds content in the page (eg via <img> tags), it generates a pre-signed URL, which provides time-limited access to private content.
The user then accesses the URL, or their browser requests data (eg images) from that URL.
Amazon S3 verifies the signature on the URL and check the validity time. If it is correct, then S3 returns the private object.
The application uses a set of IAM credentials to sign the pre-signed URL. This can be done in a couple of lines of code and does not require an API call to AWS.
The benefit of this method is that the application is responsible for determining which objects the user may access. For example, let's say a user wants to share their photos with another user. This sharing information can be stored in a database and the application can consult the database when sharing photos. If a user is entitled to view another user's photos, the application can generate a pre-signed URL without caring in which directory the photos are stored. This is a much more flexible approach than using storage location to grant access. However, it does require additional logic within the application.
See: Amazon S3 pre-signed URLs
How can I let another user access to my AWS S3 bucket without having to create an IAM role for it and sending the key/secret.
I want the third-party to decide the credentials for himself.
Is this even possible?
Basically I'm searching for something similar to OAuth for S3
By default, Amazon S3 buckets are private.
Access to objects can be granted in several ways:
A Bucket Policy can make a bucket, or part of a bucket, publicly accessible (not applicable for your use-case)
The Access Control List (ACL) on an object can make it publicly accessible (not applicable for your use-case)
IAM Users can be granted permissions on an Amazon S3 bucket (but IAM Users should only be used for your staff, not for application users)
IAM Roles can be temporarily assumed, but first require authentication (more on this below)
Pre-Signed URLs can be generated to provide time-limited access to Amazon S3 objects
For your use-case, the most applicable approach would be:
Users authenticate to your application. This could use Amazon Cognito, or whatever authentication method you wish to use.
When a user wishes to access a private object, your application determines whether they are entitled to such access (done via your own code).
If they are permitted access, the application should generate a pre-signed URL. This URL can be included in tags like <a> and <img>.
When the user accesses the URL, they will be able to access the object directly from Amazon S3.
Once the expiry time passes, the pre-signed URL no longer works.
So, you are welcome to use OAuth or any other authentication method, but it is the responsibility of your application to determine whether they are entitled to access individual objects and, if so, generate and return the pre-signed URL. (It's just a couple of lines of code, no API calls required to generate it.)
See: Share an Object with Others - Amazon Simple Storage Service
Amazon Cognito has the ability to issue credentials associated with an IAM Role, and users could then use these credentials to make API calls to AWS. However, it is generic role that would be shared by many users and is not a way to grant user-specific permissions.