I have a mobile app, each user in this app needs to read and write to a DynamoDB table, also they need to read and write from an S3 bucket. There are no user accounts in this app, no user identity, no logging in. Every user needs the same access to the same DDB table and S3 bucket.
Currently I have this working with 'unauthenticated' cognito credentials that has its role constrained to the single DynamoDB table and single S3 bucket. This seems Okay to me. But if someone finds out my identityPoolId then they can basically make any edits to my S3 bucket and DDB table. Is it sufficient to just rely on hiding the identityPoolId in the mobile app somehow?
Also relying on cognito seems maybe overkill since users don't actually have any identity in my app. Right now it just generates a new 'unauthenticated' cognito credential every time the app opens. Seems like that'd waste alot of AWS resource unnecessarily because I would just be getting thousands of unauthenticated cognito users generated each time a user opens the app. Is that how that should work or am I missing something there?
But then I was also reading about using 'temporary security credentials' which relies on AccessKeyId, SecretAccessKey and SessionToken:
https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html
Which this does not need any cognito resources at all. Is this the more appropriate thing to use in my case? Is it sufficient security to just hide that key, secret and token in my codebase and rely on that?
Related
I made a dashboard where users can upload files. I want to make it so that users can only have access to S3 urls that are files that they uploaded. How can I achieve this?
The users are application based, meaning they are, in my case, Django users.
Thanks! Anything helps!!
As this is entirely application based there are a few steps you could take to try and mitigate against accidental exposure.
Firstly organise your S3 folder structure in a way that uses prefixes for usernames, this way from a hierarchical point of view you can limit the scope of where users objects are stored. By prefix I mean the key might look like this users/$USERNAME/file.txt where $USERNAME is actually the users username.
You could enhance this in in your application by expanding to use Cognito, with a seperate user for every user you have created. When the user logs into your application you could also have the login occur via the Cognito user (this can be done programmatically).
With a successful login you'll be provided temporary credentials for IAM, all users can be assigned to a Cognito group which can have an IAM role attached. When you login it will assume this role, which allows some special properties to be supported in IAM.
By using ${cognito-identity.amazonaws.com:sub} you can actually limit the IAM permissions to only access that prefix of the S3 bucket. This moves responsibility from your application to Cognito and IAM.
More information about this is available in: Amazon S3: Allows Amazon Cognito Users to Access Objects in Their Bucket.
I am trying to understand access security as it relates to Amazon S3. I want to host some files in an S3 bucket, using CloudFront to access it via my domain. I need to limit access to certain companies/individuals. In addition I need to manage that access individually.
A second access model is project based, where I need to make a library of files available to a particular project team, and I need to be able to add and remove team members in an ad hoc manner, and then close access for the whole project at some point. The bucket in question might be the same for both scenarios.
I assume something like this is possible in AWS, but all I can find (and understand) on the AWS site involves using IAM to control access via the AWS console. I don't see any indication that I could create an IAM user, add them to an IAM group, give the group read only access to the bucket and then provide the name and password via System.Net.WebClient in PowerShell to actually download the available file. Am I missing something, and this IS possible? Or am I not correct in my assumption that this can be done with AWS?
I did find Amazon CloudFront vs. S3 --> restrict access by domain? - Stack Overflow that talks about using CloudFront to limit access by Domain, but that won't work in a WfH scenario, as those home machines won't be on the corporate domain, but the corporate BIM Manager needs to manage access to content libraries for the WfH staff. I REALLY hope I am not running into an example of AWS just not being ready for the current reality.
Content stored in Amazon S3 is private by default. There are several ways that access can be granted:
Use a bucket policy to make the entire bucket (or a directory within it) publicly accessible to everyone. This is good for websites where anyone can read the content.
Assign permission to IAM Users to grant access only to users or applications that need to access to the bucket. This is typically used within your organization. Never create an IAM User for somebody outside your organization.
Create presigned URLs to grant temporary access to private objects. This is typically used by applications to grant web-based access to content stored in Amazon S3.
To provide an example for pre-signed URLs, imagine that you have a photo-sharing website. Photos provided by users are private. The flow would be:
A user logs in. The application confirms their identity against a database or an authentication service (eg Login with Google).
When the user wants to view a photo, the application first checks whether they are entitled to view the photo (eg it is their photo). If they are entitled to view the photo, the application generates a pre-signed URL and returns it as a link, or embeds the link in an HTML page (eg in a <img> tag).
When the user accesses the link, the browser sends the URL request to Amazon S3, which verifies the encrypted signature in the signed URL. If if it is correct and the link has not yet expired, the photo is returned and is displayed in the web browser.
Users can also share photos with other users. When another user accesses a photo, the application checks the database to confirm that it was shared with the user. If so, it provides a pre-signed URL to access the photo.
This architecture has the application perform all of the logic around Access Permissions. It is very flexible since you can write whatever rules you want, and then the user is sent to Amazon S3 to obtain the file. Think of it like buying theater tickets online -- you just show the ticket and the door and you are allowed to sit in the seat. That's what Amazon S3 is doing -- it is checking the ticket (signed URL) and then giving you access to the file.
See: Amazon S3 pre-signed URLs
Mobile apps
Another common architecture is to generate temporary credentials using the AWS Security Token Service (STS). This is typically done with mobile apps. The flow is:
A user logs into a mobile app. The app sends the login details to a back-end application, which verifies the user's identity.
The back-end app then uses AWS STS to generate temporary credentials and assigns permissions to the credentials, such as being permitted to access a certain directory within an Amazon S3 bucket. (The permissions can actually be for anything in AWS, such as launching computers or creating databases.)
The back-end app sends these temporary credentials back to the mobile app.
The mobile app then uses those credentials to make calls directly to Amazon S3 to access files.
Amazon S3 checks the credentials being used and, if they have permission for the files being requests, grants access. This can be done for uploads, downloads, listing files, etc.
This architecture takes advantage of the fact that mobile apps are quite powerful and they can communicate directly with AWS services such as Amazon S3. The permissions granted are based upon the user who logs in. These permissions are determined by the back-end application, which you would code. Think of it like a temporary employee who has been granted a building access pass for the day, but they can only access certain areas.
See: IAM Role Archives - Jayendra's Blog
The above architectures are building blocks for how you wish to develop your applications. Every application is different, just like the two use-cases in your question. You can securely incorporate Amazon S3 in your applications while maintaining full control of how access is granted. Your applications can then concentrate on the business logic of controlling access, without having to actually serve the content (which is left up to Amazon S3). It's like selling the tickets without having to run the theater.
You ask whether Amazon S3 is "ready for the current reality". Many of the popular web sites you use every day run on AWS, and you probably never realize it.
If you are willing to issue IAM User credentials (max 5000 per account), the steps would be:
Create an IAM User for each user and select Programmatic access
This will provide an Access Key and Secret Key that you can provide to each user
Attach permissions to each IAM User, or put the users in an IAM Group and attach permissions to the IAM Group
Each user can run aws configure on their computer (using the AWS Command-Line Interface (CLI) to store their Access Key and Secret Key
They can then use the AWS CLI to upload/download files
If you want the users to be able to access via the Amazon S3 management console, you will need to provide some additional permissions: Grant a User Amazon S3 Console Access to Only a Certain Bucket
Alternatively, users could use a program like CyberDuck for an easy Drag & Drop interface to Amazon S3. Cyberduck will also ask for the Access Key and Secret Key.
We are building a custom application (using LoopBack) that will need to store many large files coming from multiple users, so naturally we're looking at S3. We've done something similar before, with clients uploading files to the server which then processes and uploads them to S3 under one AWS account, but for this new app, we're looking to allow the clients (using a custom iOS app) to use the iOS S3 SDK to upload directly to their own bucket or folder. User accounts will be created on the server.
Is there any way to handle S3 authentication/authorization using custom code? For example, could the iOS client request a temporary token allowing them to upload to a specific S3 bucket or folder? Or would we need to create unique IAM users for each user in our system?
Is that a terrible idea? It sounds like a terrible idea. :)
I found a similar question here but there was no conclusive answer.
Update: I found this article on Temporary Security Credentials that looks very promising. It also suggests using Cognito, which I've never used, if building a mobile app.
Cognito is the way to go. You should definetly not create IAM users for this. IAM is for managing access to the aws services programatically or from the console. Moreover you would need to hardcode the IAM access keys in the ios app, which is not a best practice.
https://docs.aws.amazon.com/cognito/latest/developerguide/what-is-amazon-cognito.html
https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_oidc_cognito.html
If users of your application are already authenticated, you could generate a pre-signed S3 url on your backend using your credentials. This URL can then be returned to the application and used to upload a file.
It would circumvent having to create individual IAM users/permissions and/or managing bucket policies.
Check out the docs on it here.
Not sure how relevant to your situation.
You can create a role that allows upload to s3 and use SAML web-based identity to authenticate and allow privileges to assume the role and get temp credentials and token.
This will keep very limited time authenticated to S3 upload. ie until the temp credentials expire.
I am creating shortlived users on AWS on the fly and while debugging why these newly created logins tended to fail with an InvalidAccessKeyId realised that just adding a small sleep solved the problem.
xref How long should I wait after applying an AWS IAM policy before it is valid? re: time for consistency throughout AWS
My follow up question to the above: is there a way to synchronously create a consistent IAM policy? Or at least a way to know they are ready to use?
Amazon IAM is not designed for providing short-lived credentials. You should create IAM Users for long-lived requirements, such as logins for humans and logins for persistent applications.
An IAM User should not be used for application login purposes. For example, if you are creating an Instagram-like application, you should maintain your own database of users or utilize Amazon Cognito for user authentication.
So, how do you then grant users access to AWS resources? For example, if you have an Instagram-like application and you wish to grant application users the ability to upload/download their pictures in Amazon S3 but want to restrict access to a certain bucket and directory?...
The answer is to create temporary credentials using the AWS Security Token Service (STS). Credentials can be created with a given policy for a specific period of time. These credentials work immediately. For example, if an Instragram-like user logs into the app, the backend app could generate temporary credentials that allow the user to access a specific directory within a specific Amazon S3 bucket for a set period of time (eg 15 minutes). These credentials are then passed to the mobile app/web browser for direct access to AWS services.
I have data from multiple users inside a single S3 account. My desktop app has an authentication system which let the app know who the user is and which folder to access on S3. but the desktop app has the access code to the whole S3 folder.
somebody told me this is not secure since a hacker could break the request from the app to the S3 and use the credentials to download all the data.
Is this true? and if so how can I avoid it? (he said I need to a client server in the AWS cloud but this isn't clear to me... )
btw. I am using Boto python library to access S3.
thanks
I just found this:
Don't store your AWS secret key in the app. A determined hacker would be able to find it eventially. One idea is that you have a web service hosted somewhere whose sole purpose is to sign the client's S3 requests using the secret key, those requests are then relayed to the S3 service. Therefore you get your users to authenticate agaist your web service using credentials that you control. To re-iterate: the clients talk directly to S3, but get their requests "rubber-stamped"/approved by you.
I don't see S3 necessarily as a flat structure - if you use filesystem notation "folder/subfolder/file.ext" for the keys.
Vanity URLs are supported by S3 see http://docs.amazonwebservices.com/AmazonS3/2006-03-01/VirtualHosting.html - basically the URL "http://s3.amazonaws.com/mybucket/myfile.ext" becomes "http://mybucket.s3.amazonaws.com/myfile.ext" and you can then setup a CNAME in your DNS that maps "www.myname.com" to "mybucket.s3.amazonaws.com" which results in "http://www.myname.com/myfile.ext"
Perfect timing! AWS just announced a feature yesterday that's likely to help you here: Variables in IAM policies.
What you would do is create an IAM account for each of your users. This will allow you to have a separate access key and secret key for each user. Then you would assign a policy to your bucket that restricts access to a portion of the bucket, based on username. (The example that I linked to above has good example of this use case).