I am creating shortlived users on AWS on the fly and while debugging why these newly created logins tended to fail with an InvalidAccessKeyId realised that just adding a small sleep solved the problem.
xref How long should I wait after applying an AWS IAM policy before it is valid? re: time for consistency throughout AWS
My follow up question to the above: is there a way to synchronously create a consistent IAM policy? Or at least a way to know they are ready to use?
Amazon IAM is not designed for providing short-lived credentials. You should create IAM Users for long-lived requirements, such as logins for humans and logins for persistent applications.
An IAM User should not be used for application login purposes. For example, if you are creating an Instagram-like application, you should maintain your own database of users or utilize Amazon Cognito for user authentication.
So, how do you then grant users access to AWS resources? For example, if you have an Instagram-like application and you wish to grant application users the ability to upload/download their pictures in Amazon S3 but want to restrict access to a certain bucket and directory?...
The answer is to create temporary credentials using the AWS Security Token Service (STS). Credentials can be created with a given policy for a specific period of time. These credentials work immediately. For example, if an Instragram-like user logs into the app, the backend app could generate temporary credentials that allow the user to access a specific directory within a specific Amazon S3 bucket for a set period of time (eg 15 minutes). These credentials are then passed to the mobile app/web browser for direct access to AWS services.
Related
I have 1 s3 bucket per customer. Customers are external entities and they dont share data with anyone else. I write to S3 and customer reads from S3. As per this architecture, I can only scale to 1000 buckets as there is a limit to s3 buckets per account. I was hoping to use APs to create 1 AP per customer and put data in one bucket. The customer can then read the files from the bucket using AP.
Bucket000001/prefix01 . -> customeraccount1
Bucket000001/prefix02 . -> customeraccount2
...
S3 access points require you to set policy for a IAM user in access point as well as the bucket level. If I have 1000s of IAM users, do I need to set policy for each of them in the bucket? This would result in one giant policy. there is a max policy size in the bucket, so I may not be able to do that.
Is this the right use case where access points can help?
The recommended approach would be:
Do NOT assign IAM Users to your customers. These types of AWS credentials should only be used by your internal staff and your own applications.
You should provide a web application (or an API) where customers can authenticate against your own user database (or you could use Amazon Cognito to manage authentication).
Once authenticated, the application should grant access either to a web interface to access Amazon S3, or the application should provide temporary credentials for accessing Amazon S3 (more details below).
Do not use one bucket per customer. This is not scalable. Instead, store all customer data in ONE bucket, with each user having their own folder. There is no limit on the amount of data you can store in Amazon S3. This also makes it easier for you to manage and maintain, since it is easier to perform functions across all content rather than having to go into separate buckets. (An exception might be if you wish to segment buckets by customer location (region) or customer type. But do not use one bucket per customer. There is no reason to do this.)
When granting access to Amazon S3, assign permissions at the folder-level to ensure customers only see their own data.
Option 1: Access via Web Application
If your customers access Amazon S3 via a web application, then you can code that application to enforce security at the folder level. For example, when they request a list of files, only display files within their folder.
This security can be managed totally within your own code.
Option 2: Access via Temporary Credentials
If your customers use programmatic access (eg using the AWS CLI or a custom app running on their systems), then:
The customer should authenticate to your application (how this is done will vary depending upon how you are authenticating users)
Once authenticated, the application should generate temporary credentials using the AWS Security Token Service (STS). While generating the credentials, grant access to Amazon S3 but specify the customer's folder in the ARN (eg arn:aws:s3:::storage-bucket/customer1/*) so that they can only access content within their folder.
Return these temporary credentials to the customer. They can then use these credentials to make API calls directly to Amazon S3 (eg from the AWS Command-Line Interface (CLI) or a custom app). They will be limited to their own folder.
This approach is commonly done with mobile applications. The mobile app authenticates against the backend, receives temporary credentials, then uses those credentials to interact directly against S3. Thus, the back-end app is only used for authentication.
Examples on YouTube:
5 Minutes to Amazon Cognito: Federated Identity and Mobile App Demo
Overview Security Token Service STS
AWS: Use the Session Token Service to Securely Upload Files to S3
We have some way to achieve your goal.
use IAM group to grant access to a folder. Create a group, add a user to a group, and assign a role to the group to access the folder.
Another way is to use bucket policy (${aws:username} in Condition) to grant Access to User-Specific Folders. Refer to this link https://aws.amazon.com/blogs/security/writing-iam-policies-grant-access-to-user-specific-folders-in-an-amazon-s3-bucket/
lets say I have a on-premise application that needs to access various AWS services such as S3, Cloudwatch etc. What is the correct way to handle this authentication? I have read recommendations to create a new iam role and then distribute the AWS keys on the server that the application runs. But wouldn't this be very bad practice in case the keys gets stolen or exposed in some way? It would also be more work to rotate credentials for example. Is it possible to assign roles in some other ways or this is the correct way to do it? Isn't it better to assign roles or that isn't possible when not running the app in AWS?
Creat an IAM user with “Programmatic Access” only, which will provide you with a key and secret pair.
As a general rule, your application can use one set of credentials to get another, more privileged set of credentials. The app must be able to authenticate somehow so it needs some basic form of service account credentials to start with.
One way you can do this is to create an IAM user with minimal privileges. This IAM user is able to assume a specific IAM service role, but nothing else. That service role actually confers permissions to interact with S3, CloudWatch etc. Your application is configured with, or somehow securely retrieves, the credentials associated with the IAM user. Your application then uses these to call STS and assume the IAM service role, getting back short-lived STS credentials (access key, secret key, and session token). You should leverage the additional 'external ID' with the IAM role, as one more security factor.
Your application is also responsible for getting a new set of credentials before the existing set expires. You can do that in a number of ways, for example by using new STS credentials for every single request you make (so they never expire) or simply paying attention to the credentials expiration time and refreshing prior.
Also, read Temporary Credentials for Users in Untrusted Environments.
If your application is running on an Amazon EC2 instance and it is the only application on that instance, then:
Create an IAM Role
Assign the appropriate permissions to the Role
Assign the IAM Role to the EC2 instance
Any software running on the instance will automatically have access to credentials to access AWS. These credentials automatically rotate every 6 hours.
If you are not running on an EC2 instance:
Create an IAM User
Assign the appropriate permissions to the User
Generate credentials for the User (Access Key, Secret Key) and store them in a credentials file on the computer being used by the application
Any software running on the instance will automatically have access to these credentials to access AWS.
We're developing an ASP.NET Core web application that will be running on-premises in a customer environment, and that application needs access to certain resources in our AWS account. Because this crosses an organizational boundary, we're creating an IAM Role in our AWS account and granting the customer access to assume that role from their account.
I believe the way this must work is that the ASP.NET Core application has "permanent" credentials to the customer's AWS account, and then obtains temporary credentials for the IAM Role in our account in order to do work. However, the application needs to do most of its work in an IHostedService/BackgroundService, which is a singleton running for the duration of the application lifetime, so if we use the standard dependency injection, we'll have a single long-lived instance of a given AWS service client. Does anyone know if the AWS SDK has any ability to continually re-obtain temporary credentials for an IAM Role? If not, I assume we'll have to stop obtaining the service clients via DI and keep periodically re-creating them ourselves to refresh the credentials.
I believe the way this must work is that the ASP.NET Core application has "permanent" credentials to the customer's AWS account, and then obtains temporary credentials for the IAM Role in our account in order to do work. However, the application needs to do most of its work in an
From the usage document, SDK's AssumeRole function just make a request to get temporary credential as simple strings. You must input these credential yourself to any object/service client that you want to use that specific credential returned from AssumeRoleRequest
Which mean there is no built-in auto-refresh that suits your needs
You may want to look into awsume and the AutoAwsume plugin for it.
This is something that can be used to auto-refresh temporary IAM role credentials on a machine.
A similar solution was used by developers in a former project of mine as a security best practice to avoid giving out long-lived credentials.
I am developing a web application with two mobile(Android & iOs) based applications of the same. Currently the files uploaded are open to all, which in terms means that anyone with the direct image link can open it using a web browser.
How can I protect or limit the file access to the users of my mobile applications or web application ?
NB: As a beginner, I am not sure about the configuration details to be provided along with question, If I need to give more details on my s3 config. please specify it, I can add it to the s question to make the question more meaningful, so sorry for the inconvenience.
I think an easier approach than pre-signed urls would be to use Amazon Cognito to provide access to AWS resources to your trusted applications, even to unauthenticated users.
To do this you would create an Identity Pool for your application (just need one pool for all 3 of your clients) and then configure it so that when a client provides a valid Identity Pool Id they can assume an IAM role with permissions to access AWS resources.
Then you control what S3 bucket permissions the IAM role they assume would have - you could allow unauthenticated users access to read the S3 objects, or force them to create accounts to be able to read/write to S3 buckets (this is very easy with Cognito - users can sign up with facebook, gmail, their own email, etc.)
There's a step-by-step guide here for setting up an identity pool with Cognito, and then allowing unauthenticated users to assume an IAM role that can access the contents of an S3 bucket
The above causes the same set of permissions for all guest user accounts - that have assumed an IAM role through Amazon Cognito by identifying themselves as part of an identity pool.
edit: I should point out that if you authenticate via Cognito, you'll need to access the S3 bucket through the S3 Transfer Manager from the AWS SDK
I was reading up per subject matter and has a query on what is the best practice to handle user's authorization to AWS resources.
Scenario:
A 2-tiers windows application that access to AWS S3 and dynamoDB.
There are 2 groups of users - Admin and Normal User. Admin has read + write access, and Normal User has just the read access.
I'm trying to see if I could avoid a 3-tiers design. At such, I would like to access AWS resources directly from my application. In another word, I do not access AWS resources thru web services (that could do my user authorization check there).
Design:
I uses Web Identity Federation (google) to authenticate user and uses STS to get temporary credentials.
I've created 2 IAM Roles - AdminRole (with write+read policy) and UserRole (with read policy).
At this point, my thoughts are stuck on what is the best practice and securely choose which role to assume from my application.
Solution 1:
Create a UserRole table in dynamoDb with UserId and Role attributes.
After user has authenticated with google, I'll check the UserRole table against the userid returned from google to get the role of this user. Assuming I've pre-setup all the user's roles in the table.
I do not want to hardcode or expose my AWS secret key onto my app, but for the above execution, I've created a secret key with a role and policy only to the [UserRole] table.
At this point, I would know which role to assume from my app when I get the temporary credential using STS.
However, with the above solution, I see that there is one security loophole. If someone is able to get hold of my application id used for my IAM role, and with some brute-force effort on my IAM roles' name, that person can easily gain temporary credential with AdminRole.
(added) Solution 2:
I create just 1 IAM role - GoogleUserRole
In the policy portion, I allow write access to admin user by using their Federated userid.
I'm still new with writing AWS policies, but I think I've read somewhere that I could have this fine-grained control to the specified user(s). This could be feasible if my users based are small, but not very feasible when my user base grows.
Welcome any thoughts and advises.
Thanks.