lets say I have a on-premise application that needs to access various AWS services such as S3, Cloudwatch etc. What is the correct way to handle this authentication? I have read recommendations to create a new iam role and then distribute the AWS keys on the server that the application runs. But wouldn't this be very bad practice in case the keys gets stolen or exposed in some way? It would also be more work to rotate credentials for example. Is it possible to assign roles in some other ways or this is the correct way to do it? Isn't it better to assign roles or that isn't possible when not running the app in AWS?
Creat an IAM user with “Programmatic Access” only, which will provide you with a key and secret pair.
As a general rule, your application can use one set of credentials to get another, more privileged set of credentials. The app must be able to authenticate somehow so it needs some basic form of service account credentials to start with.
One way you can do this is to create an IAM user with minimal privileges. This IAM user is able to assume a specific IAM service role, but nothing else. That service role actually confers permissions to interact with S3, CloudWatch etc. Your application is configured with, or somehow securely retrieves, the credentials associated with the IAM user. Your application then uses these to call STS and assume the IAM service role, getting back short-lived STS credentials (access key, secret key, and session token). You should leverage the additional 'external ID' with the IAM role, as one more security factor.
Your application is also responsible for getting a new set of credentials before the existing set expires. You can do that in a number of ways, for example by using new STS credentials for every single request you make (so they never expire) or simply paying attention to the credentials expiration time and refreshing prior.
Also, read Temporary Credentials for Users in Untrusted Environments.
If your application is running on an Amazon EC2 instance and it is the only application on that instance, then:
Create an IAM Role
Assign the appropriate permissions to the Role
Assign the IAM Role to the EC2 instance
Any software running on the instance will automatically have access to credentials to access AWS. These credentials automatically rotate every 6 hours.
If you are not running on an EC2 instance:
Create an IAM User
Assign the appropriate permissions to the User
Generate credentials for the User (Access Key, Secret Key) and store them in a credentials file on the computer being used by the application
Any software running on the instance will automatically have access to these credentials to access AWS.
Related
I am trying to follow best practices, but the documentation is not clear to me. I have a python script running locally that will move some files from my local drive to S3 for processing. Lambda picks it up from there and does the rest. So far I set up an AWS User for this process, and connected it to a "policy" that only has access to the needed resources.
Next step is to move my scripts to a docker container in my local server. But I thought best practice would be to use a Role with policies, instead of a User with policies. However, according to this documentation... in order to AssumeRole... I have to first be signed in as a user.
The calls to AWS STS AssumeRole must be signed with the access key ID
and secret access key of an existing IAM user or by using existing temporary
credentials such as those from another role. (You cannot call AssumeRole
with the access key for the root account.) The credentials can be in
environment variables or in a configuration file and will be discovered
automatically by the boto3.client() function.
So no matter what, I'll need to embed my user credentials into my docker image (or at least a separate secrets file)
If that is the case, then it seems adding a "Role" in the middle between the User and the Policies seems completely useless and redundant. Can anyone confirm or correct?
Roles and policies are for services running in AWS environments. For a Role you define a Trust Policy. The Trust Policy defines what principal (User, Role, AWS Service etc.) can assume it. You also define the permissions that the principal which assumes it has to access AWS services.
For services running inside AWS (EC2, Lambda, ECS), it is always possible to select an IAM role, which will be assumed by your service. This way your application will always get temporary credentials corresponding to the IAM role and you should never use an AWS Access Key Id and Secret.
However, this isn't possible for services running locally or outside of AWS environment. For your Docker container running locally, the only real option would be to create an Access Key ID and Secret and copy it there. There are still some things you can do to keep your account secure:
Follow the least privilege principal. Create a policy that provides access to only the absolutely required resources.
Create a user (programmatic access only) and add the policy. Use AWS Access Key ID and Secret of this user for your Docker container.
Make sure that the AWS Credentials are rotated regularly.
Make sure that the secrets aren't committed in source control, prefer a secrets file or a Vault system than environmental variables.
So I think that the simplest solution is my problem is to use AWS for everything but I wanted to understand what is possible:
I understand that IAM roles can be associated with an AWS service such as EC2 or Lambda so that an application/function running within that service can retrieve credentials to sign API requests to other AWS services.
I have a previous application running on Heroku and using Amazon S3. Currently I have an IAM user set up for this application which signs requests to the AWS API using the access keys associated with the IAM user account. I think that best practice is to use an IAM role rather than a user for application source code AWS API calls, however is it possible to set this up for the application hosted outside of AWS or would I need to migrate the application to AWS EC2 in order to use IAM roles?
It doesn't matter where the application is hosted but to assume an IAM role you will need IAM credentials (chicken and egg). Typically you would design a secure way for your app to retrieve these base credentials. This is one disadvantage of running your compute outside of AWS (because it can't automatically assume an IAM role).
One option would be to create an IAM user whose only permissions were to be able to assume a given IAM role. Supply those IAM user credentials to your application, outside of AWS, securely and have the application assume the IAM role, ideally with an ExternalId that itself is also securely stored and securely retrieved by your application. Additionally, you can manage access to the IAM role, for example defining which principals can assume the role, and under which conditions.
AWS announced a new feature AWS IAM Anywhere that should help if you need to avoid using access/secret keys. It's more complicated but follows security best practices.
AWS Identity and Access Management (IAM) now enables workloads that
run outside of AWS to access AWS resources using IAM Roles Anywhere.
IAM Roles Anywhere allows your workloads such as servers, containers,
and applications to use X.509 digital certificates to obtain temporary
AWS credentials and use the same IAM roles and policies that you have
configured for your AWS workloads to access AWS resources.
and more here:
create a trust anchor where you either reference your AWS
Certificate Manager Private Certificate Authority (ACM Private CA) or
register your own certificate authorities (CAs) with IAM Roles
Anywhere. By adding one or more roles to a profile and enabling IAM
Roles Anywhere to assume these roles, your applications can now use
the client certificate issued by your CAs to make secure requests to
AWS and get temporary credentials to access the AWS environment.
AWS Announcement: https://aws.amazon.com/about-aws/whats-new/2022/07/aws-identity-access-management-iam-roles-anywhere-workloads-outside-aws/
User Guide:
https://docs.aws.amazon.com/rolesanywhere/latest/userguide/introduction.html
From Heroku docs:
Because of the sensitive nature of your S3 credentials, you should never commit them to version control. Instead, set them as the values of config vars for the Heroku apps that will use them.
Use the heroku config:set to set both keys
heroku config:set AWS_ACCESS_KEY_ID=xxx AWS_SECRET_ACCESS_KEY=yyy
Adding config vars and restarting app... done, v21
AWS_ACCESS_KEY_ID => xxx
AWS_SECRET_ACCESS_KEY => yyy
The above is in line with AWS's own best practices for managing AWS access keys, specifically not embedding access keys directly in code.
You can't use IAM roles in the sense that it is picked up automatically by AWS, outside of AWS, without specifying credentials specifically.
Your next best option is environment variables (as detailed above), specifying the access key ID and secret access key for a user with a role granting the least privilege required for the files they need to read from S3 e.g. specific bucket name, specific files, even specific IP addresses if possible, etc.
I am confused about the use cases and advantages of STS. As per the documentation, it is to temporarily acquire a role to perform tasks within AWS which are not available for the IAM user or service. Please note that I am talking about programmatic access (NOT console access)
For example, an IAM user may not have S3 permissions. As per my understanding:
He can get temporary access key/token by contacting AWS STS and get access key and secret for S3.
With those temporary credentials, he can access S3.
My questions are:
To get temporary credentials from AWS STS, he still need his existing access token (permanent) and secret, right?
If his existing access token and secret are leaked, an attacker can still use it to first get temporary credentials from STS and then access S3, right? I understand that the attacker won't be able to directly access S3 using his permanent access token and secret.
I am trying to wrap my head around its correct use cases. I know that I'm confused, but maybe I'm thinking in loops.
Thanks in advance.
They don't so much "contact AWS STS and get access key and secret for S3". Rather, they call AssumeRole() on an IAM Role that has permission to access Amazon S3. Then, using the temporary credentials that are returned, they can access S3.
Your confusion seems to be mostly around the use-case for IAM Roles. I like to explain it by way of a story...
I am a Fire Warden in my office at work. If the Fire Alarm activates, I go to a cupboard, put on a red helmet, then walk around the office and direct people to the stairwell. Since the alarm is sounding and I'm wearing a red hat, people (mostly) do what I tell them. However, if it was a normal day with no alarm sounding and I wasn't wearing the red hat, and I asked them to exit the office via the stairwell, they would likely look at me strangely and ignore my request. The difference is that I assumed the role of a Fire Warden, which gave me extra permissions.
So, as a normal person, I can't order people out of the office. However, once I assume the role, I have extra permissions.
This is a good practice for IT systems, too. The Systems Administrator in your company probably has permissions in your AWS Account to do anything. However, it would be a bad practice for them to use an account with such powers on a day-to-day basis. Instead, their IAM User account should just have normal permissions but, if they want to do Admin-type stuff, they have the ability to Assume an Admin Role and then do powerful stuff. When they're finished, they should exit the role and keep being a normal user. This is much 'safer', since they can't accidentally do something powerful when they are a 'normal user'.
Amazon Security Token Service (STS) is also used to provide permissions to software running on Amazon EC2 instances. In this case, an IAM Role is assigned to the EC2 instance and the EC2 service 'assumes' the role on behalf of the instance. It then provides the temporary credentials via the EC2 Instance Metadata service. In this example, there was no IAM User that assumed the role. Instead, the EC2 service assumed it on behalf of the instance.
STS can also provide cross-account permissions. For example, an IAM User in Account A could call AssumeRole() on an IAM Role in Account B. If they have permission to do this, then they will be given temporary credentials that are associated with Account B. This is required because credentials from one Account can never be used to manage resources in another Account.
There are other reasons for using temporary credentials too, such as using MFA tokens, federated logins where there are no IAM Users, and reducing your own set of permissions.
I will try to extend and generalise the first answer. The example with the Fire Warden is good to understand, but I feel it needs some extension.
Generally the AWS STS is able to return role credentials based on other identity or role credentials (aws or other identity provider).
The original credentials can be either AWS credentials from the same account, another account, federated token (e. g. supported social networks) or even a custom identity broker.
see https://docs.aws.amazon.com/cli/latest/reference/sts/index.html
Common use cases:
privilege elevation - this is already mentioned, AssumeRole allows to become another role within the same or different aws account
authorization to aws resources for identities authenticated a other way (AD, SAML, OIDC,..), see services AssumeRoleWithSAML or AssumeRoleWithWebIdentity.
authorization to aws resources with custom authorization, see GetFederationToken.
To get temporary credentials from AWS STS, he still need his existing access token (permanent) and secret, right?
By default AssumeRole, the user needs to be authenticated and having permission to assume the role.
If his existing access token and secret are leaked, an attacker can still use it to first get temporary credentials from STS and then access S3, right?
yes
I understand that the attacker won't be able to directly access S3 using his permanent access token and secret
if you configure the S3 or IAM permissions that way
We have an external application resides outside of Amazon network and it needs to access our SQS and send message there, in order for our AWS resource to recognize the request from that application it needs to sign its request with the credentials of the IAM role we created, I'm wondering what is the best way for that external application to retrieve temp credentials from us? I have tried to implement it using Amazon Cognito but it looks like Cognito fits more in scenarios like user sign-up and sign-in with an User Interface, anyone has any suggestions? Thanks in advance.
To be able to obtain temporary credentials, you need a form of permanent credentials that can access (or generate) the temporary credentials.
Given your situation, you might consider creating an IAM User in your account and giving those credentials to the third-party. Grant the appropriate permissions to those credentials and they can use them directly with Amazon SQS.
Or, if you'd rather not give IAM credentials to third-parties, you could ask them to create an AWS account and an IAM User. You could then grant their IAM user access to the Amazon SQS queue.
Another option is that the third-party could access an application or API that you provide. Once they authenticate, you can provide temporary credentials created with the Security Token Service. Cognito would be an option for performing this authentication and it can also provide credentials for an associated IAM Role, thus giving them access to the Amazon SQS queue.
This page shows how to send an email using SES. The example works by reading the credentials from ~/.aws/credentials, which are the root (yet "shared"??) credentials.
The documentation advises in various places against using the root credentials.
Acquiring temporary credentials
using roles is mentioned as an option, yet assume_role() is not defined for SES client objects.
How do I send an email through SES with temporary SES-specific credentials?
Update
The context for my question is an application running on an EC2 instance.
There are a few pieces to this.
First you need an IAM policy. You can use one of the built-in policies, such as AmazonSESFullAccess or you can create your own. The holder of a particular policy will be able to access the resources and actions defined in the policy. You can create this policy manually, or work through the AWS console and it will walk you through it. IAM --> Policies --> Create Policy
Secondly, you will need a role. Also, easily done in the console. IAM --> Roles --> Create role. Trusted entity is AWS service. Highlight EC2. In the next screen, select the policy you want to associate with this role. This is the policy you created above. If your EC2 already has a role, then you can add the IAM policy to this role. Assigning an IAM policy to a role, is what they refer to as a trust policy.
Now any code that runs on your EC2 instance will be able to send messages to your SES service. The EC2 assumes the role assigned to it. And the SES policy is defined for that role. This will allow EC2 to get temporary credentials (behind the scenes).
The back story is as follows. Any API call to an AWS service needs to have a key and secret. When you make API calls from your local computer, you may use your personal key and secret (or even root ones). When you need to make API calls from another service, you do not have that key and secret. It would not be secure or practical to store the credentials on an EC2. Or even worse, in an S3 bucket. That is why AWS came up with the Role concept. Roles can request temporary credentials from an internal service called Simple Token Service (STS). A role is attached to an EC2 instance for example. And if the right policy is attached to that role, the EC2 instance can request to get temporary credentials to make an API call to another service. All of this happens behind the scenes.
Two options...
You could create IAM User credentials with the appropriate permissions and put them in the ~./aws/credentials file. Then your application will find them and use them to connect with Amazon SES.
Or, your application could use a set of IAM User credentials to call assume_role() (which is an IAM command). This will return a set of temporary credentials that could be used with Amazon SES. However, if you are going to provide a set of credentials that will be used to call assume_role(), then you may as well just use those credentials directly with Amazon SES.
An IAM User can be used for people OR applications.