I am trying to follow best practices, but the documentation is not clear to me. I have a python script running locally that will move some files from my local drive to S3 for processing. Lambda picks it up from there and does the rest. So far I set up an AWS User for this process, and connected it to a "policy" that only has access to the needed resources.
Next step is to move my scripts to a docker container in my local server. But I thought best practice would be to use a Role with policies, instead of a User with policies. However, according to this documentation... in order to AssumeRole... I have to first be signed in as a user.
The calls to AWS STS AssumeRole must be signed with the access key ID
and secret access key of an existing IAM user or by using existing temporary
credentials such as those from another role. (You cannot call AssumeRole
with the access key for the root account.) The credentials can be in
environment variables or in a configuration file and will be discovered
automatically by the boto3.client() function.
So no matter what, I'll need to embed my user credentials into my docker image (or at least a separate secrets file)
If that is the case, then it seems adding a "Role" in the middle between the User and the Policies seems completely useless and redundant. Can anyone confirm or correct?
Roles and policies are for services running in AWS environments. For a Role you define a Trust Policy. The Trust Policy defines what principal (User, Role, AWS Service etc.) can assume it. You also define the permissions that the principal which assumes it has to access AWS services.
For services running inside AWS (EC2, Lambda, ECS), it is always possible to select an IAM role, which will be assumed by your service. This way your application will always get temporary credentials corresponding to the IAM role and you should never use an AWS Access Key Id and Secret.
However, this isn't possible for services running locally or outside of AWS environment. For your Docker container running locally, the only real option would be to create an Access Key ID and Secret and copy it there. There are still some things you can do to keep your account secure:
Follow the least privilege principal. Create a policy that provides access to only the absolutely required resources.
Create a user (programmatic access only) and add the policy. Use AWS Access Key ID and Secret of this user for your Docker container.
Make sure that the AWS Credentials are rotated regularly.
Make sure that the secrets aren't committed in source control, prefer a secrets file or a Vault system than environmental variables.
Related
I have an EKS deployment with a service account with policy and role that enable access to S3.
This works well for root account in the container. The container can execute aws s3 cp ... with no issue.
The problem is that another user cannot. It gets AccessDenied from the S3 service, meaning it does not have the correct credentials.
So my question is: how to give rights to another user in the container (which is linux based) in this case?
(I don't think it's specific to Eks as service accounts are generic to kubernetes.)
You can check out the reference for how IAM roles for service accounts work in k8s here.
In short, in order to allow another user to use the IAM role the same environment variables have to be configured for that user, and it needs to be able to access the path specified in the AWS_WEB_IDENTITY_TOKEN_FILE variable.
Once both of these prerequisites are met, the user should be able to use the same identity as the root user.
So I think that the simplest solution is my problem is to use AWS for everything but I wanted to understand what is possible:
I understand that IAM roles can be associated with an AWS service such as EC2 or Lambda so that an application/function running within that service can retrieve credentials to sign API requests to other AWS services.
I have a previous application running on Heroku and using Amazon S3. Currently I have an IAM user set up for this application which signs requests to the AWS API using the access keys associated with the IAM user account. I think that best practice is to use an IAM role rather than a user for application source code AWS API calls, however is it possible to set this up for the application hosted outside of AWS or would I need to migrate the application to AWS EC2 in order to use IAM roles?
It doesn't matter where the application is hosted but to assume an IAM role you will need IAM credentials (chicken and egg). Typically you would design a secure way for your app to retrieve these base credentials. This is one disadvantage of running your compute outside of AWS (because it can't automatically assume an IAM role).
One option would be to create an IAM user whose only permissions were to be able to assume a given IAM role. Supply those IAM user credentials to your application, outside of AWS, securely and have the application assume the IAM role, ideally with an ExternalId that itself is also securely stored and securely retrieved by your application. Additionally, you can manage access to the IAM role, for example defining which principals can assume the role, and under which conditions.
AWS announced a new feature AWS IAM Anywhere that should help if you need to avoid using access/secret keys. It's more complicated but follows security best practices.
AWS Identity and Access Management (IAM) now enables workloads that
run outside of AWS to access AWS resources using IAM Roles Anywhere.
IAM Roles Anywhere allows your workloads such as servers, containers,
and applications to use X.509 digital certificates to obtain temporary
AWS credentials and use the same IAM roles and policies that you have
configured for your AWS workloads to access AWS resources.
and more here:
create a trust anchor where you either reference your AWS
Certificate Manager Private Certificate Authority (ACM Private CA) or
register your own certificate authorities (CAs) with IAM Roles
Anywhere. By adding one or more roles to a profile and enabling IAM
Roles Anywhere to assume these roles, your applications can now use
the client certificate issued by your CAs to make secure requests to
AWS and get temporary credentials to access the AWS environment.
AWS Announcement: https://aws.amazon.com/about-aws/whats-new/2022/07/aws-identity-access-management-iam-roles-anywhere-workloads-outside-aws/
User Guide:
https://docs.aws.amazon.com/rolesanywhere/latest/userguide/introduction.html
From Heroku docs:
Because of the sensitive nature of your S3 credentials, you should never commit them to version control. Instead, set them as the values of config vars for the Heroku apps that will use them.
Use the heroku config:set to set both keys
heroku config:set AWS_ACCESS_KEY_ID=xxx AWS_SECRET_ACCESS_KEY=yyy
Adding config vars and restarting app... done, v21
AWS_ACCESS_KEY_ID => xxx
AWS_SECRET_ACCESS_KEY => yyy
The above is in line with AWS's own best practices for managing AWS access keys, specifically not embedding access keys directly in code.
You can't use IAM roles in the sense that it is picked up automatically by AWS, outside of AWS, without specifying credentials specifically.
Your next best option is environment variables (as detailed above), specifying the access key ID and secret access key for a user with a role granting the least privilege required for the files they need to read from S3 e.g. specific bucket name, specific files, even specific IP addresses if possible, etc.
I am planning to use DynamoDB for the first time for my project. I initially made connection to DynamoDB from my Java application using the IAM User secret keys. But, then decided to add permissions to the IAM Role of my server, where the application runs.
Am doing it right? What's the best practice for this?
And if IAM Role is the right way to go, how do I handle applications connecting from my AWS Workspace ( my dev environment ), can I add IAM Role for that too?
IAM Role is the correct way to go. You create a role following the least permissions privilege. This means that you assign to the role only the absolutely necessary permissions. In your case the role should only have access to specific DynamoDB Tables and Indexes.
In EC2, lambda functions and in general in AWS environment you assign this role. The service you are using, will assume this role and be able to access DynamoDB. No need to create access keys.
For your local DEV environment (outside of AWS), you should create a user, assign the role you've created and create a Key Id and a Secret. This way your local environment will only have access to the needed resources.
If you also need your personal AWS credentials in a local machine, you can use profiles to manage them.
Handling creds when using the AWS SDK for Java is explained in the AWS Java Developer Guide in these topics:
Get started with the SDK for Java
Using credentials
This guide explains best practices.
lets say I have a on-premise application that needs to access various AWS services such as S3, Cloudwatch etc. What is the correct way to handle this authentication? I have read recommendations to create a new iam role and then distribute the AWS keys on the server that the application runs. But wouldn't this be very bad practice in case the keys gets stolen or exposed in some way? It would also be more work to rotate credentials for example. Is it possible to assign roles in some other ways or this is the correct way to do it? Isn't it better to assign roles or that isn't possible when not running the app in AWS?
Creat an IAM user with “Programmatic Access” only, which will provide you with a key and secret pair.
As a general rule, your application can use one set of credentials to get another, more privileged set of credentials. The app must be able to authenticate somehow so it needs some basic form of service account credentials to start with.
One way you can do this is to create an IAM user with minimal privileges. This IAM user is able to assume a specific IAM service role, but nothing else. That service role actually confers permissions to interact with S3, CloudWatch etc. Your application is configured with, or somehow securely retrieves, the credentials associated with the IAM user. Your application then uses these to call STS and assume the IAM service role, getting back short-lived STS credentials (access key, secret key, and session token). You should leverage the additional 'external ID' with the IAM role, as one more security factor.
Your application is also responsible for getting a new set of credentials before the existing set expires. You can do that in a number of ways, for example by using new STS credentials for every single request you make (so they never expire) or simply paying attention to the credentials expiration time and refreshing prior.
Also, read Temporary Credentials for Users in Untrusted Environments.
If your application is running on an Amazon EC2 instance and it is the only application on that instance, then:
Create an IAM Role
Assign the appropriate permissions to the Role
Assign the IAM Role to the EC2 instance
Any software running on the instance will automatically have access to credentials to access AWS. These credentials automatically rotate every 6 hours.
If you are not running on an EC2 instance:
Create an IAM User
Assign the appropriate permissions to the User
Generate credentials for the User (Access Key, Secret Key) and store them in a credentials file on the computer being used by the application
Any software running on the instance will automatically have access to these credentials to access AWS.
According to many advices, we should not configure IAM USER but using IAM Role instead to avoid someone managed to grab the user confidential in .aws folder.
Lets say I don't have any EC2 instances. Can I still able to perform S3 operation via AWS CLI? Says aws s3 ls
MacBook-Air:~ user$ aws s3 ls
Unable to locate credentials. You can configure credentials by running "aws configure".
You are correct that, when running applications on Amazon EC2 instances or as AWS Lambda functions, an IAM role should be assigned that will provide credentials via the EC2 metadata service.
If you are not running on EC2/Lambda, then the normal practice is to use IAM User credentials that have been created specifically for your application, with least possible privilege assigned.
You should never store the IAM User credentials in an application -- there have been many cases of people accidentally saving such files into GitHub, and bad actors grab the credentials and have access to your account.
You could store the credentials in a configuration file (eg via aws configure) and keep that file outside your codebase. However, there are still risks associated with storing the credentials in a file.
A safer option is to provide the credentials via environment variables, since they can be defined through a login profile and will never be included in the application code.
I don't think you can use service roles on your personal machine.
You can however use multi-factor authentication for AWS CLI
You can use credentials on any machine not just EC2.
Follow the steps as described by the documentation for your OS.
http://docs.aws.amazon.com/cli/latest/userguide/installing.html