AWS CDK deploy using role - amazon-web-services

You can find the cdk app that you can use to replicate my issue here varvay/issue-replication.git. The usage instruction explained in the README
I need to deploy CDK app using a role by issuing this command
cdk -r arn:aws:iam::000000000000:role/fooRole deploy
but then an error thrown
Assuming role failed: User: arn:aws:iam::000000000000:user/fooUser is not authorized to
perform: sts:AssumeRole on resource: arn:aws:iam::000000000000:role/barRole
to be sure, I tried to simulate it by assuming the arn:aws:iam::000000000000:role/barRole role using arn:aws:iam::000000000000:role/fooRole in AWS IAM Policy Simulator and it works just fine. One thing that bothers me is that the error said that a User tried to assume the role, not Role.
Why is that? or should I assume the fooRole, update the AWS-related environment variable and then deploy? if so then what's the point of having -r option on cdk
as additional information, here's the trust relationship of the barRole
{
"Version": "2008-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam:: 000000000000:root"
},
"Action": "sts:AssumeRole"
}
]
}
also I even tried to attach AdministratorAccess AWS managed policy to the fooRole used to deploy

I managed to fulfill my needs by creating a bash script to switch to the destination role and use the credential to perform the CDK command as the script written below,
#!/bin/bash
unset AWS_ACCESS_KEY_ID
unset AWS_SECRET_ACCESS_KEY
unset AWS_SESSION_TOKEN
AWS_CREDENTIAL=$(aws sts assume-role \
--role-arn <destination role ARN> \
--role-session-name <role session name> \
--duration-seconds 3600)
export AWS_ACCESS_KEY_ID=$(echo $AWS_CREDENTIAL \
| jq -r '.Credentials''.AccessKeyId')
export AWS_SECRET_ACCESS_KEY=$(echo $AWS_CREDENTIAL \
| jq -r '.Credentials''.SecretAccessKey')
export AWS_SESSION_TOKEN=$(echo $AWS_CREDENTIAL \
| jq -r '.Credentials''.SessionToken')
cdk deploy
unset AWS_ACCESS_KEY_ID
unset AWS_SECRET_ACCESS_KEY
unset AWS_SESSION_TOKEN

So there are 2 ways you might be running cdk deploy command from.
1- You're running this command from your local computer's CLI using IAM keys. In this case, this role must be assumable by the AWS account (IAM User) being used.
2- You're running this command from any AWS service (cicd agent on EC2 instance for e.g.:) then the role attached with the instance should be allowed to assume this deployment role.
mention how you're running this command and you might get a better answer.
UPDATE:
Based on the updated question:
Add assume role part in your IAM USER not your deployment role. Your IAM User from which you're trying to deploy should be allowed to assume the role through which the CDK will be deployed.
To diagramise it a bit:
(IAM-USER -> Assume -> Role) -> cdk deploy

The error is in the process of cross account role accessing, as is written in your error message.
I assume that you start with AWS configuration for one account, lets call it "Provisioning" and then you need to assume role in different account (dev or prod) depending on branches or something ?
I smell an error in setup of cross account roles.
https://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_cross-account-with-roles.html
One possibility is : the Rolle you want to assume, does not have your provisioning account as trusted entity.
Another is : the user which is trying to assume the role, does not have the policy for that.
Just follow the tutorial from AWS and see what is missing in your setup :)

Related

Can you use AWS MGN (Application Migration) without an explicitly created access key and secret key?

I have a situation where I need to migrate an on-prem environment to AWS. The tool of choice is AWS MGN (Application Migration). I am following AWS's documentation on loading and using the agent on the source application:
https://docs.aws.amazon.com/mgn/latest/ug/windows-agent.html
The documentation clearly states the need for an access key and secret key. Our environment is set up with a root account with limited users and the team accesses the AWS environment in other accounts (e.g. dev/test/prod/etc.) via SSO. The MGN service also creates four roles:
AWSApplicationMigrationConversionServerRole
AWSApplicationMigrationMGHRole
AWSApplicationMigrationReplicationServerRole
AWSServiceRoleForApplicationMigrationService
I know you can use 'aws sts assume role' to glean off the keys needed, e.g.:
AWS_SESSION_TOKEN=$(echo "$test_assume_role_json" | jq -r '.Credentials.SessionToken')
export AWS_SESSION_TOKEN
AWS_ACCESS_KEY_ID=$(echo "$test_assume_role_json" | jq -r '.Credentials.AccessKeyId')
export AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY=$(echo "$test_assume_role_json" | jq -r '.Credentials.SecretAccessKey')
These roles are create by AWS and I can't edit them to try and give my SSO user access (I can't 'aws sts assume role' these from the CLI), so assuming these roles directly is not possible. I also cannot create a role that uses AWS-managed policies.
I am not really sure if what I want to do is possible, or if there is a simple solution that I'm just not seeing.
EDIT: I was actually successful in creating a test role with the following policies on the account that I wanted to migrate into:
AdministratorAccess (AWS managed policy)
AWSApplicationMigrationAgentPolicy (AWS managed policy)
AWSApplicationMigrationFullAccess (AWS managed policy)
AWSApplicationMigrationMGHAccess (AWS managed policy)
AWSApplicationMigrationReadOnlyAccess (AWS managed policy)
AWSApplicationMigrationEC2Access (AWS managed policy)
AWSApplicationMigrationConversionServerPolicy (AWS managed policy)
AWSApplicationMigrationReplicationServerPolicy (AWS managed policy)
I then ran assume-role to get the access key and secret key :
C:\>aws sts assume-role --role-arn "arn:aws:iam::<REDACTED>:role/ApplicationMigrationTester" --role-session-name test --profile test
{
"Credentials": {
"AccessKeyId": "<REDACTED>",
"SecretAccessKey": "<REDACTED>",
"SessionToken": "<REDACTED>",
"Expiration": "2021-07-15T22:55:08+00:00"
},
"AssumedRoleUser": {
"AssumedRoleId": "<REDACTED>":test",
"Arn": "arn:aws:sts::<REDACTED>:assumed-role/ApplicationMigrationTester/test"
}
}
I tried to run the agent again as Administrator using those vaules when prompted:
C:\>.\AwsReplicationWindowsInstaller.exe
The installation of the AWS Replication Agent has started.
AWS Region Name: us-east-1
AWS Access Key ID: <REDACTED>
AWS Secret Access Key:
Verifying that the source server has enough free disk space to install the AWS Replication Agent.
(a minimum of 2 GB of free disk space is required)
Identifying volumes for replication.
Choose the disks you want to replicate. Your disks are: c:
To replicate some of the disks, type the path of the disks, separated with a comma (for example, C:,D:). To replicate all disks, press Enter:
Disk to replicate identified: c:0 of size 620 GiB
All volumes for replication were successfully identified.
Downloading the AWS Replication Agent onto the source server...
Failed to validate AWS credentials
Installation failed.
Learn more about installation issues in our documentation at https://docs.aws.amazon.com/mgn/latest/ug/Troubleshooting-Agent-Issues.html#Error-Installation-Failed
Press Enter to close...
As you can see, it errored out. I put in a ticket to AWS to see if my approach is even currently feasible. When they respond, I will update.
You only should provide the credentials from the command line (powerShell or CMD)
C:\Downloads>.\AwsReplicationWindowsInstaller.exe --region us-east-1 --aws-access-key-id AKIA4HN2LIGPIERPMPXT --aws-secret-access-key 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

Pass AWS CodeBuild IAM Role inside Docker container [unable to locate credentials]

The role configured on CodeBuild project works fine with the runtime environment but doesn't work when we run a command from inside the container, it says "unable to locate credentials".
Let me know how can we use the role out of the box inside the container.
You can make use of credential source "EcsContainer" to assume role seamlessly without having to export new credentials in your buildspec.yml.
credential_source - The credential provider to use to get credentials for the initial assume-role call. This parameter cannot be provided alongside source_profile. Valid values are:
Environment to pull source credentials from environment variables.
Ec2InstanceMetadata to use the EC2 instance role as source credentials.
EcsContainer to use the ECS container credentials as the source credentials.
From: https://docs.aws.amazon.com/cli/latest/topic/config-vars.html
Steps:
Step-0: Create a new Role 'arn:aws:iam::0000000000:role/RoleToBeAssumed' and attach required policies to provide the permission required for the commands you are running during the build.
Step-1: Add sts:assumeRole permissions to your CodeBuild Service Role. Here is a sample policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "sts:*",
"Resource": "arn:aws:iam::0000000000:role/RoleToBeAssumed"
}
]
}
Step-2: Configure your build container to use the credential metadata as source for assuming the role. Here is a buildspec example:
version: 0.2
phases:
install:
runtime-versions:
nodejs: 8
commands:
- aws sts get-caller-identity
- mkdir ~/.aws/ && touch ~/.aws/config
- echo "[profile buildprofile]" > ~/.aws/config
- echo "role_arn = arn:aws:iam::0000000000:role/RoleToBeAssumed" >> ~/.aws/config
- echo "credential_source = EcsContainer" >> ~/.aws/config
- aws sts get-caller-identity --profile buildprofile
If you need to run a Docker container in a build environment and the container requires AWS credentials, you must pass through the credentials from the build environment to the container.
docker run -e AWS_DEFAULT_REGION -e AWS_CONTAINER_CREDENTIALS_RELATIVE_URI your-image-tag aws s3 ls
https://docs.aws.amazon.com/codebuild/latest/userguide/troubleshooting.html#troubleshooting-versions
Another way is to assume the role manually and export the auth tokens. Make sure you have ASSUME_ROLE_ARN available as environment variable -
commands:
- TEMP_ROLE=`aws sts assume-role --role-arn $ASSUME_ROLE_ARN --role-session-name temp`
- export TEMP_ROLE
- export AWS_ACCESS_KEY_ID=$(echo "${TEMP_ROLE}" | jq -r '.Credentials.AccessKeyId')
- export AWS_SECRET_ACCESS_KEY=$(echo "${TEMP_ROLE}" | jq -r '.Credentials.SecretAccessKey')
- export AWS_SESSION_TOKEN=$(echo "${TEMP_ROLE}" | jq -r '.Credentials.SessionToken')
- docker push $ECR_IMAGE_URL:$IMAGE_TAG

Spinnaker + ECR access

I'm having trouble setting up Spinnaker with ECR access.
Background: I installed spinnaker using helm on an EKS cluster and I've confirmed that the cluster has the necessary ECR permissions (by manually running ECR commands from within the clouddriver pod). I am following the instructions here to get Spinnaker+ECR set up: https://www.spinnaker.io/setup/install/providers/docker-registry/
Issue: When I run:
hal config provider docker-registry account add my-ecr-registry \
--address $ADDRESS \
--username AWS \
--password-command "aws --region us-west-2 ecr get-authorization-token --output text --query 'authorizationData[].authorizationToken' | base64 -d | sed 's/^AWS://'"
I get the following output:
+ Get current deployment
Success
- Add the some-ecr-registry account
Failure
Problems in default.provider.dockerRegistry.some-ecr-registry:
- WARNING Resolved Password was empty, missing dependencies for
running password command?
- WARNING You have a supplied a username but no password.
! ERROR Unable to fetch tags from the docker repository: code, 400
Bad Request
? Can the provided user access this repository?
- WARNING None of your supplied repositories contain any tags.
Spinnaker will not be able to deploy any docker images.
? Push some images to your registry.
Problems in halconfig:
- WARNING There is a newer version of Halyard available (1.28.0),
please update when possible
? Run 'sudo apt-get update && sudo apt-get install
spinnaker-halyard -y' to upgrade
- Failed to add account some-ecr-registry for provider
dockerRegistry.
I have confirmed that the aws-cli is installed on the clouddriver pod. And I've confirmed that I can the password-command directly from the clouddriver pod and it successfully returns a token.
I've also confirmed that if I manually generate an ECR token and run hal config provider docker-registry account add my-ecr-registry --address $ADDRESS --username AWS --password-command "echo $MANUALLY_GENERATED_TOKEN" everything works fine. So there is something specific to the password-command that is going wrong and I'm not sure how to debug this.
One other odd behavior: if I simplify the password command to be: hal config provider docker-registry account add some-ecr-registry --address $ADDRESS --username AWS --repositories code --password-command "aws --region us-west-2 ecr get-authorization-token" , I get an addt'l piece of output that says "- WARNING Password command returned non 0 return code stderr/stdout was:bash: aws: command not found". This output only appears for this simplified command.
Any advice on how to debug this would be much appreciated.
If like me your ECR registry is in another account, then you have to forcibly assume the role for the target account where your registry resides
passwordCommand: read -r AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN <<< `aws sts assume-role --role-arn arn:aws:iam::<AWS_ACCOUNT>:role/<SPINNAKER ROLE_NAME> --query "[Credentials.AccessKeyId, Credentials.SecretAccessKey, Credentials.SessionToken]" --output text --role-session-name spinnakerManaged-w2`; export AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN; aws ecr get-authorization-token --region us-west-2 --output text --query 'authorizationData[].authorizationToken' --registry-ids <AWS_ACCOUNT> | base64 -d | sed 's/^AWS://'
Credits to https://github.com/spinnaker/spinnaker/issues/5374#issuecomment-607468678
I also installed Spinnaker on AKS and all i did was by using an AWS Managing User with the correct AWS IAM policy to ECR:* i have access to the ECR repositories directly.
I dont think that hal being java based will execute the Bash command in --password-command
set the AWS ECS provider in your spinnaker deployment
Use the Following AWS IAM policy (SpinnakerManagingPolicy) to be attached to the AWS MAnaging User to give access to ECR. Please replace the AWS Accounts based on your need.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:*",
"cloudformation:*",
"ecr:*"
],
"Resource": [
"*"
]
},
{
"Action": "sts:AssumeRole",
"Resource": [
"arn:aws:iam::123456789012:role/SpinnakerManagedRoleAccount1",
"arn:aws:iam::101121314157:role/SpinnakerManagedRoleAccount2",
"arn:aws:iam::202122232425:role/SpinnakerManagedRoleAccount3"
],
"Effect": "Allow"
}
]
}

Error when creating aws emr default-roles

I'm trying to create a cluster using aws cli emr command. However, I can't seem to be able to create-default-roles needed before calling aws emr create-cluster
$ aws emr create-default-roles
A client error (NoSuchEntity) occurred when calling the GetRole operation: Unknown
I have made sure that my user has the following permissions:
IAMFullAccess - AWS Managed policy
AmazonElasticMapReduceforEC2Role - AWS Managed policy
AmazonElasticMapReduceFullAccess - AWS Managed policy
Any tips? Is there a place where I can just copy the roles json and create them manually?
The reason I started to do this is because when I run aws emr create-cluster it returns a cluster-id. But when that cluster-id is queries it state is set to terminated with the error: EMR service role arn:aws:iam::141703095098:role/EMR_DefaultRole is invalid
I DID manage to add these roles using the console by going to:
My Security Credentials > Roles > Create New Role
First Role with the following properties:
name: EMR_DefaultRole
policy: AmazonElasticMapReduceRole
Second Role with the following properties:
name: EMR_EC2_DefaultRole
policy: AmazonElasticMapReduceforEC2Role
Unfortunately I didn't get the command-line to work, but I suspect I might be something to do with my local setup.
I had issues with the console. With the client this worked:
# upgrade aws cli (can't hurt)
pip install --upgrade --user awscli
# aws configure process if you haven't (look it up)
# delete all the defunct shizzles
aws iam remove-role-from-instance-profile --instance-profile-name EMR_EC2_DefaultRole \
--role-name EMR_EC2_DefaultRole
aws iam delete-instance-profile \
--instance-profile-name EMR_EC2_DefaultRole
aws iam detach-role-policy \
--role-name EMR_EC2_DefaultRole \
--policy-arn arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceforEC2Role
aws iam delete-role --role-name EMR_EC2_DefaultRole
aws iam detach-role-policy --role-name EMR_DefaultRole \
--policy-arn arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceRole
aws iam delete-role --role-name EMR_DefaultRole
# now re-create them
aws emr create-default-roles
Note if you have attached policies, you might have to go into the console and delete them or find the appropriate aws cli command.
Source (our product is buggy and our role system is cumbersome, but if you buy premium support we'll tell you the workarounds):
https://aws.amazon.com/premiumsupport/knowledge-center/emr-default-role-invalid/

How to use MFA with AWS CLI?

How do I type in the MFA code when using the AWS CLI? I have checked the documentation page of IAM http://docs.aws.amazon.com/cli/latest/reference/iam/index.html.
I have the MFA-Devices already enabled under my username.
aws iam list-mfa-devices --user-name X
returns
{
"MFADevices": [
{
"UserName": "X",
"SerialNumber": "arn:aws:iam::+++:mfa/X",
"EnableDate": "2016-01-13T23:15:43Z"
}
]
}
The CLI can manage a lot of this for you if you're using roles. Described here: http://docs.aws.amazon.com/cli/latest/userguide/cli-roles.html
In my credentials file I have:
[my_iam_user]
aws_access_key_id = AKIABLAHBLAHBLAHBLAH
aws_secret_access_key = <blah>
region = us-east-1
[my_admin_role]
role_arn = arn:aws:iam::123456789123:role/my_admin_role
source_profile = my_iam_user
mfa_serial = arn:aws:iam::123456789123:mfa/my_iam_user
region = us-east-1
Note the mfa_serial entry. You can get this value from your user details in the AWS IAM console. This entry tells the CLI that MFA is required for that role.
When I call aws s3 ls --profile my_admin_role it says Enter MFA code:, after I paste in the code it returns the listing.
Note: I haven't found a way to get the CLI to ask for MFA when calling a user profile (--profile my_iam_user) only calling a role profile triggers the MFA request.
The MFA token is then carried forward and the user profile can be used as well:
aws sts get-caller-identity --profile my_iam_user
# {
# "Account": "123456789123",
# "UserId": "AIDABLAHBLAHBLAHBLAH",
# "Arn": "arn:aws:iam::123456789123:user/my_iam_user"
# }
aws sts get-caller-identity --profile my_admin_role
# {
# "Account": "123456789123",
# "UserId": "AROABLAHBLAHBLAHBLAH:AWS-CLI-session-1234567890",
# "Arn": "arn:aws:sts::123456789123:assumed-role/my_admin_role/AWS-CLI-session-1234567890"
# }
Call aws sts get-session-token --serial-number <serial> --token-code <code> documented here. This will give you a temporary security token. Documentation on using the temporary security token can be found here.
Step-by-step manual solution:
Request a session token with MFA
aws sts get-session-token --serial-number arn-of-the-mfa-device --token-code code-from-token
arn-of-the-mfa-device: visible from your user IAM
Option: Use CLI to retrieve: aws iam list-mfa-devices --user-name ryan
Option: View in IAM console: IAM --> Users --> --> Security Credentials
code-from-token: 6 digit code from your configured MFA device
Create a profile with the returned credentials
aws configure --profile cli
aws configure set --profile cli aws_session_token <SESSION_TOKEN_HERE>
aws_session_token is not included in aws configure
Test command
aws s3 ls --profile cli
I have published a PR for aws-cli, which will allow to use mfa_serial in the credentials, that will force you to enter the token before making request to AWS (and it will be cached while token is valid)
Issue: https://github.com/aws/aws-cli/issues/3172
botocore PR: https://github.com/boto/botocore/pull/1399
aws-cli PR: https://github.com/aws/aws-cli/pull/3174
Feel free to vote, if you want to get it in.
aws-mfa acts as a wrapper around sts and works really well: https://github.com/broamski/aws-mfa
Wrote a tool to add MFA support for standard IAM user profiles until #outcoldman PR gets merged: https://github.com/tongueroo/aws-mfa-secure
Setup for those in a hurry
Install gem
gem install aws-mfa-secure
Setup your ~/.aws/credentials with mfa_serial
~/.aws/credentials:
[mfa]
aws_access_key_id = BKCAXZ6ODJLQ1EXAMPLE
aws_secret_access_key = ABCDl4hXikfOHTvNqFAnb2Ea62bUuu/eUEXAMPLE
mfa_serial = arn:aws:iam::112233445566:mfa/MFAUser
Add the alias to your ~/.bash_profile
alias aws="aws-mfa-secure session"
Restart your terminal.
Example with Output
$ export AWS_PROFILE=mfa
$ aws s3 ls
Please provide your MFA code: 751888
2019-09-21 15:53:34 my-example-test-bucket
$ aws s3 ls
2019-09-21 15:53:34 my-example-test-bucket
$
Assume Role Profiles
Assume role profiles work already for the AWS CLI, here's an example:
~/.aws/credentials:
[mfa]
aws_access_key_id = BKCAXZ6ODJLQ1EXAMPLE
aws_secret_access_key = ABCDl4hXikfOHTvNqFAnb2Ea62bUuu/eUEXAMPLE
mfa_serial = arn:aws:iam::112233445566:mfa/MFAUser
[assumed-role]
role_arn = arn:aws:iam::112233445566:role/Admin
source_profile = mfa
role_session_name = MFAUser
mfa_serial = arn:aws:iam::112233445566:mfa/MFAUser
On Windows
I'm on windows and I created a batch file to pass in my MFA code and have it automatically set up my credentials. First, you need to set up your production credentials in AWS:
aws configure --profile prod
Answer the questions appropriately with your key and secret. Then, I run my script like this:
C:\> mfa-getCreds.bat 229168
Your credentials are set up, and will expire on 2019-05-12T04:04:13Z
Now you should be able to run aws commands like this: aws s3 ls
Here are the contents of my mfa-getCreds.bat:
#echo off
set TOKEN=%1
if not defined TOKEN goto showUsage
#call aws sts get-session-token --profile prod --serial-number "arn:aws:iam::109627855994:mfa/ryan.shillington" --token-code %* > c:\temp\mfa-getCreds.json
FOR /F "tokens=* USEBACKQ" %%g IN (`jq -r ".Credentials.AccessKeyId" c:\temp\mfa-getCreds.json`) do (SET AWS_ACCESS_KEY=%%g)
FOR /F "tokens=*" %%g IN ('jq -r ".Credentials.SecretAccessKey" c:\temp\mfa-getCreds.json') do (SET "AWS_SECRET_KEY=%%g")
FOR /F "tokens=*" %%g IN ('jq -r ".Credentials.SessionToken" c:\temp\mfa-getCreds.json') do (SET "AWS_SESSION_TOKEN=%%g")
FOR /F "tokens=*" %%g IN ('jq -r ".Credentials.Expiration" c:\temp\mfa-getCreds.json') do (SET "EXPIRATION=%%g")
set AWS_ACCESS_KEY_ID=%AWS_ACCESS_KEY%
set "AWS_SECRET_ACCESS_KEY=%AWS_SECRET_KEY%"
echo.
echo Your credentials are set up, but will expire on %EXPIRATION%
echo.
echo Now you should be able to run aws commands like this: aws s3 ls
goto :EOF
:showUsage
echo Usage: %0 [MFA Token]
goto :EOF
For this to run, you'll need the excellent jq package in your path.
My use-case is I have a root account where all IAM users are created and assigned to IAM groups which in turn have the capability to assume roles on a different account with varying degree of access depending on the group they are on. I have a few house rules in place;
No one is allowed to do anything on the root account except to manage their own IAM Users account.
Required password reset.
Required MFA.
You cannot switch accounts without logging in with MFA.
This has been set up using AWS Shared Organizations.
Previously, I've been using a python script I wrote to let my users to login via cli with MFA and switch accounts. This is done by manipulating the ~/.aws/credentials.
I've since migrated to using this project https://gitlab.com/severity1/aws-auth, which is written in Go and allows me to do the same without much setup and it works on windows, macosx and linux.
This effectively gives all my users the ability to do local testing while developing Apps for AWS without having to hardcode AWS Credentials into their code.
Run the sts get-session-token AWS CLI command, replacing the variables with information from your account, resources, and MFA device:
$ aws sts get-session-token --serial-number arn-of-the-mfa-device --token-code code-from-token
https://aws.amazon.com/premiumsupport/knowledge-center/authenticate-mfa-cli/
I wrote a small bash script to get over this annoying problem.
You can find it here: https://gist.github.com/geekgunda/db4c9c8d850c08a48d1d60f119628032
Assumptions:
Your original AWS Creds should be stored at ~/.aws/credentials
You've corrected ARN for MFA device (search for FIXME)
You've given correct MFA Code as cli argument
You have jq installed. Ref: https://stedolan.github.io/jq/
We documented a few considerations for AWS API multifactor in general (where to add the conditions, what are the implications etc.) in the documentation for some custom tooling (https://github.com/kreuzwerker/awsu) we developed for using Yubikeys as source for the TOTP tokens. This makes working with roles and long-term credentials + session tokens pretty easy.
I have forked Chinmay's gist and updated it to pull the device serial from aws instead of hardcoding it. I have also updated the exits to return a status of 1 instead of just exiting.
Available here:
https://gist.github.com/jpribyl/e44021ae5cbf7fd1b4549598e85b5341
I am using it in deploy scripts like this (I renamed the script to awsMfaCli.sh):
. awsMfaCli.sh
script_status=$?
if [[ $script_status -ne 1 ]]; then
echo "Building production"
if npm run build ; then
echo "Build Successful"
else
echo "Error building, exiting.."
return 1
fi
echo "Removing all files on bucket.."
aws s3 rm --recursive s3://mybucket
echo "Uploading site.."
aws s3 sync build/ s3://mybucket
echo "S3 Upload complete.."
echo "Deployment complete."
else
return 1
fi
AWS MFA use on the command line can be rather unpleasant and cumbersome, especially if you have multiple profiles and roles.
I have released awscli-mfa.sh script that makes MFA/role session management on the command line a lot easier. A companion script enable-disable-vmfa-device.sh similarly makes it easy to enable or disable a virtual MFA device on an IAM user account.
awscli-mfa.sh persists a started session in ~/.aws/credentials (with some info in ~/.aws/config), or allows you to start an in-env session only so that its details don't get persisted. When executed in Windows Subsystem for Linux, the script also provides session activation strings for PowerShell and Windows command line. However, the script itself only runs in bash (written for macOS, Linux, and WSL bash with Ubuntu).
You can find the scripts and the example MFA policies in my GitHub repo at https://github.com/vwal/awscli-mfa
A one-liner to authorize and append an MFA user to your credentials file using jq:
SERIAL_NUMBER=arn:aws:iam::000000000000:mfa/john TOKEN_CODE=123123 PROFILE_NAME=my_aws_2fa; \
aws sts get-session-token --serial-number $SERIAL_NUMBER --token-code $TOKEN_CODE \
| jq -r '.Credentials | ("aws_access_key_id = " + .AccessKeyId), ("aws_secret_access_key = " + .SecretAccessKey), ("aws_session_token = " + .SessionToken)' \
| (echo "\n[$PROFILE_NAME]" && cat) >> ~/.aws/credentials
Summary
Authenticating with MFA is done with the aws sts command. The basic process is:
Set up a "permanent" profile in ~/.aws/credentials.
Call the aws sts command with the "permanent" profile.
Create a "temporary" profile in ~/.aws/credentials with the result from #2
Use the "temporary" profile with your aws commands.
Details
#1:
[myaccountPerm]
aws_access_key_id = REDACTED
aws_secret_access_key = REDACTED
#2:
$ aws sts get-session-token --token-code 123456 --profile myaccountPerm --serial-number arn:aws:iam::REDACTED:mfa/REDACTED
{
"Credentials": {
"AccessKeyId": "REDACTED",
"SecretAccessKey": "REDACTED",
"SessionToken": "REDACTED",
"Expiration": "2021-03-13T09:37:10Z"
}
}
#3:
[myaccount]
aws_access_key_id = REDACTED
aws_secret_access_key = REDACTED
aws_session_token = REDACTED
Solution
You'll notice that this is kind of an excessive process for logging in. I recommend using a utility/script for this purpose. Several exist, including some mentioned in other answers.
I have also written such a utility, as well as an in-depth article on how to authenticate with MFA using the AWS CLI (includes a couple of other details, like how to require MFA).
My utility is just a short script, and doesn't use a pip installer, which makes it easy to inspect.
Overview of process
Utility for automation
I think my answer will be useful for people trying to follow the official docs, which sadly leave out a lot of details. I discovered a few nuances when trying to figure out how to do this according to the official docs so I wrote this up to help.
I originally had a non-MFA CLI user configured in my ~/.aws/credentials file
(this will be important later)
I then created a 2nd user "mfa-test" to which I attached a custom policy named Force_MFA. I sourced the policy from this aws doc
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowListActions",
"Effect": "Allow",
"Action": [
"iam:ListUsers",
"iam:ListVirtualMFADevices"
],
"Resource": "*"
},
{
"Sid": "AllowIndividualUserToManageTheirOwnMFA",
"Effect": "Allow",
"Action": [
"iam:CreateVirtualMFADevice",
"iam:DeleteVirtualMFADevice",
"iam:ListMFADevices",
"iam:EnableMFADevice",
"iam:ResyncMFADevice"
],
"Resource": [
"arn:aws:iam::*:mfa/${aws:username}",
"arn:aws:iam::*:user/${aws:username}"
]
},
{
"Sid": "AllowIndividualUserToDeactivateOnlyTheirOwnMFAOnlyWhenUsingMFA",
"Effect": "Allow",
"Action": [
"iam:DeactivateMFADevice"
],
"Resource": [
"arn:aws:iam::*:mfa/${aws:username}",
"arn:aws:iam::*:user/${aws:username}"
],
"Condition": {
"Bool": {
"aws:MultiFactorAuthPresent": "true"
}
}
},
{
"Sid": "BlockMostAccessUnlessSignedInWithMFA",
"Effect": "Deny",
"NotAction": [
"iam:CreateVirtualMFADevice",
"iam:EnableMFADevice",
"iam:ListMFADevices",
"iam:ListUsers",
"iam:ListVirtualMFADevices",
"iam:ResyncMFADevice"
],
"Resource": "*",
"Condition": {
"BoolIfExists": {
"aws:MultiFactorAuthPresent": "false"
}
}
}
]
}
Nuances I learned about this command:
aws sts get-session-token --serial-number arn:aws:iam::853680132675:mfa/mfa-test --token-code 630011
You should update to the latest version of the AWS CLI before running the command.
The docs mention it might fail if your aws cli isn't updated, but it's easy to miss that suggestion.
The TOTP(time-based one-time password) token code 630011 comes from your virtual MFA device
arn:aws:iam::853680132675:mfa/mfa-test can be looked up by viewing the Security credentials tab of the IAM user.
The command will fail (without a useful error message) if ~/.aws/credentials corresponds to another account (like non-MFA CLI user).
[default]
aws_access_key_id = lalala
aws_secret_access_key = lalala
In other words, there's a hard requirement that the credentials used when running the aws sts get-session-token --serial-number arn:aws:iam::853680132675:mfa/mfa-test --token-code 630011 command, map to the user referenced in that command (username = mfa-test). (The only exception is if you have additional profiles defined in ~/.aws/credentials & ~/.aws/config and reference them
using --profile additional_profile)
So I'd recommend running aws sts get-caller-identity and making sure you see the user (mfa-test) in the results (.../user/mfa-test) matches the user mentioned in the ARN of the mfa device (.../mfa/mfa-test)
"Arn": "arn:aws:iam::853680132675:user/mfa-test"
Also if you install jq (json query) cli tool then unix users can use the following trick to streamline the trade-off of non-MFA CLI creds to MFA CLI creds that expire after 12 hours by default (note the | tr -d '"' in the command is read as translate delete ", it strips out the double quotes)
export MFA_CLI_SESSION=$(aws sts get-session-token --serial-number arn:aws:iam::853680132675:mfa/mfa-test --token-code 630011)
export AWS_ACCESS_KEY_ID=$( echo $MFA_CLI_SESSION | jq .Credentials.AccessKeyId | tr -d '"' )
export AWS_SECRET_ACCESS_KEY=$( echo $MFA_CLI_SESSION | jq .Credentials.SecretAccessKey | tr -d '"' )
export AWS_SESSION_TOKEN=$( echo $MFA_CLI_SESSION | jq .Credentials.SessionToken | tr -d '"' )
You can run these commands as a smoke test to verify the variables aren't empty.
echo $MFA_CLI_SESSION
echo $AWS_ACCESS_KEY_ID
echo $AWS_SECRET_ACCESS_KEY
echo $AWS_SESSION_TOKEN
From this point on your AWS CLI commands will work for the next 12 hours within that terminal session (since they're implemented as env vars). Also they'll work without having to add --profile additional_profile flag to your commands.
Snippet easy to copy and paste.
cat <<EOF >aws-credentials.sh
export AWS_ACCESS_KEY_ID=<key>
export AWS_SECRET_ACCESS_KEY=<secret>
export AWS_DEFAULT_REGION=us-east-1
export AWS_USER=<aws_user>
export AWS_ACCOUNT=<aws_account>
EOF
# Almost one liner:
source aws-credentials.sh
source <(paste -d= <(echo -ne "export AWS_ACCESS_KEY_ID\nexport AWS_SECRET_ACCESS_KEY\nexport AWS_SESSION_TOKEN\nexport EXPIRATION\n") <(aws sts get-session-token --duration-seconds 129600 --serial-number arn:aws:iam::$AWS_ACCOUNT:mfa/$AWS_USER --token-code $(read -p "Token:" TOKEN; echo $TOKEN) | jq '.Credentials | .[]'))
# Example
aws s3 ls s3://my-bucket/