How to use MFA with AWS CLI? - amazon-web-services

How do I type in the MFA code when using the AWS CLI? I have checked the documentation page of IAM http://docs.aws.amazon.com/cli/latest/reference/iam/index.html.
I have the MFA-Devices already enabled under my username.
aws iam list-mfa-devices --user-name X
returns
{
"MFADevices": [
{
"UserName": "X",
"SerialNumber": "arn:aws:iam::+++:mfa/X",
"EnableDate": "2016-01-13T23:15:43Z"
}
]
}

The CLI can manage a lot of this for you if you're using roles. Described here: http://docs.aws.amazon.com/cli/latest/userguide/cli-roles.html
In my credentials file I have:
[my_iam_user]
aws_access_key_id = AKIABLAHBLAHBLAHBLAH
aws_secret_access_key = <blah>
region = us-east-1
[my_admin_role]
role_arn = arn:aws:iam::123456789123:role/my_admin_role
source_profile = my_iam_user
mfa_serial = arn:aws:iam::123456789123:mfa/my_iam_user
region = us-east-1
Note the mfa_serial entry. You can get this value from your user details in the AWS IAM console. This entry tells the CLI that MFA is required for that role.
When I call aws s3 ls --profile my_admin_role it says Enter MFA code:, after I paste in the code it returns the listing.
Note: I haven't found a way to get the CLI to ask for MFA when calling a user profile (--profile my_iam_user) only calling a role profile triggers the MFA request.
The MFA token is then carried forward and the user profile can be used as well:
aws sts get-caller-identity --profile my_iam_user
# {
# "Account": "123456789123",
# "UserId": "AIDABLAHBLAHBLAHBLAH",
# "Arn": "arn:aws:iam::123456789123:user/my_iam_user"
# }
aws sts get-caller-identity --profile my_admin_role
# {
# "Account": "123456789123",
# "UserId": "AROABLAHBLAHBLAHBLAH:AWS-CLI-session-1234567890",
# "Arn": "arn:aws:sts::123456789123:assumed-role/my_admin_role/AWS-CLI-session-1234567890"
# }

Call aws sts get-session-token --serial-number <serial> --token-code <code> documented here. This will give you a temporary security token. Documentation on using the temporary security token can be found here.

Step-by-step manual solution:
Request a session token with MFA
aws sts get-session-token --serial-number arn-of-the-mfa-device --token-code code-from-token
arn-of-the-mfa-device: visible from your user IAM
Option: Use CLI to retrieve: aws iam list-mfa-devices --user-name ryan
Option: View in IAM console: IAM --> Users --> --> Security Credentials
code-from-token: 6 digit code from your configured MFA device
Create a profile with the returned credentials
aws configure --profile cli
aws configure set --profile cli aws_session_token <SESSION_TOKEN_HERE>
aws_session_token is not included in aws configure
Test command
aws s3 ls --profile cli

I have published a PR for aws-cli, which will allow to use mfa_serial in the credentials, that will force you to enter the token before making request to AWS (and it will be cached while token is valid)
Issue: https://github.com/aws/aws-cli/issues/3172
botocore PR: https://github.com/boto/botocore/pull/1399
aws-cli PR: https://github.com/aws/aws-cli/pull/3174
Feel free to vote, if you want to get it in.

aws-mfa acts as a wrapper around sts and works really well: https://github.com/broamski/aws-mfa

Wrote a tool to add MFA support for standard IAM user profiles until #outcoldman PR gets merged: https://github.com/tongueroo/aws-mfa-secure
Setup for those in a hurry
Install gem
gem install aws-mfa-secure
Setup your ~/.aws/credentials with mfa_serial
~/.aws/credentials:
[mfa]
aws_access_key_id = BKCAXZ6ODJLQ1EXAMPLE
aws_secret_access_key = ABCDl4hXikfOHTvNqFAnb2Ea62bUuu/eUEXAMPLE
mfa_serial = arn:aws:iam::112233445566:mfa/MFAUser
Add the alias to your ~/.bash_profile
alias aws="aws-mfa-secure session"
Restart your terminal.
Example with Output
$ export AWS_PROFILE=mfa
$ aws s3 ls
Please provide your MFA code: 751888
2019-09-21 15:53:34 my-example-test-bucket
$ aws s3 ls
2019-09-21 15:53:34 my-example-test-bucket
$
Assume Role Profiles
Assume role profiles work already for the AWS CLI, here's an example:
~/.aws/credentials:
[mfa]
aws_access_key_id = BKCAXZ6ODJLQ1EXAMPLE
aws_secret_access_key = ABCDl4hXikfOHTvNqFAnb2Ea62bUuu/eUEXAMPLE
mfa_serial = arn:aws:iam::112233445566:mfa/MFAUser
[assumed-role]
role_arn = arn:aws:iam::112233445566:role/Admin
source_profile = mfa
role_session_name = MFAUser
mfa_serial = arn:aws:iam::112233445566:mfa/MFAUser

On Windows
I'm on windows and I created a batch file to pass in my MFA code and have it automatically set up my credentials. First, you need to set up your production credentials in AWS:
aws configure --profile prod
Answer the questions appropriately with your key and secret. Then, I run my script like this:
C:\> mfa-getCreds.bat 229168
Your credentials are set up, and will expire on 2019-05-12T04:04:13Z
Now you should be able to run aws commands like this: aws s3 ls
Here are the contents of my mfa-getCreds.bat:
#echo off
set TOKEN=%1
if not defined TOKEN goto showUsage
#call aws sts get-session-token --profile prod --serial-number "arn:aws:iam::109627855994:mfa/ryan.shillington" --token-code %* > c:\temp\mfa-getCreds.json
FOR /F "tokens=* USEBACKQ" %%g IN (`jq -r ".Credentials.AccessKeyId" c:\temp\mfa-getCreds.json`) do (SET AWS_ACCESS_KEY=%%g)
FOR /F "tokens=*" %%g IN ('jq -r ".Credentials.SecretAccessKey" c:\temp\mfa-getCreds.json') do (SET "AWS_SECRET_KEY=%%g")
FOR /F "tokens=*" %%g IN ('jq -r ".Credentials.SessionToken" c:\temp\mfa-getCreds.json') do (SET "AWS_SESSION_TOKEN=%%g")
FOR /F "tokens=*" %%g IN ('jq -r ".Credentials.Expiration" c:\temp\mfa-getCreds.json') do (SET "EXPIRATION=%%g")
set AWS_ACCESS_KEY_ID=%AWS_ACCESS_KEY%
set "AWS_SECRET_ACCESS_KEY=%AWS_SECRET_KEY%"
echo.
echo Your credentials are set up, but will expire on %EXPIRATION%
echo.
echo Now you should be able to run aws commands like this: aws s3 ls
goto :EOF
:showUsage
echo Usage: %0 [MFA Token]
goto :EOF
For this to run, you'll need the excellent jq package in your path.

My use-case is I have a root account where all IAM users are created and assigned to IAM groups which in turn have the capability to assume roles on a different account with varying degree of access depending on the group they are on. I have a few house rules in place;
No one is allowed to do anything on the root account except to manage their own IAM Users account.
Required password reset.
Required MFA.
You cannot switch accounts without logging in with MFA.
This has been set up using AWS Shared Organizations.
Previously, I've been using a python script I wrote to let my users to login via cli with MFA and switch accounts. This is done by manipulating the ~/.aws/credentials.
I've since migrated to using this project https://gitlab.com/severity1/aws-auth, which is written in Go and allows me to do the same without much setup and it works on windows, macosx and linux.
This effectively gives all my users the ability to do local testing while developing Apps for AWS without having to hardcode AWS Credentials into their code.

Run the sts get-session-token AWS CLI command, replacing the variables with information from your account, resources, and MFA device:
$ aws sts get-session-token --serial-number arn-of-the-mfa-device --token-code code-from-token
https://aws.amazon.com/premiumsupport/knowledge-center/authenticate-mfa-cli/

I wrote a small bash script to get over this annoying problem.
You can find it here: https://gist.github.com/geekgunda/db4c9c8d850c08a48d1d60f119628032
Assumptions:
Your original AWS Creds should be stored at ~/.aws/credentials
You've corrected ARN for MFA device (search for FIXME)
You've given correct MFA Code as cli argument
You have jq installed. Ref: https://stedolan.github.io/jq/

We documented a few considerations for AWS API multifactor in general (where to add the conditions, what are the implications etc.) in the documentation for some custom tooling (https://github.com/kreuzwerker/awsu) we developed for using Yubikeys as source for the TOTP tokens. This makes working with roles and long-term credentials + session tokens pretty easy.

I have forked Chinmay's gist and updated it to pull the device serial from aws instead of hardcoding it. I have also updated the exits to return a status of 1 instead of just exiting.
Available here:
https://gist.github.com/jpribyl/e44021ae5cbf7fd1b4549598e85b5341
I am using it in deploy scripts like this (I renamed the script to awsMfaCli.sh):
. awsMfaCli.sh
script_status=$?
if [[ $script_status -ne 1 ]]; then
echo "Building production"
if npm run build ; then
echo "Build Successful"
else
echo "Error building, exiting.."
return 1
fi
echo "Removing all files on bucket.."
aws s3 rm --recursive s3://mybucket
echo "Uploading site.."
aws s3 sync build/ s3://mybucket
echo "S3 Upload complete.."
echo "Deployment complete."
else
return 1
fi

AWS MFA use on the command line can be rather unpleasant and cumbersome, especially if you have multiple profiles and roles.
I have released awscli-mfa.sh script that makes MFA/role session management on the command line a lot easier. A companion script enable-disable-vmfa-device.sh similarly makes it easy to enable or disable a virtual MFA device on an IAM user account.
awscli-mfa.sh persists a started session in ~/.aws/credentials (with some info in ~/.aws/config), or allows you to start an in-env session only so that its details don't get persisted. When executed in Windows Subsystem for Linux, the script also provides session activation strings for PowerShell and Windows command line. However, the script itself only runs in bash (written for macOS, Linux, and WSL bash with Ubuntu).
You can find the scripts and the example MFA policies in my GitHub repo at https://github.com/vwal/awscli-mfa

A one-liner to authorize and append an MFA user to your credentials file using jq:
SERIAL_NUMBER=arn:aws:iam::000000000000:mfa/john TOKEN_CODE=123123 PROFILE_NAME=my_aws_2fa; \
aws sts get-session-token --serial-number $SERIAL_NUMBER --token-code $TOKEN_CODE \
| jq -r '.Credentials | ("aws_access_key_id = " + .AccessKeyId), ("aws_secret_access_key = " + .SecretAccessKey), ("aws_session_token = " + .SessionToken)' \
| (echo "\n[$PROFILE_NAME]" && cat) >> ~/.aws/credentials

Summary
Authenticating with MFA is done with the aws sts command. The basic process is:
Set up a "permanent" profile in ~/.aws/credentials.
Call the aws sts command with the "permanent" profile.
Create a "temporary" profile in ~/.aws/credentials with the result from #2
Use the "temporary" profile with your aws commands.
Details
#1:
[myaccountPerm]
aws_access_key_id = REDACTED
aws_secret_access_key = REDACTED
#2:
$ aws sts get-session-token --token-code 123456 --profile myaccountPerm --serial-number arn:aws:iam::REDACTED:mfa/REDACTED
{
"Credentials": {
"AccessKeyId": "REDACTED",
"SecretAccessKey": "REDACTED",
"SessionToken": "REDACTED",
"Expiration": "2021-03-13T09:37:10Z"
}
}
#3:
[myaccount]
aws_access_key_id = REDACTED
aws_secret_access_key = REDACTED
aws_session_token = REDACTED
Solution
You'll notice that this is kind of an excessive process for logging in. I recommend using a utility/script for this purpose. Several exist, including some mentioned in other answers.
I have also written such a utility, as well as an in-depth article on how to authenticate with MFA using the AWS CLI (includes a couple of other details, like how to require MFA).
My utility is just a short script, and doesn't use a pip installer, which makes it easy to inspect.
Overview of process
Utility for automation

I think my answer will be useful for people trying to follow the official docs, which sadly leave out a lot of details. I discovered a few nuances when trying to figure out how to do this according to the official docs so I wrote this up to help.
I originally had a non-MFA CLI user configured in my ~/.aws/credentials file
(this will be important later)
I then created a 2nd user "mfa-test" to which I attached a custom policy named Force_MFA. I sourced the policy from this aws doc
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowListActions",
"Effect": "Allow",
"Action": [
"iam:ListUsers",
"iam:ListVirtualMFADevices"
],
"Resource": "*"
},
{
"Sid": "AllowIndividualUserToManageTheirOwnMFA",
"Effect": "Allow",
"Action": [
"iam:CreateVirtualMFADevice",
"iam:DeleteVirtualMFADevice",
"iam:ListMFADevices",
"iam:EnableMFADevice",
"iam:ResyncMFADevice"
],
"Resource": [
"arn:aws:iam::*:mfa/${aws:username}",
"arn:aws:iam::*:user/${aws:username}"
]
},
{
"Sid": "AllowIndividualUserToDeactivateOnlyTheirOwnMFAOnlyWhenUsingMFA",
"Effect": "Allow",
"Action": [
"iam:DeactivateMFADevice"
],
"Resource": [
"arn:aws:iam::*:mfa/${aws:username}",
"arn:aws:iam::*:user/${aws:username}"
],
"Condition": {
"Bool": {
"aws:MultiFactorAuthPresent": "true"
}
}
},
{
"Sid": "BlockMostAccessUnlessSignedInWithMFA",
"Effect": "Deny",
"NotAction": [
"iam:CreateVirtualMFADevice",
"iam:EnableMFADevice",
"iam:ListMFADevices",
"iam:ListUsers",
"iam:ListVirtualMFADevices",
"iam:ResyncMFADevice"
],
"Resource": "*",
"Condition": {
"BoolIfExists": {
"aws:MultiFactorAuthPresent": "false"
}
}
}
]
}
Nuances I learned about this command:
aws sts get-session-token --serial-number arn:aws:iam::853680132675:mfa/mfa-test --token-code 630011
You should update to the latest version of the AWS CLI before running the command.
The docs mention it might fail if your aws cli isn't updated, but it's easy to miss that suggestion.
The TOTP(time-based one-time password) token code 630011 comes from your virtual MFA device
arn:aws:iam::853680132675:mfa/mfa-test can be looked up by viewing the Security credentials tab of the IAM user.
The command will fail (without a useful error message) if ~/.aws/credentials corresponds to another account (like non-MFA CLI user).
[default]
aws_access_key_id = lalala
aws_secret_access_key = lalala
In other words, there's a hard requirement that the credentials used when running the aws sts get-session-token --serial-number arn:aws:iam::853680132675:mfa/mfa-test --token-code 630011 command, map to the user referenced in that command (username = mfa-test). (The only exception is if you have additional profiles defined in ~/.aws/credentials & ~/.aws/config and reference them
using --profile additional_profile)
So I'd recommend running aws sts get-caller-identity and making sure you see the user (mfa-test) in the results (.../user/mfa-test) matches the user mentioned in the ARN of the mfa device (.../mfa/mfa-test)
"Arn": "arn:aws:iam::853680132675:user/mfa-test"
Also if you install jq (json query) cli tool then unix users can use the following trick to streamline the trade-off of non-MFA CLI creds to MFA CLI creds that expire after 12 hours by default (note the | tr -d '"' in the command is read as translate delete ", it strips out the double quotes)
export MFA_CLI_SESSION=$(aws sts get-session-token --serial-number arn:aws:iam::853680132675:mfa/mfa-test --token-code 630011)
export AWS_ACCESS_KEY_ID=$( echo $MFA_CLI_SESSION | jq .Credentials.AccessKeyId | tr -d '"' )
export AWS_SECRET_ACCESS_KEY=$( echo $MFA_CLI_SESSION | jq .Credentials.SecretAccessKey | tr -d '"' )
export AWS_SESSION_TOKEN=$( echo $MFA_CLI_SESSION | jq .Credentials.SessionToken | tr -d '"' )
You can run these commands as a smoke test to verify the variables aren't empty.
echo $MFA_CLI_SESSION
echo $AWS_ACCESS_KEY_ID
echo $AWS_SECRET_ACCESS_KEY
echo $AWS_SESSION_TOKEN
From this point on your AWS CLI commands will work for the next 12 hours within that terminal session (since they're implemented as env vars). Also they'll work without having to add --profile additional_profile flag to your commands.

Snippet easy to copy and paste.
cat <<EOF >aws-credentials.sh
export AWS_ACCESS_KEY_ID=<key>
export AWS_SECRET_ACCESS_KEY=<secret>
export AWS_DEFAULT_REGION=us-east-1
export AWS_USER=<aws_user>
export AWS_ACCOUNT=<aws_account>
EOF
# Almost one liner:
source aws-credentials.sh
source <(paste -d= <(echo -ne "export AWS_ACCESS_KEY_ID\nexport AWS_SECRET_ACCESS_KEY\nexport AWS_SESSION_TOKEN\nexport EXPIRATION\n") <(aws sts get-session-token --duration-seconds 129600 --serial-number arn:aws:iam::$AWS_ACCOUNT:mfa/$AWS_USER --token-code $(read -p "Token:" TOKEN; echo $TOKEN) | jq '.Credentials | .[]'))
# Example
aws s3 ls s3://my-bucket/

Related

AWS CDK deploy using role

You can find the cdk app that you can use to replicate my issue here varvay/issue-replication.git. The usage instruction explained in the README
I need to deploy CDK app using a role by issuing this command
cdk -r arn:aws:iam::000000000000:role/fooRole deploy
but then an error thrown
Assuming role failed: User: arn:aws:iam::000000000000:user/fooUser is not authorized to
perform: sts:AssumeRole on resource: arn:aws:iam::000000000000:role/barRole
to be sure, I tried to simulate it by assuming the arn:aws:iam::000000000000:role/barRole role using arn:aws:iam::000000000000:role/fooRole in AWS IAM Policy Simulator and it works just fine. One thing that bothers me is that the error said that a User tried to assume the role, not Role.
Why is that? or should I assume the fooRole, update the AWS-related environment variable and then deploy? if so then what's the point of having -r option on cdk
as additional information, here's the trust relationship of the barRole
{
"Version": "2008-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam:: 000000000000:root"
},
"Action": "sts:AssumeRole"
}
]
}
also I even tried to attach AdministratorAccess AWS managed policy to the fooRole used to deploy
I managed to fulfill my needs by creating a bash script to switch to the destination role and use the credential to perform the CDK command as the script written below,
#!/bin/bash
unset AWS_ACCESS_KEY_ID
unset AWS_SECRET_ACCESS_KEY
unset AWS_SESSION_TOKEN
AWS_CREDENTIAL=$(aws sts assume-role \
--role-arn <destination role ARN> \
--role-session-name <role session name> \
--duration-seconds 3600)
export AWS_ACCESS_KEY_ID=$(echo $AWS_CREDENTIAL \
| jq -r '.Credentials''.AccessKeyId')
export AWS_SECRET_ACCESS_KEY=$(echo $AWS_CREDENTIAL \
| jq -r '.Credentials''.SecretAccessKey')
export AWS_SESSION_TOKEN=$(echo $AWS_CREDENTIAL \
| jq -r '.Credentials''.SessionToken')
cdk deploy
unset AWS_ACCESS_KEY_ID
unset AWS_SECRET_ACCESS_KEY
unset AWS_SESSION_TOKEN
So there are 2 ways you might be running cdk deploy command from.
1- You're running this command from your local computer's CLI using IAM keys. In this case, this role must be assumable by the AWS account (IAM User) being used.
2- You're running this command from any AWS service (cicd agent on EC2 instance for e.g.:) then the role attached with the instance should be allowed to assume this deployment role.
mention how you're running this command and you might get a better answer.
UPDATE:
Based on the updated question:
Add assume role part in your IAM USER not your deployment role. Your IAM User from which you're trying to deploy should be allowed to assume the role through which the CDK will be deployed.
To diagramise it a bit:
(IAM-USER -> Assume -> Role) -> cdk deploy
The error is in the process of cross account role accessing, as is written in your error message.
I assume that you start with AWS configuration for one account, lets call it "Provisioning" and then you need to assume role in different account (dev or prod) depending on branches or something ?
I smell an error in setup of cross account roles.
https://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_cross-account-with-roles.html
One possibility is : the Rolle you want to assume, does not have your provisioning account as trusted entity.
Another is : the user which is trying to assume the role, does not have the policy for that.
Just follow the tutorial from AWS and see what is missing in your setup :)

What is the AWS CLI equivalent of boto3's Session.get_credentials() function?

I am working in an EC2 instance that can has connectivity to an S3 bucket and to a Redshift instance. From the EC2 instances's command line, I can do aws s3 ls s3://my-bucket/my-folder/ and it magically works. However, from within Redshift, when I want to import data with the copy command, I need to supply credentials.
The credentials that that the aws CLI are using do not exist within my ~/.aws/credentials file. I have found I can infer my (temporary, apparently) credentials using the boto3 Python library as follows:
session = boto3.Session()
creds = session.get_credentials()
creds = creds.get_frozen_credentials()
credentials = {
'aws_access_key_id': creds.access_key,
'aws_secret_access_key': creds.secret_key,
'token': creds.token,
}
credStr = ';'.join(k+'='+v for k,v in sorted(credentials.items()))
This gives me a credentials string that looks like this:
aws_access_key=XXX;aws_secret_access_key=YYY;token=ZZZ
I can then use that credentials string to write copy statements in Redshift as follows:
copy
my_table
from
's3://my-bucket/my.manifest'
credentials 'aws_access_key=XXX;aws_secret_access_key=YYY;token=ZZZ'
manifest
I have written a helper script that I can call from the command line to spit out my credentials, but it would be cleaner if I could get the above information from the aws cli tool, rather than having to call up a homegrown Python script.
SO: How do I do the equivalent of boto3.Session().get_credentials().get_frozen_credentials() using the aws CLI?
Thanks in advance for your help.
EDIT: Please note: Running aws sts get-caller-identity implies I am running as a role, and not a user. Therefore, calling aws sts get-session-token will not work, since roles cannot request temporary credentials.
try aws sts get-session-token
Your output should look something like this:
{
"Credentials": {
"AccessKeyId": "XXXXXXXXX",
"SecretAccessKey": "XXXXXXXX",
"SessionToken": "XXXXXXX",
"Expiration": "2020-03-18T08:15:08Z"
}
}
For anyone finding this via Google, this page led me to this solution:
role=$(curl http://169.254.169.254/latest/meta-data/iam/security-credentials)
# The below also works to get the role, but it is slower and has more dependencies.
#role=$(aws sts get-caller-identity | jq -r ".Arn" | sed -r 's#.*assumed-role/(.*)/.*#\1#')
api_token=$(curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
read -r key secret token <<<$(curl -H "X-aws-ecw-metadata-token: $api_token" -v http://169.254.169.254/latest/meta-data/iam/security-credentials/$role | jq -r '.AccessKeyId,.SecretAccessKey,.Token')
creds="aws_access_key_id=$key;aws_secret_access_key=$secret;token=$token"
I would still be interested to know whether there is a way to do this as a one-step process in the aws CLI, if anyone knows...
EDIT: Incorporated Michael's helpful suggestion below.
EDIT: Fixed typo in aws_access_key.

Pass AWS CodeBuild IAM Role inside Docker container [unable to locate credentials]

The role configured on CodeBuild project works fine with the runtime environment but doesn't work when we run a command from inside the container, it says "unable to locate credentials".
Let me know how can we use the role out of the box inside the container.
You can make use of credential source "EcsContainer" to assume role seamlessly without having to export new credentials in your buildspec.yml.
credential_source - The credential provider to use to get credentials for the initial assume-role call. This parameter cannot be provided alongside source_profile. Valid values are:
Environment to pull source credentials from environment variables.
Ec2InstanceMetadata to use the EC2 instance role as source credentials.
EcsContainer to use the ECS container credentials as the source credentials.
From: https://docs.aws.amazon.com/cli/latest/topic/config-vars.html
Steps:
Step-0: Create a new Role 'arn:aws:iam::0000000000:role/RoleToBeAssumed' and attach required policies to provide the permission required for the commands you are running during the build.
Step-1: Add sts:assumeRole permissions to your CodeBuild Service Role. Here is a sample policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "sts:*",
"Resource": "arn:aws:iam::0000000000:role/RoleToBeAssumed"
}
]
}
Step-2: Configure your build container to use the credential metadata as source for assuming the role. Here is a buildspec example:
version: 0.2
phases:
install:
runtime-versions:
nodejs: 8
commands:
- aws sts get-caller-identity
- mkdir ~/.aws/ && touch ~/.aws/config
- echo "[profile buildprofile]" > ~/.aws/config
- echo "role_arn = arn:aws:iam::0000000000:role/RoleToBeAssumed" >> ~/.aws/config
- echo "credential_source = EcsContainer" >> ~/.aws/config
- aws sts get-caller-identity --profile buildprofile
If you need to run a Docker container in a build environment and the container requires AWS credentials, you must pass through the credentials from the build environment to the container.
docker run -e AWS_DEFAULT_REGION -e AWS_CONTAINER_CREDENTIALS_RELATIVE_URI your-image-tag aws s3 ls
https://docs.aws.amazon.com/codebuild/latest/userguide/troubleshooting.html#troubleshooting-versions
Another way is to assume the role manually and export the auth tokens. Make sure you have ASSUME_ROLE_ARN available as environment variable -
commands:
- TEMP_ROLE=`aws sts assume-role --role-arn $ASSUME_ROLE_ARN --role-session-name temp`
- export TEMP_ROLE
- export AWS_ACCESS_KEY_ID=$(echo "${TEMP_ROLE}" | jq -r '.Credentials.AccessKeyId')
- export AWS_SECRET_ACCESS_KEY=$(echo "${TEMP_ROLE}" | jq -r '.Credentials.SecretAccessKey')
- export AWS_SESSION_TOKEN=$(echo "${TEMP_ROLE}" | jq -r '.Credentials.SessionToken')
- docker push $ECR_IMAGE_URL:$IMAGE_TAG

Spinnaker + ECR access

I'm having trouble setting up Spinnaker with ECR access.
Background: I installed spinnaker using helm on an EKS cluster and I've confirmed that the cluster has the necessary ECR permissions (by manually running ECR commands from within the clouddriver pod). I am following the instructions here to get Spinnaker+ECR set up: https://www.spinnaker.io/setup/install/providers/docker-registry/
Issue: When I run:
hal config provider docker-registry account add my-ecr-registry \
--address $ADDRESS \
--username AWS \
--password-command "aws --region us-west-2 ecr get-authorization-token --output text --query 'authorizationData[].authorizationToken' | base64 -d | sed 's/^AWS://'"
I get the following output:
+ Get current deployment
Success
- Add the some-ecr-registry account
Failure
Problems in default.provider.dockerRegistry.some-ecr-registry:
- WARNING Resolved Password was empty, missing dependencies for
running password command?
- WARNING You have a supplied a username but no password.
! ERROR Unable to fetch tags from the docker repository: code, 400
Bad Request
? Can the provided user access this repository?
- WARNING None of your supplied repositories contain any tags.
Spinnaker will not be able to deploy any docker images.
? Push some images to your registry.
Problems in halconfig:
- WARNING There is a newer version of Halyard available (1.28.0),
please update when possible
? Run 'sudo apt-get update && sudo apt-get install
spinnaker-halyard -y' to upgrade
- Failed to add account some-ecr-registry for provider
dockerRegistry.
I have confirmed that the aws-cli is installed on the clouddriver pod. And I've confirmed that I can the password-command directly from the clouddriver pod and it successfully returns a token.
I've also confirmed that if I manually generate an ECR token and run hal config provider docker-registry account add my-ecr-registry --address $ADDRESS --username AWS --password-command "echo $MANUALLY_GENERATED_TOKEN" everything works fine. So there is something specific to the password-command that is going wrong and I'm not sure how to debug this.
One other odd behavior: if I simplify the password command to be: hal config provider docker-registry account add some-ecr-registry --address $ADDRESS --username AWS --repositories code --password-command "aws --region us-west-2 ecr get-authorization-token" , I get an addt'l piece of output that says "- WARNING Password command returned non 0 return code stderr/stdout was:bash: aws: command not found". This output only appears for this simplified command.
Any advice on how to debug this would be much appreciated.
If like me your ECR registry is in another account, then you have to forcibly assume the role for the target account where your registry resides
passwordCommand: read -r AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN <<< `aws sts assume-role --role-arn arn:aws:iam::<AWS_ACCOUNT>:role/<SPINNAKER ROLE_NAME> --query "[Credentials.AccessKeyId, Credentials.SecretAccessKey, Credentials.SessionToken]" --output text --role-session-name spinnakerManaged-w2`; export AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN; aws ecr get-authorization-token --region us-west-2 --output text --query 'authorizationData[].authorizationToken' --registry-ids <AWS_ACCOUNT> | base64 -d | sed 's/^AWS://'
Credits to https://github.com/spinnaker/spinnaker/issues/5374#issuecomment-607468678
I also installed Spinnaker on AKS and all i did was by using an AWS Managing User with the correct AWS IAM policy to ECR:* i have access to the ECR repositories directly.
I dont think that hal being java based will execute the Bash command in --password-command
set the AWS ECS provider in your spinnaker deployment
Use the Following AWS IAM policy (SpinnakerManagingPolicy) to be attached to the AWS MAnaging User to give access to ECR. Please replace the AWS Accounts based on your need.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:*",
"cloudformation:*",
"ecr:*"
],
"Resource": [
"*"
]
},
{
"Action": "sts:AssumeRole",
"Resource": [
"arn:aws:iam::123456789012:role/SpinnakerManagedRoleAccount1",
"arn:aws:iam::101121314157:role/SpinnakerManagedRoleAccount2",
"arn:aws:iam::202122232425:role/SpinnakerManagedRoleAccount3"
],
"Effect": "Allow"
}
]
}

AWS Role being assumed not being used for future CLI Commands

I am trying to assume a role in a docker container but it doesn't seem like the role is sticking. Below is the output of aws sts assume-role followed by aws sts get-caller-identity
jenkins#0d4794bcdd62:~/.aws$ aws sts get-caller-identity
{
"UserId": "ABCDEFG1234567:i-01234546789abc",
"Arn": "arn:aws:sts::555555555555:assumed-role/jenkins-masterRole/i-01234546789abc",
"Account": "555555555555"
}
jenkins#0d4794bcdd62:~/.aws$ aws sts assume-role --role-arn arn:aws:iam::555555555555:role/ec2/CloudFormationRole --role-session-name "test-session-name"
{
"Credentials": {
"Expiration": "2019-01-28T16:37:04Z",
"SessionToken": "[redacted]",
"AccessKeyId": "[redacted]",
"SecretAccessKey": "[redacted]"
},
"AssumedRoleUser": {
"AssumedRoleId": "[redacted]:test-session-name",
"Arn": "arn:aws:sts::5555555555555:assumed-role/CloudFormationRole/test-session-name"
}
}
jenkins#0d4794bcdd62:~/.aws$ aws sts get-caller-identity
{
"Account": "555555555555",
"UserId": "ABCDEFG1234567:i-01234546789abc",
"Arn": "arn:aws:sts::555555555555:assumed-role/jenkins-masterRole/i-01234546789abc"
}
As you can see, the aws sts-assume role is working, but it doesn't seem like it is being applied. Below are the contents of my ~/.aws/config file
jenkins#0d4794bcdd62:~/.aws$ cat ~/.aws/config
[default]
region = us-east-1
I don't have any AWS CLI environment variables set except for AWS_CONFIG_FILE=~/.aws/config. Which I originally wasn't set, but I did that while testing.
Calling assume-role is not going to change the configuration of the AWS CLI. AWS CLI is using the credentials stored in ~/.aws/credentials to make the calls. The ones returned by assume-role are not copied automatically to AWS CLI configuration file, they are just displayed on your screen.
So, one naive way to solve your problem, would be to copy / paste the access key, secret key and session id to ~/.aws/credentials Of course, this would not scale, nor be convenient. There is a better way.
You can configure a profile in the CLI to use the role you want. Assuming that your base identity has permission to assume the role (which you do as per your call to assume-role in your question)
This is how I configured my ~/.aws/config file to get temporary access to my personal account, from my work account. Of course the same technique also works within the same account.
[default]
region=eu-west-1
[profile perso]
region=eu-west-1
role_arn=arn:aws:iam::[redacted]:role/admin
source_profile=default
The role ARN is the role you want to assume in the target account.
then you can simply type :
aws --profile perso s3 ls
to switch role.
You can also (ab)use the --query option to generate environment variable assignments for the temporary role credentials, like this:
$ roleSessionName=my-example-session
$ arnOfRoleToAssume=arn:aws:iam::123456789012:role/demo
$ assumeRoleEnv=$(aws sts assume-role \
--role-session-name="$roleSessionName" \
--role-arn="$arnOfRoleToAssume"\
--output text \
--query='Credentials.[
join(`=`, [`AWS_ACCESS_KEY_ID`, AccessKeyId]),
join(`=`, [`AWS_SECRET_ACCESS_KEY`, SecretAccessKey]),
join(`=`, [`AWS_SESSION_TOKEN`, SessionToken])
]')
$ eval "export $assumeRoleEnv"
$ aws sts get-caller-identity # this and further aws calls use the assumed role
One advantage of this is not having to mess with shared config/credentials. For example, that's why I chose this approach in awscli-with-assume-role (Docker image).