I found way to configure AWS credentials by
aws configure
command. But this is not very comfortable for me since I'm using multiple AWS accounts. Is there any way to make it easy to configure AWS credentials and switch between them?
Yes. You can configure multiple profiles.
The easiest way is to use:
aws configure --profile <name>
You can then use it with:
aws s3 ls --profile <name>
If --profile is not specified, it will use the default profile.
All configuration information is stored in the ~/.aws/credentials and ~/.aws/config files.
See: Named profiles - AWS Command Line Interface
ok so I am baffled by this aws cli behavior. Basically what is going on is that when I set my AWS creds related in environment variable, AWS CLI forces me to pass --profile flag each time I use the CLI.
So basically when AWS_ACCESS_KEY_ID AND AWS_SECRET_ACCESS_KEY then I cannot run commands like aws s3 ls without passing --profile flag to it even though my profile is [default]
Also, jus to note the environment variable values and the values inside my /.aws/credentials
file is exactly same. Also, I tried to set both AWS_PROFILE and AWS_DEFAULT_PROFILE to default hoping that if all values such as keys,secret and profile are set in environment variable then I do not have to pass any --profile flag explicitly. Not having to pass this flag explicitly is very important for me at this point because if I am running an application which connects with aws and picks up default credentials, there is no easy way to pass profile information to that app.
my credentials file look like following:
[default]
aws_access_key_id = AKIA****
aws_secret_access_key = VpR***
My config file looks like following:
[default]
region = us-west-1
output = json
And my environment variables do have the same values for corresponding entries. for key, secret and profile at least.
Any idea on how to solve this issue?
The AWS CLI looks for credentials using a series of providers in a particular order. (https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html#config-settings-and-precedence)
Specifically:
Command line options – You can specify --region, --output, and --profile as parameters on the command line.
Environment variables – You can store values in the environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN. If they are present, they are used.
CLI credentials file – This is one of the files that is updated when you run the command aws configure. The file is located at ~/.aws/credentials on Linux or macOS, or at C:\Users\USERNAME\.aws\credentials on Windows. This file can contain the credential details for the default profile and any named profiles.
CLI configuration file – This is another file that is updated when you run the command aws configure. The file is located at ~/.aws/config on Linux or macOS, or at C:\Users\USERNAME\.aws\config on Windows. This file contains the configuration settings for the default profile and any named profiles.
Container credentials – You can associate an IAM role with each of your Amazon Elastic Container Service (Amazon ECS) task definitions. Temporary credentials for that role are then available to that task's containers. For more information, see IAM Roles for Tasks in the Amazon Elastic Container Service Developer Guide.
Instance profile credentials – You can associate an IAM role with each of your Amazon Elastic Compute Cloud (Amazon EC2) instances. Temporary credentials for that role are then available to code running in the instance. The credentials are delivered through the Amazon EC2 metadata service. For more information, see IAM Roles for Amazon EC2 in the Amazon EC2 User Guide for Linux Instances and Using Instance Profiles in the IAM User Guide.
Another potential option for you would be to unset any colliding variables in your env and rely on the aws credentials file to provide the appropriate access credentials from the default entry.
I am trying to configure AWS CONFIGURE via AWS CLI on my laptop having Windows 10 professional. This is not the first time I am configuring AWS CONFIGURE. I already have many profiles settings.
I have enough free system memory and storage and have sufficient rights to run aws configure. I am using Python 3.6
Here is the detail, how I am trying to setup AWS CONFIGURE.
When I run this command again, it asks all the values again.
Even if I run an AWS CLI command using this new profile; to create a lambda function ( aws lambda create-function xxxxxxxxx --profile lambdaprofile ).
It gives below error.
The config profile (lambdaprofile) could not be found.
Please help me.
Some times this issue happens and AWS CLI is unable to set new profile and settings in config file.
Here is a fix of this issue.
Run below command from windows console.
notepad %USERPROFILE%\.aws\credentials
You will see last line of credentials file with overlapping line.
e.g in your particular case, it would be showing like below.
region = us-east-1[lambdaprofile]
aws_access_key_id = AKIAIGCOZJBAKIAIGCOZJB
aws_secret_access_key = gHZWwhUxRLtwQRUknGgHZWwhUxRLtwQRUknG
region = use-east-1
Similar issue would be in config file, which can be checked by opening this file.
notepad %USERPROFILE%\.aws\config
To fix this issue set [lambdaprofile] to new line, preferably add another empty line before [lambdaprofile]. It should look like below.
region = us-east-1
[lambdaprofile]
aws_access_key_id = AKIAIGCOZJBAKIAIGCOZJB
aws_secret_access_key = gHZWwhUxRLtwQRUknGgHZWwhUxRLtwQRUknG
region = use-east-1
Also do the same fix in config file. After fixing it, if you will run aws configure --profile lambdaprofile it should show the previously saved values in credentials and config files.
You may also check if values are saved or not with below command.
aws configure list --profile lambdaprofile
Alternately to the fix detail mentioned above, you can also set new profile directly with AWS CONFIGURE SET
e.g in your particular case.
aws configure --profile lambdaprofile set aws_access_key_id AKIAIGCOZJBAKIAIGCOZJB
aws configure --profile lambdaprofile set aws_secret_access_key gHZWwhUxRLtwQRUknGgHZWwhUxRLtwQRUknG
aws configure --profile lambdaprofile set region use-east-1
or
aws configure set profile.lambdaprofile.aws_access_key_id AKIAIGCOZJBAKIAIGCOZJB
aws configure set profile.lambdaprofile.aws_secret_access_key gHZWwhUxRLtwQRUknGgHZWwhUxRLtwQRUknG
aws configure set profile.lambdaprofile.region use-east-1
In my development environment, I regularly switch between several AWS access keys. So in my ~/.aws/credentials file, I have several profiles. I can then use these profiles with aws-cli by specifying the --profile <name> flag.
I develop several apps that use the Ruby AWS SDK. They are setup to read credentials from the ~/.aws/credentials file (the SDK does this by default, see this configuration article).
The problem is that the SDK always reads from the default profile. I would like to be able to configure it to read from some of the other profiles. Is this possible?
I've tried setting the default profile in my environment with:
export AWS_DEFAULT_PROFILE=<profile name>
as outlined in configuring the AWS Command Line Interface but it seems that only works with the aws-cli and not with the SDK.
For SDK:
By setting the AWS_PROFILE environment variable to the profile you
want to use.
For CLI:
From: A New and Standardized Way to Manage Credentials in the AWS SDKs
"AWS_DEFAULT_PROFILE is correct for the AWS CLI. The AWS CLI respects
AWS_PROFILE if it is set, but AWS_DEFAULT_PROFILE takes precedence."
I've got two different apps that I am hosting (well the second one is about to go up) on Amazon EC2.
How can I work with both accounts at the command line (Mac OS X) but keep the EC2 keys & certificates separate? Do I need to change my environment variables before each ec2-* command?
Would using an alias and having it to the setting of the environment in-line work? Something like: alias ec2-describe-instances1 = export EC2_PRIVATE_KEY=/path; ec2-describe-instances
You can work with two accounts by creating two profiles on the aws command line.
It will prompt you for your AWS Access Key ID, AWS Secret Access Key and desired region, so have them ready.
Examples:
$ aws configure --profile account1
$ aws configure --profile account2
You can then switch between the accounts by passing the profile on the command.
$ aws dynamodb list-tables --profile account1
$ aws s3 ls --profile account2
Note:
If you name the profile to be default it will become default profile i.e. when no --profile param in the command.
More on default profile
If you spend more time using account1, you can make it the default by setting the AWS_DEFAULT_PROFILE environment variable. When the default environment variable is set, you do not need to specify the profile on each command.
Linux, OS X Example:
$ export AWS_DEFAULT_PROFILE=account1
$ aws dynamodb list-tables
Windows Example:
$ set AWS_DEFAULT_PROFILE=account1
$ aws s3 ls
How to set "manually" multiple AWS accounts ?
1) Get access - key
AWS Console > Identity and Access Management (IAM) > Your Security Credentials > Access Keys
2) Set access - file and content
~/.aws/credentials
[default]
aws_access_key_id={{aws_access_key_id}}
aws_secret_access_key={{aws_secret_access_key}}
[{{profile_name}}]
aws_access_key_id={{aws_access_key_id}}
aws_secret_access_key={{aws_secret_access_key}}
3) Set profile - file and content
~/.aws/config
[default]
region={{region}}
output={{output:"json||text"}}
[profile {{profile_name}}]
region={{region}}
output={{output:"json||text"}}
4) Run - file with params
Install command-line app - and use AWS Command Line it, for example for product AWS EC2
aws ec2 describe-instances -- default
aws ec2 describe-instances --profile {{profile_name}} -- [{{profile_name}}]
Ref
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html
IMHO, the easiest way is to edit .aws/credentials and .aws/config files manually.
It's easy and it works for Linux, Mac and Windows. Just read this for more detail (1 minute read).
.aws/credentials file:
[default]
aws_access_key_id=AKIAIOSFODNN7EXAMPLE
aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
[user1]
aws_access_key_id=AKIAI44QH8DHBEXAMPLE
aws_secret_access_key=je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY
.aws/config file:
[default]
region=us-west-2
output=json
[profile user1] <-- 'profile' in front of 'profile_name' (not for default)!!
region=us-east-1
output=text
You should be able to use the following command-options in lieu of the EC2_PRIVATE_KEY (and even EC2_CERT) environment variables:
-K <private key>
-C <certificate>
You can put these inside aliases, e.g.
alias ec2-describe-instances1 ec2-describe-instances -K /path/to/key.pem
Create or edit this file:
vim ~/.aws/credentials
List as many key pairs as you like:
[default]
aws_access_key_id=AKIAIOSFODNN7EXAMPLE
aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
[user1]
aws_access_key_id=AKIAI44QH8DHBEXAMPLE
aws_secret_access_key=je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY
Set a local variable to select the pair of keys you want to use:
export AWS_PROFILE=user1
Do what you like:
aws s3api list-buckets # any aws cli command now using user1 pair of keys
You can also do it command by command by including --profile user1 with each command:
aws s3api list-buckets --profile user1
# any aws cli command now using user1 pair of keys
More details: Named profiles for the AWS CLI
The new aws tools now support multiple profiles.
If you configure access with the tools, it automatically creates a default in ~/.aws/config.
You can then add additional profiles - more details at: Getting started with the AWS CLI
I created a simple tool, aaws, to switch between AWS accounts.
It works by setting the AWS_DEFAULT_PROFILE in your shell. Just make sure you have some entries in your ~/.aws/credentials file and it will easily switch between multiple accounts.
/tmp
$ aws s3 ls
Unable to locate credentials. You can configure credentials by running "aws configure".
/tmp
$ aaws luk3
[luk3] 🔐 /tmp
$ aws s3 ls
2013-11-05 21:40:04 luk3thomas.com
I wrote a toolkit to switch default AWS profile.
The mechanism is physically moving the profile key to the default section in config and credentials files.
The better solution today should be one of the following ways:
Use aws command option --profile.
Use environment variable AWS_PROFILE.
I don't remember why I didn't use the solution of --profile, maybe I was not realized its existence.
However the toolkit can still be useful by doing other things. I'll add a soft switch flag by using the way of AWS_PROFILE in the future.
$ xsh list aws/cfg
[functions] aws/cfg/move
[functions] aws/cfg/set
[functions] aws/cfg/activate
[functions] aws/cfg/get
[functions] aws/cfg/delete
[functions] aws/cfg/list
[functions] aws/cfg/copy
Repo: https://github.com/xsh-lib/aws
Install:
curl -s https://raw.githubusercontent.com/alexzhangs/xsh/master/boot | bash && . ~/.xshrc
xsh load xsh-lib/aws
Usage:
xsh aws/cfg/list
xsh aws/cfg/activate <profilename>
You can write shell script to set corresponding values of environment variables for each account based on user input. Doing so, you don't need to create any aliases and, furthermore, tools like ELB tools, Auto Scaling Command Line Tools will work under multiple accounts as well.
To use an IAM role, you have to make an API call to STS:AssumeRole, which will return a temporary access key ID, secret key, and security token that can then be used to sign future API calls. Formerly, to achieve secure cross-account, role-based access from the AWS Command Line Interface (CLI), an explicit call to STS:AssumeRole was required, and your long-term credentials were used. The resulting temporary credentials were captured and stored in your profile, and that profile was used for subsequent AWS API calls. This process had to be repeated when the temporary credentials expired (after 1 hour, by default).
More details: How to Use a Single IAM User to Easily Access All Your Accounts by Using the AWS CLI