currently i am using tasks in ansible .The task include aws cli command like below:
command: aws ec2 describe-instances --region myregion --profile myawsprofile --filters
It works perfectly when ran through
ansible-playbook my.yml
Now my question is : I have multiple aws profiles and don't want hard code in the aws cli commands(in tasks).
wanted to use variable for profile,so whatever variable passed to profile it will execute task and run ansible-playbook.
Request anyone to suggest how to proceed for this.
I looked at Environment Variables in aws cli , but didn't helped much.
If you really want to use the command task and not the ec2_instance tasks then you could this:
command: aws ec2 describe-instances --region myregion --profile {{ myawsprofile }} --filters
and then to run your playbook
ansible-playbook my.yml -e "myawsprofile=production"
You have two options (change the [profile name] below with the actual value):
Either provide the --profile [profile name] as a command option for every command.
Alternatively, set the profile to be used as an environment variable. On Mac and Linux using either AWS_PROFILE or AWS_DEFAULT_PROFILE
$ export AWS_PROFILE=[profile name]
or on Windows
C:\> setx AWS_PROFILE [profile name]
The advantage of using the latter method is that once configured it keeps its value for the duration of the terminal session (or until it is being replaced with another profile name).
Reference: the Named Profiles chapter of the AWS CLI documentation
Related
I have a few different projects I work on. I use AWS and I use Kubernetes. I have a number of AWS credentials stored in my ~/.aws/credentials each with a label like
[account-1]
aws_access_key = x
aws_secret_access_key = y
[account-2]
aws_access_key = x
aws_secret_access_key = y
How can i toggle between them and easily set my config?
Currently I type aws configure in the terminal and manually paste the key/secret/regionn every time i want to switch between them.
When you hit the aws configure command, every time a new profile will be created in the ~/.aws/credentials. You can generate all the required profiles single time and then set environment variable based on the project you're working.
For example,
while working on project 1, set the environment variable
export AWS_PROFILE=account-1
and while working on project 2, set the environment variable
export AWS_PROFILE=account-2
If you are using zsh and oh-my-zsh with the aws plugin, you have the asp command.
asp account-1
and if your theme is set up nicely, your commandline prompt will tell you what account you're in.
In addition to #Gunjan answer you can also pass the profile name like this
$ aws ecr get-login-password --region us-east-1 --profile account-1
If you want to connect to multiple eks clusters
$ aws eks --region us-east-1 update-kubeconfig --name account-1-eks --region eu-west-1 --profile account-1
You need to have proper IAM permissions to run this command
This command will generate a kube config file in ~/.kube
move that file to some another location and add alias in your bash_profile or .zshrc like this line
account-1-eks='export KUBECONFIG=:/path/to/the/account-1-eks.config
Now reload your shell and you can switch using the alias like account-1-eks
You can repeat the steps for multiple accounts
I have created EC2 instance in the my office PC. It was successfully and I used it well. But when I logged to the AWS console from my home laptom - no one instance exist there and I have the error You do not have any instances in this region. I try to search instance information in other regions but do not found any.
How I can found my created instance or list all instances independently of region?
Quick way would be to use combination of AWS CLI, jq and a simple Bash for loop to iterate through each region and list the instances. Be sure to set your credentials before running
for region in `aws ec2 describe-regions | jq .Regions\[\].RegionName -r`
do
echo -e "\tRegion: ${region}"
aws ec2 describe-instances --query "Reservations[*].Instances[*].{InstanceID:InstanceId}" --output=table --region ${region}
done
You can copy post the code in your Linux shell, or run them in AWS CloudShell which gives you an authenticated shell with aws cli preinstalled
Interactively, I can use "aws configure" to change or see the default region. Is there a "pwd" like function, documented or not that allows me to determine or confirm the current region mid-script ? Even if AWS_DEFAULT_REGION is not defined ? I want a script to run under a number of profiles. I can scrape from aws configure list, but is there something neater ?
aws configure get region will get you the current region at that point in your script.
If you are using a profile, then type aws configure get region --profile $PROFILE_NAME.
Perhaps, AWS has not provide to get the current region. However, instead of getting the current region, They provide to get a current availability zone via an instance metadata. All availability zones include a current region, so you can determine the current region with you replace a part of the current availability zone in a script on the EC2 instance.
For example:
curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone | sed 's/\(.*\)[a-z]/\1/'
The following gives the region actually used by the CLI regardless of whether environment variables are or are not set:
aws ec2 describe-availability-zones --output text --query 'AvailabilityZones[0].[RegionName]'
The region that will be used by the CLI is determined by the following order of precedence:
Command line --region option
Value of AWS_DEFAULT_REGION environment variable
Region specified in the 'Current profile', which is determined by the following order of precedence
Command line --profile option
Value of AWS_PROFILE environment variable
Otherwise profile is [default]
Unfortunately using aws configure get region only returns the region as specified by your 'Current profile', and ignores the AWS_DEFAULT_REGION environment variable.
Rather than assuming any precedence rules, you can just use the CLI to make an actual API call and get the region from the result. The only call I could find that should always work was to aws ec2 describe-availability-zones. Another advantage of this method is that you can specify --region or --profile options and still get the right answer.
Taken from #RichVel's comment in this answer, to get the resolved region set from either AWS_DEFAULT_REGION or the region in the aws config file (aws configure get region only gives the value set in the config file) use:
aws configure list | grep region | awk '{print $2}'
Example:
with $AWS_DEFAULT_REGION unset:
$ echo $AWS_DEFAULT_REGION
$ cat ~/.aws/config
[foo]
region = us-east-1
$ aws configure list | grep region | awk '{print $2}'
us-east-1
$ aws configure get region
us-east-1
with $AWS_DEFAULT_REGION set:
$ export AWS_DEFAULT_REGION=us-west-2
$ echo $AWS_DEFAULT_REGION
us-west-2
$ cat ~/.aws/config
[foo]
region = us-east-1
$ aws configure list | grep region | awk '{print $2}'
us-west-2
$ aws configure get region
us-east-1
aws configure get region is neat but I wanted to be able to know the region even when AWS_DEFAULT_REGION was set. Unfortunately, according to the documentation:
Note that aws configure get only looks at values in the AWS configuration file. It does not resolve configuration variables specified anywhere else, including environment variables, command line arguments, etc.
Instead, assuming you have Python and boto3 installed, you can run:
python -c 'import boto3; print(boto3.Session().region_name)'
E.g.
$ AWS_DEFAULT_REGION=us-east-1 python -c 'import boto3; print(boto3.Session().region_name)'
us-east-1
The region is in the following:
curl http://169.254.169.254/latest/dynamic/instance-identity/document
So...
curl -s http://169.254.169.254/latest/dynamic/instance-identity/document|jq -r .region
This grows upon the answers by Jeshan Babooa and Alastair McCormack which have these known limitations:
aws configure get region doesn't return the region if the region is defined by the environment variable AWS_DEFAULT_REGION.
boto3 may not always be available, even though python -c 'import boto3; print(boto3.Session().region_name)' works when it is available.
To address the above limitations, consider in Bash:
AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION:-$(aws configure get region)}
This uses the value of the AWS_DEFAULT_REGION if available, otherwise it uses the output of aws configure get region.
If you have region info for your profiles in .aws/config, you can do as follows:
Example of .aws/config (note that dev profile is called profile dev). The profile word is imporant.
[default]
region = ap-southeast-2
[profile dev]
region = us-east-1
Then using cli, you can:
aws configure get profile.default.region
which gives ap-southeast-2 and
aws configure get profile.dev.region
which gives us-east-1.
Not sure if this works for you but I came to this page looking for similar advice. I am often switching between AWS Profiles which are configured to use different regions. In my case, the profile I am using is controlled by an environment variable - AWS_PROFILE
To get the current region, regardless of which profile I am using, I found this command useful:
aws configure get region --profile=$AWS_PROFILE
The region can be obtained using:
curl -s http://169.254.169.254/latest/meta-data/placement/region
Just execute
vi ~/.aws/config
to see your config file.
I am trying to use aws container service as per the documentation in http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_GetStarted.html
The below error is thrown when running the command:
aws ecs list-container-instances --cluster default
You must specify a region. You can also configure your region by running "aws configure".
The documentation does not mention anything about specifying a default region. How do we do it in a console?
I think you need to use for example:
aws ecs list-container-instances --cluster default --region us-east-1
This depends of your region of course.
"You must specify a region" is a not an ECS specific error, it can happen with any AWS API/CLI/SDK command.
For the CLI, either set the AWS_DEFAULT_REGION environment variable. e.g.
export AWS_DEFAULT_REGION=us-east-1
or add it into the command (you will need this every time you use a region-specific command)
AWS_DEFAULT_REGION=us-east-1 aws ecs list-container-instances --cluster default
or set it in the CLI configuration file: ~/.aws/config
[default]
region=us-east-1
or pass/override it with the CLI call:
aws ecs list-container-instances --cluster default --region us-east-1
#1- Run this to configure the region once and for all:
aws configure set region us-east-1 --profile admin
Change admin next to the profile if it's different.
Change us-east-1 if your region is different.
#2- Run your command again:
aws ecs list-container-instances --cluster default
If you have configured all what is needed in .aws/config and .aws/credentials but still have this error - double-check names in square brackets.
It should be [profile myLovelyAccName] in config and [myLovelyAccName] in credentials.
Two points to note:
the word "profile" and one space after - in the config file only
no typos in the acc name!
Just to add to answers by Mr. Dimitrov and Jason, if you are using a specific profile and you have put your region setting there,then for all the requests you need to add
"--profile" option.
For example:
Lets say you have AWS Playground profile, and the ~/.aws/config has [profile playground] which further has something like,
[profile playground]
region=us-east-1
then, use something like below
aws ecs list-container-instances --cluster default --profile playground
I posted too soon however the ways to configure are given in below link
http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
and way to get access keys are given in below link
http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-set-up.html#cli-signup
I've got two different apps that I am hosting (well the second one is about to go up) on Amazon EC2.
How can I work with both accounts at the command line (Mac OS X) but keep the EC2 keys & certificates separate? Do I need to change my environment variables before each ec2-* command?
Would using an alias and having it to the setting of the environment in-line work? Something like: alias ec2-describe-instances1 = export EC2_PRIVATE_KEY=/path; ec2-describe-instances
You can work with two accounts by creating two profiles on the aws command line.
It will prompt you for your AWS Access Key ID, AWS Secret Access Key and desired region, so have them ready.
Examples:
$ aws configure --profile account1
$ aws configure --profile account2
You can then switch between the accounts by passing the profile on the command.
$ aws dynamodb list-tables --profile account1
$ aws s3 ls --profile account2
Note:
If you name the profile to be default it will become default profile i.e. when no --profile param in the command.
More on default profile
If you spend more time using account1, you can make it the default by setting the AWS_DEFAULT_PROFILE environment variable. When the default environment variable is set, you do not need to specify the profile on each command.
Linux, OS X Example:
$ export AWS_DEFAULT_PROFILE=account1
$ aws dynamodb list-tables
Windows Example:
$ set AWS_DEFAULT_PROFILE=account1
$ aws s3 ls
How to set "manually" multiple AWS accounts ?
1) Get access - key
AWS Console > Identity and Access Management (IAM) > Your Security Credentials > Access Keys
2) Set access - file and content
~/.aws/credentials
[default]
aws_access_key_id={{aws_access_key_id}}
aws_secret_access_key={{aws_secret_access_key}}
[{{profile_name}}]
aws_access_key_id={{aws_access_key_id}}
aws_secret_access_key={{aws_secret_access_key}}
3) Set profile - file and content
~/.aws/config
[default]
region={{region}}
output={{output:"json||text"}}
[profile {{profile_name}}]
region={{region}}
output={{output:"json||text"}}
4) Run - file with params
Install command-line app - and use AWS Command Line it, for example for product AWS EC2
aws ec2 describe-instances -- default
aws ec2 describe-instances --profile {{profile_name}} -- [{{profile_name}}]
Ref
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html
IMHO, the easiest way is to edit .aws/credentials and .aws/config files manually.
It's easy and it works for Linux, Mac and Windows. Just read this for more detail (1 minute read).
.aws/credentials file:
[default]
aws_access_key_id=AKIAIOSFODNN7EXAMPLE
aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
[user1]
aws_access_key_id=AKIAI44QH8DHBEXAMPLE
aws_secret_access_key=je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY
.aws/config file:
[default]
region=us-west-2
output=json
[profile user1] <-- 'profile' in front of 'profile_name' (not for default)!!
region=us-east-1
output=text
You should be able to use the following command-options in lieu of the EC2_PRIVATE_KEY (and even EC2_CERT) environment variables:
-K <private key>
-C <certificate>
You can put these inside aliases, e.g.
alias ec2-describe-instances1 ec2-describe-instances -K /path/to/key.pem
Create or edit this file:
vim ~/.aws/credentials
List as many key pairs as you like:
[default]
aws_access_key_id=AKIAIOSFODNN7EXAMPLE
aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
[user1]
aws_access_key_id=AKIAI44QH8DHBEXAMPLE
aws_secret_access_key=je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY
Set a local variable to select the pair of keys you want to use:
export AWS_PROFILE=user1
Do what you like:
aws s3api list-buckets # any aws cli command now using user1 pair of keys
You can also do it command by command by including --profile user1 with each command:
aws s3api list-buckets --profile user1
# any aws cli command now using user1 pair of keys
More details: Named profiles for the AWS CLI
The new aws tools now support multiple profiles.
If you configure access with the tools, it automatically creates a default in ~/.aws/config.
You can then add additional profiles - more details at: Getting started with the AWS CLI
I created a simple tool, aaws, to switch between AWS accounts.
It works by setting the AWS_DEFAULT_PROFILE in your shell. Just make sure you have some entries in your ~/.aws/credentials file and it will easily switch between multiple accounts.
/tmp
$ aws s3 ls
Unable to locate credentials. You can configure credentials by running "aws configure".
/tmp
$ aaws luk3
[luk3] 🔐 /tmp
$ aws s3 ls
2013-11-05 21:40:04 luk3thomas.com
I wrote a toolkit to switch default AWS profile.
The mechanism is physically moving the profile key to the default section in config and credentials files.
The better solution today should be one of the following ways:
Use aws command option --profile.
Use environment variable AWS_PROFILE.
I don't remember why I didn't use the solution of --profile, maybe I was not realized its existence.
However the toolkit can still be useful by doing other things. I'll add a soft switch flag by using the way of AWS_PROFILE in the future.
$ xsh list aws/cfg
[functions] aws/cfg/move
[functions] aws/cfg/set
[functions] aws/cfg/activate
[functions] aws/cfg/get
[functions] aws/cfg/delete
[functions] aws/cfg/list
[functions] aws/cfg/copy
Repo: https://github.com/xsh-lib/aws
Install:
curl -s https://raw.githubusercontent.com/alexzhangs/xsh/master/boot | bash && . ~/.xshrc
xsh load xsh-lib/aws
Usage:
xsh aws/cfg/list
xsh aws/cfg/activate <profilename>
You can write shell script to set corresponding values of environment variables for each account based on user input. Doing so, you don't need to create any aliases and, furthermore, tools like ELB tools, Auto Scaling Command Line Tools will work under multiple accounts as well.
To use an IAM role, you have to make an API call to STS:AssumeRole, which will return a temporary access key ID, secret key, and security token that can then be used to sign future API calls. Formerly, to achieve secure cross-account, role-based access from the AWS Command Line Interface (CLI), an explicit call to STS:AssumeRole was required, and your long-term credentials were used. The resulting temporary credentials were captured and stored in your profile, and that profile was used for subsequent AWS API calls. This process had to be repeated when the temporary credentials expired (after 1 hour, by default).
More details: How to Use a Single IAM User to Easily Access All Your Accounts by Using the AWS CLI