I'm trying to import AWS credentials from csv file with headers
User name,Password,Access key ID,Secret access key,Console login link
and using command
aws configure import --csv file://myfile.csv --region us-east-1
But the region comes back as empty. I tried setting region header in CSV file too but nothing works.
Below is the output of aws configure list
Name Value Type Location
---- ----- ---- --------
profile deployment_admin env ['AWS_PROFILE', 'AWS_DEFAULT_PROFILE']
access_key ****************. shared-credentials-file
secret_key ****************. shared-credentials-file
region <not set> None None
I don't think that's supported. The --region parameter is listed under the Global Options, i.e. can be used with all CLI operations.
Global Options
[...]
--region (string)
The region to use. Overrides config/env settings.
— docs
Usually, you can use it to direct an API call to a specific region. Since there is no API call here, it shouldn't do anything.
Looking at the implementation, it seems like the code really only considers the 'User Name', 'Access Key ID', and 'Secret Access key' columns in the CSV so there's no sneaky way to add the region.
Related
I usually use my AWS CLI commands after setting a profile, with the environment variable AWS_PROFILE, with the ~/.aws/credentials file. This works.
What I'm currently trying to do is to set up access via environment variables. To do so, I'm setting those variables in my .bash_profile file - I literally copied the aws_access_key_id and aws_secret_access_key entries from the credentials files and put them in my bash_profile file, under the names of AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.
The environment variables are being correctly set, and, yet, when I try to access AWS resources (in this case, I'm trying to run a ls S3 command over a bucket, so the region doesn't matter), I get the message
An error occurred (InvalidAccessKeyId) when calling the ListObjectsV2 operation: The AWS Access Key Id you provided does not exist in our records
which is very weird to me, since the keys are exactly the same. To confirm this, I switch to my credential profile with the AWS_PROFILE environment variable, and then the command works normally.
I suspected that, somehow, I was setting the wrong environment variables, or something like that. Then, I read this AWS guide, and ran the command aws configure list, which, in the first case (the case with environment variables only), returned
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key ****************AAAA env
secret_key ****************AAAA env
region us-east-1 env ['AWS_REGION', 'AWS_DEFAULT_REGION']
For the second case (with the profile set), it returned
Name Value Type Location
---- ----- ---- --------
profile dev-staging manual --profile
access_key ****************AAAA shared-credentials-file
secret_key ****************AAAA shared-credentials-file
region us-east-1 env ['AWS_REGION', 'AWS_DEFAULT_REGION']
In other words, the environment variables are being correctly set, the AWS CLI acknowledges them, their values are the same as when they are set via the credentials file, and, yet, for some reason, it doesn't work that way.
I thought it could be due to the aws_session_token, which I also tried to set as an environment variable, to no avail.
I need to access AWS resources this way to simulate the environment in which my code will run, and I don't see why this would not work the way I'm intending.
Any ideas on how to solve it are appreciated.
You need to edit your ~/.aws/config file when you would like to refer to the credentials from environment variables instead of credentials file.
With AWS Access Keys in credentials file, you must be having profile setup as OR there is no such source_profile config for any profile:
[default]
source_profile = default
However, when you would like to use the credentials set in your environment variables or bash_profile, change/add this setting to every profile in your config file:
[default]
credential_source = Environment
With this change, it should work with your Environment variables as well.
In case you've multiple profiles in ~/.aws/config file, just replace/add source_profile = <profile-name> with credential_source = Environment
In case someone stumbles on this, a possible culprit for this might be the AWS_SESSION_TOKEN and AWS_SECURITY_TOKEN environment variables.
If you were using different AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY environment variables before and an AWS CLI command is run, directly or indirectly, then after first auth the above two token variables are set. And after we overwrite the existing AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY with new values, the older token variables still remain as they are and AWS CLI doesn't have an explicit check to see if the access/secret keys were updated and it continues to use older tokens resulting in older keys being used internally and will continue to do so till the tokens expire.
The aws configure will continue to show new access keys, but internally it would be using older access keys because of cached tokens.
So if you want to continue to use the environment variables in such scenarios you will need to unset the two environment variables containing tokens and in your case also add an unset command for two token variables after setting the new access/secret keys in environment variables.
unset AWS_SESSION_TOKEN
unset AWS_SECURITY_TOKEN
This behavior is one of the reasons people prefer to use different profiles either using aws configure or editing the ~/.aws/* files, and explicitly specify them using the --profile in commands instead of using environment variables.
Per the AWS cli configuration precedence order the usage of ~/.aws/config file is at the top of the precedence order of where AWS CLI picks up the auth to be used, so it overrides the token environment variables and works in your case.
I have created an instance with instance profile associated.
I am trying to run packer on the instance without using access and secret key, hence I am using the profile option.
However, the packer requires a profile name, but the instance profile doesn't create a profile name and also doesn't create credentials under ~/.aws.
I can see the access and secret key associated via the AWS CLI, however it doesn't have a profile name.
[ec2-user#ip-172-31-19-132 ~]$ aws configure list
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key ****************3IO2 iam-role
secret_key ****************cFvD iam-role
region ap-southeast-2 imds
As I understand from this answer, access and secret keys are rotated frequently, I am wondering how can I set the profile name to the credentials permanently, so I can refer that in packer.
There won't be a profile name as you're using an IAM role. Instead you should natively use the IAM role instead.
Finally, Packer will use credentials provided by the task's or instance's IAM role, if it has one.
This is a preferred approach over any other when running in EC2 as you can avoid hard coding credentials. Instead these are leased on-the-fly by Packer, which reduces the chance of leakage.
Remove the references to the profile option, and it should just natively work just like the SDK / CLI functionality would.
I am trying to download all the available files from my s3 bucket to my local machine. I have installed AWS cli. and then I have used aws configure to setup access key and secret key too. I am facing issue while trying to execute the following command:
$ aws s3 sync s3://tempobjects .
Setup commands
LAMU02XRK97:s3 vsing$ export AWS_ACCESS_KEY_ID=*******kHXE
LAMU02XRK97:s3 vsing$ export AWS_SECRET_ACCESS_KEY=******Ssv
LAMU02XRK97:s3 vsing$ aws configure list
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key ****************kHXE shared-credentials-file
secret_key ****************pSsv shared-credentials-file
region us-east-1 config-file ~/.aws/config
Error:
LAMU02XRK97:s3 vsing$ aws s3 sync s3://tempobjects .
fatal error: An error occurred (InvalidAccessKeyId) when calling the ListObjectsV2 operation: The AWS Access Key Id you provided does not exist in our records.
I have replicated the scenario and to make it work you need to make sure that the user you are using for CLI is having the same access keys configured in the IAM.
Below is what configured in AWS CLI.
Below is what configured in AWS IAM for the same user :
Access Key ending with QYHP is configured at both the places and hence it is working fine for me.
I have an ec2 instance with a specific role,
when typing the awscli command:
[TEST#JenkinsSlave ~]$ aws configure list
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key ****************E25I iam-role
secret_key ****************Yl4d iam-role
region <not set> None None
I can see that I have an access key and secret key which I didn't create,
This is a clean amazon Linux ami with no configuration.
I thought it was the role keys but there is no such thing as assigning keys to a role, and every time I create a new server I get different keys so it looks like random keys.
does anyone know how I got those credentials?
and How I can delete them from my configuration (as you can see by Location=None they are not stored in ~/.aws/)
Even if you don't have an instance role assigned, your EC2 instance does have instance-based credentials. You can see these with curl http://169.254.169.254/latest/meta-data/identity-credentials/ec2/security-credentials/ec2-instance.
However, when I run aws configure list, those credentials don't show up. It's possible that you're running an earlier or later version than I am (aws-cli/1.16.153) and the behavior has changed. You can compare the credentials from that curl request to see if they're the same.
You should verify that you don't, in fact, have an instance role set. Either check the console or use curl http://169.254.169.254/latest/meta-data/iam/ from the instance. If you don't have a role set you'll get a 404. If you do have a role set this command will show a file named info, and that file will contain the instance role information.
Edit: the only thing that I've tried that does produce the output you're seeing is to assign an instance profile to the instance.
In response to comment asking about different tokens for each machine: this is how instance profiles work. Each machine gets a time-limited set of credentials that can be used to make requests, and automatically renews those credentials as needed. This is a Good Thing, as it means that those credentials can not be taken from the machine and used to gain unauthorized access to the services.
Given I have the following config file:
[default]
aws_access_key_id=default_access_key
aws_secret_access_key=default_secret_key
[profile testing]
aws_access_key_id=testing_access_key
aws_secret_access_key=testing_secret_key
region=us-west-2
And given the name of my default profile is foo
What CLI commands do I need to type in to get the name of my default profile. Something like:
$ aws describe-default-profile
{
...
"default_profile_name": 'foo'
}
Or list all profiles and it ouputs the default too:
$ aws list-all-profiles
{
[{
...
profile_name: 'foo',
"is_default": true
}]
}
There is a get-instance-profile on iam (docs), but it requires the name of the profile be specified:
$ aws iam get-instance-profile --instance-profile-name ExampleInstanceProfile
You can run aws configure list to list your current profile
List the AWS CLI configuration data. This command will show you the
current configuration data. For each configuration item, it will show
you the value, where the configuration value was retrieved, and the
configuration variable name. For example, if you provide the AWS
region in an environment variable, this command will show you the name
of the region you've configured, it will tell you that this value came
from an environment variable, and it will tell you the name of the
environment variable.
To show your current configuration values:
$ aws configure list
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key ****************ABCD config_file ~/.aws/config
secret_key ****************ABCD config_file ~/.aws/config
region us-west-2 env AWS_DEFAULT_REGION
If you want to review your configuration for a specific profile, you can run aws configure list --profile foo
Since version 2 you can use:
$ aws configure list-profiles
default
test
To show all the available profiles.
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html
There are no commands in the AWS Command-Line Interface (CLI) for viewing the profile. You would need to look at the configuration files for this information.
The aws iam get-instance-profile command is unrelated to the AWS CLI. It is a way of assigning a Role to an Amazon EC2 instance.