I'm trying to spin up a s3 bucket via the AWS CLI and I'm getting the error "upload failed: .\mylocalfile.txt to s3://testBucket/mylocalfile.txt Unable to locate credentials".
I've ran aws configure and aws configure set on a default profile and i've done aws configure on a specific profile as well. In either case when I run AWS configure list I get the following
Name Value Type Location
---- ----- ---- --------
profile TestProfile manual --profile
access_key <not set> None None
secret_key <not set> None None
region us-east-1 config-file ~/.aws/config
the secret key is set in the same config file as the region but they are being located for some reason.
When I run AWS configure list --profile TestProfile I expect the credentials set in AWS configure to appear properly.
After typing aws configure list command in my project path, I get:
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key ****************skey env
secret_key ****************tkey env
region <not set> None None
My ~/.aws/credentials and ~/.aws/config looks like below:
~/.aws/credentials
[default]
aws_access_key_id=xxx
aws_secret_access_key=xxx
~/.aws/config
[default]
region=us-east-1
output=json
I tried to set aws configure in the path of this project, but the only change is region:
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key ****************skey env
secret_key ****************tkey env
region us-east-01 config-file ~/.aws/config
Conclusion: even though ~/.aws/credentials and ~/.aws/config are correct and I tried to set them in the project path, it still doesn't read the credentials correctly.
The only change was in the region because it was set to none; the rest is still read from env
How to unset these credentials and make them read from the above mentioned files (default user)?
I did it manually using unset:
unset AWS_ACCESS_KEY_ID
unset AWS_SECRET_ACCESS_KEY
After that, both the access key and the secret key read from the file:
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key ****************AYX shared-credentials-file
secret_key ****************vwT shared-credentials-file
region us-east-01 config-file ~/.aws/config
I've tried to set the default profile using the command below and now it reads it as default but type is env - not the shared-credentials-file or ~/.aws/config or ~/.aws/credential file.
export AWS_DEFAULT_PROFILE=default
It seems to work, but I don't think it's a valid solution...
The AWS CLI credentials and configuration settings take precedence in the following order:
command line options (--region, --output, and --profile)
environment variables
CLI credentials file
CLI config file
container credentials
EC2 instance profiles credentials
So, to force the AWSCLI to read the default configuration from the credentials/config files in ~/.aws/, don't supply --profile on the command line and don't have the AWS_PROFILE or AWS_ACCESS_KEY_ID environment variables set.
Read more at Configuration settings and precedence.
Also, I see no mention of AWS_DEFAULT_PROFILE at AWS CLI supported environment variables. I believe that it was deprecated some time ago in favor of AWS_PROFILE.
I can run the AWS CLI aws configure command and it lists my credentials. I even have a named profile I use.
However, my system does not have Users\{profile}\.aws\credentials file. Where could my credentials be stored?
The following command returns this but ~/.aws/config is empty for all the accounts I have on this machine. I have hidden files shown.
$aws configure list
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key ****************BBBB shared-credentials-file
secret_key ****************BBBB shared-credentials-file
region us-east-1 config-file ~/.aws/config
As others pointed out the location of your AWS credentials should be in %USERPROFILE%\.aws
Follow these steps and let's see what you get
Open terminal
Run: cd %UserProfile%\.aws
Run: type credentials
Another option is that you have added AWS_SHARED_CREDENTIALS_FILE environment variable. This changes the default location of the shared credentials file (https://docs.aws.amazon.com/cli/latest/topic/config-vars.html#the-shared-credentials-file). Try these steps:
Open terminal
Run: echo %AWS_SHARED_CREDENTIALS_FILE%
Run: type %AWS_SHARED_CREDENTIALS_FILE%
I have solved this creating a profile:
aws configure --profile {profile-name}
set any values
The /.aws/ folder have been created and now you can edit their files.
I have the default region set in ~/.aws/config file:
[default]
region=us-west-2
However, when I try describe-instances command for some specific profile it is failing with the following message:
$ aws ec2 describe-instances --profile my_profile
You must specify a region. You can also configure your region by running "aws configure".
Shouldn't it used the default profile configured in ~/.aws/config file? What am I missing here?
Output of aws configure list:
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key ******************** shared-credentials-file
secret_key ******************** shared-credentials-file
region us-west-2 config-file ~/.aws/config
Output of aws configure list --profile my_profile
Name Value Type Location
---- ----- ---- --------
profile my_profile manual --profile
access_key ******************** shared-credentials-file
secret_key ******************** shared-credentials-file
region <not set> None None
p.s. new to AWS. pardon me if this is a very basic question.
You are using named profiles, by default default profile is used. Just like you configure default, you can/need to configure your profile.
aws configure --profile my_profile
It will prompt you to fill access key, secret access key, AWS Region, and output format as it is stated here
You can create additional configurations that you can refer to with a name by specifying the --profile option and assigning a name. The following example creates a profile named produser. You can specify credentials from a completely different account and Region than the other profiles.
I'm trying to set up Amazon AWS EC2 instance to talk to s3. The basic command is
aws configure
then follow the prompt to enter
AWS Access Key ID [None]: my-20-digit-id
AWS Secret Access Key [None]: my-40-digit-secret-key
Default region name [None]: us-east-1
Default output format [None]: text
However, what I really want is to have the command
aws configure
automatically without interaction, i.e., no prompt and wait for input
I know there are files at
~.aws/credentials
~.aws/config
where I put those 4 key=value pairs. And the "credentials" file looks like
[default]
aws_secret_access_key = my-40-digit-secret-key
aws_access_key_id = my-20-digit-id
while the "config" file looks like
[default]
region = us-east-1
output = text
However, with those file at ~/.aws/, I get into ~/.aws/, and at the command line, I type and enter command
aws configure
I still got the prompt to ask me
AWS Access Key ID [None]:
AWS Secret Access Key [None]:
Default region name [None]:
Default output format [None]:
If I don't enter valid values at prompt, I won't be able to connect to s3, for example via command
aws s3 ls s3://mybucket
I turned help to amazon aws documentation pages. At this page, it mentions this option
"Command line options – region, output format and profile can be specified as command options to override default settings."
as the first option for aws configure
https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
However, it didn't mention how to use the command line options. I tried something like this
aws configure --region us-east-1
but I still got
AWS Access Key ID [None]:
AWS Secret Access Key [None]:
Default region name [None]:
Default output format [None]:
exactly like I have no option of "--region us-east-1"
If I try to
aws configure --aws_access_key_id my-20-digit-id --aws_secret_access_key my-40-digit-secret-key --region us-east-1
I get this
usage: aws [options] <command> <subcommand> [parameters]
aws: error: argument subcommand: Invalid choice, valid choices are:
How I can run the command
aws configure
automatically, no prompt, no interaction.
Please help! TIA
Edit and response to helloV, as the format in main post is much clearer than comment.
I tried the command helloV mentioned, but I got error
aws configure set aws_access_key_id my-20-digit-id
usage: aws [options] <command> <subcommand> [parameters]
aws: error: argument subcommand: Invalid choice, valid choices are:
Thanks though.
Continue on "aws configure set"
On another EC2 instance where I've already set connection to s3, I enter
aws configure set region us-east-1
runs and returns to command prompt ">"
aws configure set aws_access_key_id my-20-digit-id
runs and returns to command prompt ">"
aws configure set aws_secret_access_key my-40-digit-secret-key
runs and returns to command prompt ">"
aws configure
runs but comes with prompts and waits for interaction
AWS Access Key ID [****************ABCD]:
AWS Secret Access Key [****************1234]:
Default region name [us-east-1]:
Default output format [text]:
helloV:
here is my screen looks like
ubuntu#ip-11111:~/.aws$ more config
[default]
region = us-east-1
output = text
ubuntu#ip-11111:~/.aws$ more credentials
[default]
aws_secret_access_key = my-40-digit-secret-key
aws_access_key_id = my-20-digit-id
ubuntu#ip-11111:~/.aws$ aws s3 ls s3://
I got this
Unable to locate credentials. You can configure credentials by running "aws configure".
After this, I run
aws configure list
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key <not set> None None
secret_key <not set> None None
region us-east-1 config_file ~/.aws/config
Looks like it does not check ~/.aws/credentials file, but ~/.aws/config file is in the list.
These commands worked for me. If this doesn't works for you. Try do the first time using the interaction mode aws configure
aws --profile default configure set aws_access_key_id "my-20-digit-id"
aws --profile default configure set aws_secret_access_key "my-40-digit-secret-key"
I figured out, finally. Use export such as
export AWS_ACCESS_KEY_ID=my-20-digit-id
export AWS_SECRET_ACCESS_KEY=my-40-digit-secret-key
export AWS_DEFAULT_REGION=us-east-1
then run
aws s3 ls s3://
would work. Don't run "aws configure" as others mentioned.
Thank you all.
You describe the file very well. Why not just create a file and put it in the right place? I just tried... it's exactly the same as running aws configure
UPDATE: You mention that you want to access S3 from EC2 instance. In this case you shouldn't be using credentials at all. You should user Roles instead
The solution is that you actually don't have to run aws configure! After you run it for the 1st time and established the credentials (~/.aws/credentials) and config (~/.aws/config), going forward you simply have to run the required aws command. I tested this with a cron job and did a "aws s3 ls" command and it worked without having to provide a configure command before it.
Follow this command
$aws configure set aws_access_key_id default_access_key
$ aws configure set aws_secret_access_key default_secret_key
$ aws configure set default.region us-west-2
or
aws configure set aws_access_key_id <key_id> && aws configure set aws_secret_access_key <key> && aws configure set default.region us-east-1
For more details use this link
https://awscli.amazonaws.com/v2/documentation/api/latest/reference/configure/set.html
I use something like this:
aws configure --profile my-profile-name <<-EOF > /dev/null 2>&1
${AWS_ACCESS_KEY_ID}
${AWS_SECRET_ACCESS_KEY}
${AWS_REGION}
text
EOF
also to cleanup after automated process, and not remove `~/.aws/ directory (since some other credentials might be stored there) I run:
aws configure --profile my-profile-name <<-EOF > /dev/null 2>&1
null
null
null
text
EOF