I was working on a E-commerce project ( for study ) and wanted to sync my webfiles from S3 to EC2.
I used this command in the Linux SSH session:
#6. download the FleetCart zip from s3 to the html directory on the ec2 instance
sudo aws s3 sync s3://deg-s3bucketwebfiles /var/www/html
Entering the command, I get the following error message:
-- > fatal error: Unable to locate credentials
Not sure, what is wrong ? I checked that there's a directory /var/www/html but somehow the files cannot be sync across to EC2.
Appreciate any guide.
Thanks
Unable to locate credentials means that the aws command is unable to locate any credentials on the EC2 instance. The credentials are used to identify you to AWS so it knows that you are entitled to access the deg-s3bucketwebfiles bucket.
Option 1: Use an IAM Role
Since you are using an Amazon EC2 instance, the correct way to provide credentials to the instance is to associate an IAM Role to the instance. The role would need permission to access S3.
Option 2: Use credentials from an IAM User
Alternatively, you can use credentials associated with your IAM User. Go to the IAM Console, select your IAM User and go to the Security Credentials tab. You will find a Create access key button.
It will provide an Access Key and a Secret Key. The Access Key starts with AKIA, while the Secret Key is a long jumble of characters.
Once you have these credentials, run this command on the EC2 instance:
aws configure
Provide the credentials when prompted.
Related
Stupidly enough, I did delete by mistake my default AWS IAM user!
I used it for example do aws s3 sync...
Now the error I get is:
$ aws s3 sync build/ s3://mybucket.mydomain.com
fatal error: An error occurred (InvalidAccessKeyId) when calling the ListObjects operation: The AWS Access Key Id you provided does not exist in our records.
Is there a way to recover?
I think I need instructions how to create a new user with the sufficient roles to enable my local aws cli to be able to do aws s3 sync ...
UPDATE: I did just create a new user on my AWS console, and added a policy (to start with) to list my bucket. The problem is I don't know how to attach my aws cli to that new user... :-(
If you are the only person using this AWS Account, then add the AdministratorAccess Policy to your IAM User. That will grant complete access.
Then, in the Security credentials tab of the IAM User click Create access key. Copy the Access Key and Secret Access Key.
On the command line, run aws configure and provide those keys to configure the user.
Test with: aws s3 ls
We are trying to upload and display a file to and from S3 bucket through our .Net Script.
We are currently using the user's access key and secret key in our code, Which is a bad practice.
Could anyone let me if there is a way that we can use roles in the pace of these keys directly? If there is then how ?
As you're going to run this on EC2 the answer is yes you can attach an IAM role to an EC2 host.
This is indeed the best practice for running your scripts on your EC2 host. Once attached the EC2 your script will have access to all permissions that your EC2 has as long as you do not provide an IAM key/secret in the credentials of the SDK or have any of the environment variables set as these will override the IAM role.
More information is available in the IAM roles for Amazon EC2 documentation.
If you run your application in EC2, try to attach the role to EC2 directly.
If you are run on your local server, try to save your credentials on your server by using aws configure command
My requirement to upload file from local to s3 using aws cli but don't want to use access ID and secret access key while running in command line.
Any suggestions!
It is recommended that you never put AWS credentials in program code.
If the code is running on an Amazon EC2 instance, assign an IAM Role to the instance. The code will automatically detect and use these credentials.
If the code is running on your own computer, run the AWS Command-Line Interface (CLI) aws configure command and enter your IAM credentials (Access Key + Secret Key). They will be stored in the ~/.aws/credentials file and will be automatically accessed by your code.
I have EC2 instance.
I'm trying to call aws s3 from it but getting an error
Unable to locate credentials
I tried aws configure which does show everything as empty.
I see IAM role for S3 full permissions assigned to this instance.
Do I need any additional configuration?
If you run aws on an Amazon EC2 instance that has an assigned role, then it should find the credentials automatically.
You can also use this to view the credentials from the instance:
curl http://169.254.169.254/latest/meta-data/iam/security-credentials/
The role name should be listed. Then append it to the command:
curl http://169.254.169.254/latest/meta-data/iam/security-credentials/ROLE-NAME/
You should then be presented with AccessKeyId, SecretAccessKey, etc.
If that information does not appear, it suggests that your role is not correctly attached to the instance. Try unassigning the role and then assign it to the instance again.
First read and follow John Rotenstein's advice.
The SDK (from which the CLI is built from) searches for credentials in the following order:
Environment variables (AWS_ACCESS_KEY_ID ....)
Default Profile (credentials file)
ECS Container Credentials (if running on ECS)
Instance Profile
After verifying that your EC2 instance has credentials in the metadata, double check 1 & 2 to make sure that there are no other credentials present even if empty.
Note: This link argues with my last point (empty credentials file). I never create (store) credentials on an EC2 instance and I only use IAM Roles.
Instance Metadata
In the logstash configuration file, I gave the following input plugin.
input{
s3{
bucket => 'bucket_name'
region => 'eu-west-1'
}
}
When I started logstash, it threw an error asking for AWS credentials. But I cannot provide AWS access_key_id and secret_key. I found that with IAM roles on EC2 instances, we shouldn't have to provide AWS credentials explicitly for an application that make those requests. I tried to understand how to configure IAM roles from a logstash Github issue, but failed. Please suggest how I should configure logstash file so that I can get data from S3 without providing AWS credentials explicitly.
When you are creating the EC2 instance, you may assign it an IAM role. If you don't give it a role when you create it, you cannot ever give it a role. You can modify the properties of a role (e.g. give the role more or less access) but you cannot change what role an instance has after the instance has been created.
The easiest way to test if an instance has the credentials you need is with the command-line tools:
$ echo $AWS_ACCESS_KEY_ID
$
That shows that we don't have the environment variable set, so if we're getting credentials, they must be instance creds.
$ aws --region=eu-west-1 s3 ls my.bucket
PRE some-directory/
PRE another-dir/
2015-11-05 18:03:53 464 little-file
2014-10-28 15:32:13 19740 bigger-file.html
$
I was able to list the bucket contents, so the EC2 instance where I'm running this command must have an IAM role that allows it to list this bucket contents!