I have EC2 instance.
I'm trying to call aws s3 from it but getting an error
Unable to locate credentials
I tried aws configure which does show everything as empty.
I see IAM role for S3 full permissions assigned to this instance.
Do I need any additional configuration?
If you run aws on an Amazon EC2 instance that has an assigned role, then it should find the credentials automatically.
You can also use this to view the credentials from the instance:
curl http://169.254.169.254/latest/meta-data/iam/security-credentials/
The role name should be listed. Then append it to the command:
curl http://169.254.169.254/latest/meta-data/iam/security-credentials/ROLE-NAME/
You should then be presented with AccessKeyId, SecretAccessKey, etc.
If that information does not appear, it suggests that your role is not correctly attached to the instance. Try unassigning the role and then assign it to the instance again.
First read and follow John Rotenstein's advice.
The SDK (from which the CLI is built from) searches for credentials in the following order:
Environment variables (AWS_ACCESS_KEY_ID ....)
Default Profile (credentials file)
ECS Container Credentials (if running on ECS)
Instance Profile
After verifying that your EC2 instance has credentials in the metadata, double check 1 & 2 to make sure that there are no other credentials present even if empty.
Note: This link argues with my last point (empty credentials file). I never create (store) credentials on an EC2 instance and I only use IAM Roles.
Instance Metadata
Related
We currently have 2 AWS accounts that we use. For most of the stuff we want to use the AWS account that our web app is hosted on in an EC2 instance so this works fine:
services.AddDefaultAWSOptions(this.Configuration.GetAWSOptions());
services.AddAWSService<IAmazonSQS>();
services.AddAWSService<IAmazonSimpleSystemsManagement>();
However, I want to access EC2 instances in another AWS account. I've configured it to work locally using credentials and from following this guide (where it mentions about using multiple services): https://docs.aws.amazon.com/sdk-for-net/latest/developer-guide/net-dg-config-netcore.html
services.AddDefaultAWSOptions(this.Configuration.GetAWSOptions());
services.AddAWSService<IAmazonSQS>();
services.AddAWSService<IAmazonSimpleSystemsManagement>();
if (this.WebHostEnvironment.IsDevelopment())
{
// This works fine locally, but I don't want to use credential file in production
var other = this.Configuration.GetAWSOptions("other");
services.AddAWSService<IAmazonEC2>(other);
}
else
{
// How do I register other here without putting a credential file on my ec2 instance?
services.AddAWSService<IAmazonEC2>();
}
I'm not sure how to register IAmazonEC2 to use my other account. I don't want to put a credential file on my instance which is how I get it working locally but it doesn't seem right to me on production servers.
I have configured an IAM role that has access to my other account and given it to my EC2 instance. But how do I translate that IAM role to a profile to use where I am registering IAmazonEC2 above?
Any help appreciated. Thanks
There are really two ways to do it...
Option 1: Use an IAM Role
Let's say that the Amazon EC2 instance is running in Account-A and it now wants to query information about Account-B. You could:
Create an IAM Role in Account-B, with a trust policy that trusts the IAM Role being used by the EC2 instance in Account-A
Your code running on the EC2 instance in Account-A can call AssumeRole() (using the normal credentials from Account-A). This will return a set of temporary credentials.
Use those temporary credentials to make API calls to Account-B
Option 2: Use credentials from Account-B
Alternatively, give your program a set of IAM User credentials from Account-B. These could be stored in AWS Systems Manager Parameter Store - AWS Systems Manager or AWS Secrets Manager, and retrieved by using the normal credentials assigned to the EC2 instance in Account-A.
On my client's AWS account, security credentials are generated everytime we login to their AWS sandbox account. This credentials file is automatically generated and downloaded via a Chrome plugin(SAML to AWS STS Key Conversion).
We then have to place the generated content to the ./aws/credentials file inside an EC2 instance in the same AWS account. This is little inconvenient as we have to update the generated credentials and session_token into the credentials file inside the EC2 instance every time we launch a Terraform script.
Is there any way we can attach any role so that we can just use the EC2 instance without entering the credentials into the credentials file.
Please suggest.
Work out what a reasonable, minimal set of permissions the Terraform script needs to create its AWS resources, then create an IAM role with those permissions, then add that IAM role to the instance (or launch a new instance with the role). Don't have a ~/.aws/credentials file on the instance or it will take precedence over the IAM role-based credentials.
I'm fairly new to AWS. I'm setting up an EC2 instance (an Ubuntu 18.04 LAMP server).
I've installed the aws CLI on the instance, so I can automate EBS snapshots for backup.
I've also created an IAM role with the needed permissions to run aws ec2 create-snapshot, and I've assigned this role to my EC2 instance.
My question: is there any need to run aws configure on the EC2 instance, in order to set the AWS Access Key ID and AWS Secret Access Key? I'm still wrapping my head around AWS IAM roles – but (since the EC2 instance has a role), it sounds like the instance will acquire the needed keys from IAM automagically. Therefore, I assume that there's never any need to run aws configure. (In fact, it seems like this would be counterproductive, since the keys set via aws configure would override the keys acquired automatically via the role.)
Is all of that accurate?
No, the AWS CLI will progress through a list of credential providers. The instance metadata service will eventually be reached, even if you have not configured the AWS cli:
https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#config-settings-and-precedence
And yes, if you add keys to the AWSCLI config file, they will be used with higher priority than those obtained from the instance metadata service.
I am mounting an AWS S3 bucket as a filesystem using s3fs-fuse. It requires a file which contains AWS Access Key Id and AWS Secret Access Key.
How do I avoid the access using this file? And instead use AWS IAM roles?
As per Fuse Over Amazon document, you can specify the credentials using 4 methods. If you don't want to use a file, then you can set AWSACCESSKEYID and AWSSECRETACCESSKEY environment variables.
Also, if your goal is to use AWS IAM instance profile, then you need to run your s3fs-fuse from an EC2 instance. In that case, you don't have to set these credential files/environment variables. This is because while creating the instance, if you attach the instance role and policy, the EC2 instance will get the credentials at boot time. Please see the section 'Using Instance Profiles' in page 190 of AWS IAM User Guide
there is an argument -o iam_role=--- which helps you to avoid AccessKey and SecretAccessKey
The Full steps to configure this is given below
https://www.nxtcloud.io/mount-s3-bucket-on-ec2-using-s3fs-and-iam-role/
In the logstash configuration file, I gave the following input plugin.
input{
s3{
bucket => 'bucket_name'
region => 'eu-west-1'
}
}
When I started logstash, it threw an error asking for AWS credentials. But I cannot provide AWS access_key_id and secret_key. I found that with IAM roles on EC2 instances, we shouldn't have to provide AWS credentials explicitly for an application that make those requests. I tried to understand how to configure IAM roles from a logstash Github issue, but failed. Please suggest how I should configure logstash file so that I can get data from S3 without providing AWS credentials explicitly.
When you are creating the EC2 instance, you may assign it an IAM role. If you don't give it a role when you create it, you cannot ever give it a role. You can modify the properties of a role (e.g. give the role more or less access) but you cannot change what role an instance has after the instance has been created.
The easiest way to test if an instance has the credentials you need is with the command-line tools:
$ echo $AWS_ACCESS_KEY_ID
$
That shows that we don't have the environment variable set, so if we're getting credentials, they must be instance creds.
$ aws --region=eu-west-1 s3 ls my.bucket
PRE some-directory/
PRE another-dir/
2015-11-05 18:03:53 464 little-file
2014-10-28 15:32:13 19740 bigger-file.html
$
I was able to list the bucket contents, so the EC2 instance where I'm running this command must have an IAM role that allows it to list this bucket contents!