I have an ECS task that runs an image build from Amazon Linux.
container_pull(
name = "amazonlinux",
registry = "registry.hub.docker.com",
repository = "library/amazonlinux",
tag = "2022.0.20220315.0",
digest = "sha256:c74e77c670519cd69e3f5ce3fa714c02c582a40d786dd7e97113e717e7655e4d",
)
However when I run the image on ECS and try to perform an operation on S3, I get this error:
Unable to get IAM security credentials from EC2 Instance Metadata Service.
This surprises me because I thought the image would contain the necessary services to communicate with ECS and obtain IAM credentials.
The role has permissions s3:PutObject and s3:GetObject.
How do I gain access to S3 inside my image?
Note I do not want to pass AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables if I can help it.
The issue was that I had not assigned a Task Role (I had only assigned an Execution Role).
The Task Role has permission to access S3.
Related
I have a ec2 instance with a role attached to it. The role is called webserver and has all the relevant policies attached to it.
I am trying to invoke my lambda function from my PHP code, but I get the following error:
Failed attempt at deleting data/ account: exception 'Aws\Lambda\Exception\LambdaException' with message 'Error executing "Invoke" on "https://lambda.eu-west-2.amazonaws.com/2015-03-31/functions/blahFunction/invocations"; AWS HTTP error: Client error: `POST https://lambda.eu-west-2.amazonaws.com/2015-03-31/functions/blahFunction/invocations` resulted in a `403 Forbidden` response:
{"Message":"User: arn:aws:iam::34234324324342:user/SecretGuy is not authorized to perform: lambda:InvokeFunction on resour (truncated...)
AccessDeniedException (client): User: arn:aws:iam::34234324324342:user/SecretGuy is not authorized to perform: lambda:InvokeFunction on resource: arn:aws:lambda:eu-west-2:34234324324342:function:blahFunction because no identity-based policy allows the lambda:InvokeFunction action - {"Message":"User: arn:aws:iam::34234324324342:user/SecretGuy is not authorized to perform: lambda:InvokeFunction on resource: arn:aws:lambda:eu-west-2:34234324324342:function:blahFunction because no identity-based policy allows the lambda:InvokeFunction action"}'
Now SecretGuy is a user I created a long time ago, and somehow my ec2 is trying to use that.
I am wondering if anyone can help with this?
Thanks
From my understanding, you are running PHP code on an EC2 instance, and your code invokes the Lambda function.
And this EC2 instance has attached IAM Role with proper permissions to invoke the Lambda function. Then you tried to run your code and faced that the EC2 instance is using an unexpected IAM identity(IAM User named SecretGuy here), not the role you attached.
There's a chance that you might have IAM credentials set by environment variables for that Linux user or static credentials set on your EC2 instance.
AWS SDK client has an order to retrieve credentials on the machine. The official docs of PHP AWS SDK say,
When you initialize a new service client without providing any credential arguments, the SDK uses the default credential provider chain to find AWS credentials. The SDK uses the first provider in the chain that returns credentials without an error.
The default provider chain looks for and uses credentials as follows, in this order:
Use credentials from environment variables.
Setting environment variables is useful if you're doing development work on a machine other than an Amazon EC2 instance.
Use the AWS shared credentials file and profiles.
This credentials file is the same one used by other SDKs and the AWS CLI. If you're already using a shared credentials file, you can use that file for this purpose.
We use this method in most of our PHP code examples.
Assume an IAM role.
IAM roles provide applications on the instance with temporary security credentials to make AWS calls. For example, IAM roles offer an easy way to distribute and manage credentials on multiple Amazon EC2 instances.
To retrieve IAM credentials from the role attached,
You can check which IAM Identity you are using to call AWS API with the below command on the EC2 instance. (as that Linux user, you are running PHP code)
aws sts get-caller-identity
Then it will show result as below,
{
"UserId": "ABCDEFGHIJKLMNOPQRSTU",
"Account": "34234324324342",
"Arn": "arn:aws:iam:: 34234324324342:user/SecretGuy"
}
Then you need to look for environment variables set or static credentials files on that EC2 instance.
My guess is maybe someone used aws CLI on that EC2 before, with SecretGuy credentials, So there would be a file $HOME/.aws/credentials.
If the file exists and is confirmed as a SecretGuy access key, you have to delete that file. (If EC2 runs some critical application, you might want to copy all permissions of SecretGuy to the IAM Role you attached before deleting it to avoid unexpected service outage)
Or, you can look for environment variables.
echo $AWS_ACCESS_KEY_ID
If the above commands return the access key id value, you might have to unset environment variables.
After that, your code will retrieve credentials from IAM Role.
I'm trying to create a botocore session (that does not use my local AWS credentials on ~/.aws/credentials). In other words, I want to create a "burner AWS account". With that burner credentials/session, I want to setup an STS client and with that client, assume a role in order to access a DynamoDB database. Can someone provide some example code which accomplishes exactly this?
Because if I want my system to go into production environment, I CANNOT store the AWS credentials on Github because AWS will scan for it. I'm trying to implement a workaround such that we don't have to store ~/.aws/credentials file on Github.
The running a task in Amazon ECS, simply assign an IAM Role to the task.
Amazon ECS will then generate temporary credentials for that IAM Role. Any code that uses an AWS SDK (such as boto3 for Python) knows how to access those credentials via the metadata service.
The result is that your code using boto3 will automatically receive credentials that have the permissions associated with the IAM Role assigned to the task.
See: IAM roles for tasks - Amazon Elastic Container Service
We currently have 2 AWS accounts that we use. For most of the stuff we want to use the AWS account that our web app is hosted on in an EC2 instance so this works fine:
services.AddDefaultAWSOptions(this.Configuration.GetAWSOptions());
services.AddAWSService<IAmazonSQS>();
services.AddAWSService<IAmazonSimpleSystemsManagement>();
However, I want to access EC2 instances in another AWS account. I've configured it to work locally using credentials and from following this guide (where it mentions about using multiple services): https://docs.aws.amazon.com/sdk-for-net/latest/developer-guide/net-dg-config-netcore.html
services.AddDefaultAWSOptions(this.Configuration.GetAWSOptions());
services.AddAWSService<IAmazonSQS>();
services.AddAWSService<IAmazonSimpleSystemsManagement>();
if (this.WebHostEnvironment.IsDevelopment())
{
// This works fine locally, but I don't want to use credential file in production
var other = this.Configuration.GetAWSOptions("other");
services.AddAWSService<IAmazonEC2>(other);
}
else
{
// How do I register other here without putting a credential file on my ec2 instance?
services.AddAWSService<IAmazonEC2>();
}
I'm not sure how to register IAmazonEC2 to use my other account. I don't want to put a credential file on my instance which is how I get it working locally but it doesn't seem right to me on production servers.
I have configured an IAM role that has access to my other account and given it to my EC2 instance. But how do I translate that IAM role to a profile to use where I am registering IAmazonEC2 above?
Any help appreciated. Thanks
There are really two ways to do it...
Option 1: Use an IAM Role
Let's say that the Amazon EC2 instance is running in Account-A and it now wants to query information about Account-B. You could:
Create an IAM Role in Account-B, with a trust policy that trusts the IAM Role being used by the EC2 instance in Account-A
Your code running on the EC2 instance in Account-A can call AssumeRole() (using the normal credentials from Account-A). This will return a set of temporary credentials.
Use those temporary credentials to make API calls to Account-B
Option 2: Use credentials from Account-B
Alternatively, give your program a set of IAM User credentials from Account-B. These could be stored in AWS Systems Manager Parameter Store - AWS Systems Manager or AWS Secrets Manager, and retrieved by using the normal credentials assigned to the EC2 instance in Account-A.
On my client's AWS account, security credentials are generated everytime we login to their AWS sandbox account. This credentials file is automatically generated and downloaded via a Chrome plugin(SAML to AWS STS Key Conversion).
We then have to place the generated content to the ./aws/credentials file inside an EC2 instance in the same AWS account. This is little inconvenient as we have to update the generated credentials and session_token into the credentials file inside the EC2 instance every time we launch a Terraform script.
Is there any way we can attach any role so that we can just use the EC2 instance without entering the credentials into the credentials file.
Please suggest.
Work out what a reasonable, minimal set of permissions the Terraform script needs to create its AWS resources, then create an IAM role with those permissions, then add that IAM role to the instance (or launch a new instance with the role). Don't have a ~/.aws/credentials file on the instance or it will take precedence over the IAM role-based credentials.
I am trying to use aws cookbook with iam roles, but when I trying to not include aws_access_key and aws_secret_access_key in the aws_ebs_volume block, the chef keep showing an error: RightAws::AwsError: AWS access keys are required to operate on EC2.
I assume when cookbook mean omit the resource parameters aws_secret_access_key and aws_access_key, I just delete them from the block.
aws_ebs_volume "userhome_volume" do
provider "aws_ebs_volume"
volume_id node['myusers']['usershome_ebs_volid']
availability_zone node['myusers']['usershome_ebs_zone']
device node['myusers']['usershome_ebs_dev_id']
action :attach
end
Does anyone have the example of aws cookbook with iam roles please?
update:
Do I still need to define aws creeds data bag if I have already have proper iam role attached to the instance?
When I use iam role and aws cookbook, what does the was_ebs_volume block look like?
In order to manage AWS components, you need to provide authentication credentials to the nodein one of two ways:
explicitly pass credentials parameter to the resource
or let the resource pick up credentials from the IAM role assigned to the instance
When you provision the instance, you should assign it the appropriate role in "Step 3. Configure Instance Details" (when using the console). The setting "IAM role" for EC2 automatically deploys and rotates AWS credentials for you, eliminating the need to store your AWS access keys with your application. On an instance provisioned this way, you no longer need to include aws_access_key and aws_secret_access_key in the aws_ebs_volume block.
Here are code examples on how to launch an instance with an IAM role using the IAM and Amazon EC2 CLIs:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html
and here are some code examples:
http://www.getchef.com/blog/2013/12/19/automating-iam-credentials-with-ruby-and-chef/
When you assign the appropriate IAM role during instance provisioning, your code should work without aws_access_key and aws_secret_access_key.
Here are the steps:
Set up your S3, Chef server, and IAM role as described here:
https://securosis.com/blog/using-amazon-iam-roles-to-distribute-security-credentials-for-chef
Execute “knife client ./” to create client.rb and validation.pem, then transfer them from your Chef server into your bucket.
Launch a new instance with the appropriate IAM Role you set up for Chef and your S3 bucket.
Specify your customized cloud-init script in the User Data field or command-line argument as described here:
https://securosis.com/blog/using-cloud-init-and-s3cmd-to-automatically-download-chef-credentials
You can also host the script as a file and load it from a central repository using an include.
Execute chef-client.