I'm trying to use the CloudWatch logs agent on a RedHat instance with an IAM role attached. The role has full access to CloudWatch. I installed and setup the agent using the instructions here:
http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/QuickStartEC2Instance.html#running-ec2-step-2
Even though the IAM role is definitely attached to the instance, I keep seeing this message in /var/log/awslogs.log:
NoCredentialsError: Unable to locate credentials
When I run aws configure list, I can see the details for the IAM role.
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key ******************** iam-role
secret_key ******************** iam-role
region us-east-1 config-file ~/.aws/config
Here is the contents of /var/awslogs/etc/aws.conf.
[plugins]
cwlogs = cwlogs
[default]
region = us-east-1
So why can't the CloudWatch logs agent find and use the IAM role?
So after much banging my head against the wall, I finally figured out what my problem was. I'm using a proxy to enable the CloudWatch agent to communicate with CloudWatch, and I forgot to add NO_PROXY=169.254.169.254 to /var/awslogs/etc/proxy.conf. So when the agent attempted to query the metadata for information about the IAM role, it tried to go through the proxy to get it. Once I added the NO_PROXY in, it worked fine.
Related
We have SSO configured in the main AWS account and we log in to the child AWS account using that SSO link. Now we have created an EKS cluster in the child account but we are not able to view the Node and other resources due to aws-auth config settings. How to check the current role we have assumed in the child account so that we can update the same in the aws-auth configmap of the EKS cluster so that we would be able to see them?
Use the aws cli,
aws sts get-caller-identity --profile <profileName>
will return the assumed role in the form off
"arn:aws:sts:AccountId:assumed-role/RoleName/SSOemail"
and pass the RoleName in --role-name parameter as shown below, this should give you what you ask for.
aws iam get-role --role-name RoleName --profile profileName
Adding some additional info on setting up sso login via your localhost using aws cli, essentially you just need to have aws cli and a configs file that has entries, you can create the configs file on your host and then source it using env variable.
ConfigFile:
[default]
region = region
output = yaml
[profile myProfileName]
sso_start_url =
sso_region =
sso_account_id =
sso_role_name =
region =
output = json
and set env variable to the path of your file that holds the profiles,
AWS_CONFIG_FILE=/path/to/the/config/file
then you can login to you account using
aws sso login --profile myProfileName
and then you will be able to execute the above commands, this is a very neat way to manage and troubleshoot your organization accounts via a single point.
aws deploy register-on-premises-instance --instance-name XXXXX --iam-user-arn arn:aws:iam::XXXXXXXXXXXX:user/LightSailCodeDeployUser --region ap-south-1
An error occurred (AccessDeniedException) when calling the RegisterOnPremisesInstance operation: User: arn:aws:sts::XXXXXXXXXXX:assumed-role/AmazonLightsailInstanceRole/i-XXXXXXXXXXXXXX is not authorized to perform: codedeploy:RegisterOnPremisesInstance on resource: arn:aws:codedeploy:ap-south-1:XXXXXXXXXX:instance:XXXXXXXXXXXX
I didn't even create the role AmazonLightsailInstanceRole, then how did it come in the picture. My user have all permissions on codedeploy though. I am following this link to set up. https://aws.amazon.com/blogs/compute/using-aws-codedeploy-and-aws-codepipeline-to-deploy-applications-to-amazon-lightsail/
I made the same mistake and then realized that command is meant to be run on your local machine and not the instance!
AmazonLightsailInstanceRole is a service-linked role automatically created by aws:
Service-linked roles are predefined by the service and include all the permissions that the service requires to call other AWS services on your behalf.
The error you are getting is not about you not having the codedeploy:RegisterOnPremisesInstance permission.
The error is about the AmazonLightsailInstanceRole not having it. It does not matter if you (i.e. your IAM user) has all CodeDeploy permissions.
Normally you would add the missing permissions to the role. How to work with the AmazonLightsailInstanceRole is described in the following AWS documentaiton:
Using Service-Linked Roles for Amazon Lightsail
Editing a Service-Linked Role
However, I'm not sure if you can modify the AmazonLightsailInstanceRole and add the missing permissions. Some service-linked roles can be modified, some not.
The documentation is a bit confusing. Create a new user in IAM with admin role (full privileges) and use the credentials of that user to run the command in your local machine.
Goal: Retrieve secret from secretsmanager on an aws ec2 instance programmatically through command line.
I have created an IAM role with policies that grant full-access to AWSSecretsManager and AWSEC2instance also to assume the role and modify the role of any aws ec2 instance.
I created an aws instance and attached the IAM role to it and executed the following steps:
- aws secretsmanager list-secrets
An error occurred (UnrecognizedClientException) when calling the ListSecrets operation: The security token included in the request is invalid.
I get an error. I am able to retrieve the security credentials using the metadata of the instance.
- Am I missing something here? I basically want to retrieve the secret in an aws instance in a secure way.
- When I try to run the above command to list-secrets. The cli complains that it needs an region. My ec2-instance and secrets all are in us-east-2. So, I use the same region. And it still does not work.
Any suggestions/pointers would be highly appreciated. Thanks!
Here is How I would troubleshoot.
check whether the instance is aware of the IAM role attached to that.
aws sts get-caller-identity
try passing the region to the command
aws secretsmanager list-secrets --region us-east-2
I would check whether the AWS_REGION or AWS_DEFAULT_REGION, but even if these values are set, passing --region should override it.
Hope this help you get somewhere.
Have you run "aws configure" on the instance? Sounds like it might be using the token in there rather that the EC2 instance role. See references below for the sequence it checks but basically, the EC2 role is the last place it looks, if it gets credentials earlier, it will use them.
See here for the priority/sequence: https://docs.aws.amazon.com/amazonswf/latest/awsrbflowguide/set-up-creds.html
https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html ("Using the Default Credential Provider Chain")
In the logstash configuration file, I gave the following input plugin.
input{
s3{
bucket => 'bucket_name'
region => 'eu-west-1'
}
}
When I started logstash, it threw an error asking for AWS credentials. But I cannot provide AWS access_key_id and secret_key. I found that with IAM roles on EC2 instances, we shouldn't have to provide AWS credentials explicitly for an application that make those requests. I tried to understand how to configure IAM roles from a logstash Github issue, but failed. Please suggest how I should configure logstash file so that I can get data from S3 without providing AWS credentials explicitly.
When you are creating the EC2 instance, you may assign it an IAM role. If you don't give it a role when you create it, you cannot ever give it a role. You can modify the properties of a role (e.g. give the role more or less access) but you cannot change what role an instance has after the instance has been created.
The easiest way to test if an instance has the credentials you need is with the command-line tools:
$ echo $AWS_ACCESS_KEY_ID
$
That shows that we don't have the environment variable set, so if we're getting credentials, they must be instance creds.
$ aws --region=eu-west-1 s3 ls my.bucket
PRE some-directory/
PRE another-dir/
2015-11-05 18:03:53 464 little-file
2014-10-28 15:32:13 19740 bigger-file.html
$
I was able to list the bucket contents, so the EC2 instance where I'm running this command must have an IAM role that allows it to list this bucket contents!
I am trying to use aws cookbook with iam roles, but when I trying to not include aws_access_key and aws_secret_access_key in the aws_ebs_volume block, the chef keep showing an error: RightAws::AwsError: AWS access keys are required to operate on EC2.
I assume when cookbook mean omit the resource parameters aws_secret_access_key and aws_access_key, I just delete them from the block.
aws_ebs_volume "userhome_volume" do
provider "aws_ebs_volume"
volume_id node['myusers']['usershome_ebs_volid']
availability_zone node['myusers']['usershome_ebs_zone']
device node['myusers']['usershome_ebs_dev_id']
action :attach
end
Does anyone have the example of aws cookbook with iam roles please?
update:
Do I still need to define aws creeds data bag if I have already have proper iam role attached to the instance?
When I use iam role and aws cookbook, what does the was_ebs_volume block look like?
In order to manage AWS components, you need to provide authentication credentials to the nodein one of two ways:
explicitly pass credentials parameter to the resource
or let the resource pick up credentials from the IAM role assigned to the instance
When you provision the instance, you should assign it the appropriate role in "Step 3. Configure Instance Details" (when using the console). The setting "IAM role" for EC2 automatically deploys and rotates AWS credentials for you, eliminating the need to store your AWS access keys with your application. On an instance provisioned this way, you no longer need to include aws_access_key and aws_secret_access_key in the aws_ebs_volume block.
Here are code examples on how to launch an instance with an IAM role using the IAM and Amazon EC2 CLIs:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html
and here are some code examples:
http://www.getchef.com/blog/2013/12/19/automating-iam-credentials-with-ruby-and-chef/
When you assign the appropriate IAM role during instance provisioning, your code should work without aws_access_key and aws_secret_access_key.
Here are the steps:
Set up your S3, Chef server, and IAM role as described here:
https://securosis.com/blog/using-amazon-iam-roles-to-distribute-security-credentials-for-chef
Execute “knife client ./” to create client.rb and validation.pem, then transfer them from your Chef server into your bucket.
Launch a new instance with the appropriate IAM Role you set up for Chef and your S3 bucket.
Specify your customized cloud-init script in the User Data field or command-line argument as described here:
https://securosis.com/blog/using-cloud-init-and-s3cmd-to-automatically-download-chef-credentials
You can also host the script as a file and load it from a central repository using an include.
Execute chef-client.