I'm trying to add aws cloudwatch agent to see additional metrics using tutorial
A brief review of what I did:
Create AIM role and attach to EC2 instance doc (NOTE: I do not use Parameter Store just for communication between EC2 and cloudwatch)
Install Agent using s3 link
Create agent configuration file docs
Run agent using CLI dosc
But it still not working and in agent log, I see errors like
ec2tagger: Unable to initialize EC2 Instance Tags : +NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
While googling I found not much related to cloudwath just only that in AIM role in 'Trust Relationship' config ec2 should be mentioned in service section and it is:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
Any ideas, thanks!?
In my case the instance had an IAM role attached, but the role was missing the ec2:DescribeTags permission. Adding that fixed the problem.
"The first procedure creates the IAM role that you must attach to each Amazon EC2 instance that runs the CloudWatch agent. This role provides permissions for reading information from the instance and writing it to CloudWatch." in docs
please attach IAM role that you created to your ec2 instance first,it works for me
The cloudwatch agent process that runs in the ec2 should be able to describe the tags of ec2. The permission required for that is ec2:DescribeTags.
Attaching instance role with the managed policy arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy will resolve the problem.
Check to see if the CloudWatch Agent service is running (started)
I got the same issue, resolve by using below command, refresh routes
Import-Module C:\ProgramData\Amazon\EC2-Windows\Launch\Module\Ec2Launch.psm1; Add-Routes
Solved by running aws configure from inside the instance
Related
I've currently writing a Terraform module for EC2 ASGs with ECS. Everything about starting the instances works, including IAM-dependent actions such as KMS-encrypted volumes. However, the ECS agent fails with:
Error getting ECS instance credentials from default chain: NoCredentialProviders: no valid providers in chain.
Unfortunately, most posts I find about this are about the CLI being run without credentials configured, however this should of course use the instance profile.
The posts I do find regarding that are about missing IAM policies for the instance role. However, I do have this trust relationship
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"ec2.amazonaws.com",
"ecs.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}
(Added ECS because someone on SO had it in there, I don't think that's right. Also removed some conditions.)
It has these policies attached:
AmazonSSMManagedInstanceCore
AmazonEC2ContainerServiceforEC2Role
When I connect via SSH and try to run awscli commands, I get the error
Unable to locate credentials. You can configure credentials by running "aws configure".
running any command.
But with
curl http://169.254.169.254/latest/meta-data/iam/info
I see the correct instance profile ARN and with
curl http://169.254.169.254/latest/meta-data/identity-credentials/ec2/security-credentials/ec2-instance
I see temporary credentials. With these in the awscli configuration,
aws sts get-caller-identity
returns the correct results. There are no iptables rules or routes blocking access to the metadata service and I've deactivated IMDSv2 token requirement for the time being.
I'm using the latest stable version of ECS-optimized Amazon Linux 2.
What might be the issue here?
I want creat a connector for Debezium. I am unable to create a kafka connector using the AWSServiceRoleForKafkaConnect role. Here is the error I get when clicking Create Connector on the last page in the form:
Error creating connector There was a problem creating a connector. If the problem persists, contact AWS Support. API response Invalid parameter serviceExecutionRoleArn: A service linked role ARN cannot be provided as service execution role ARN.
Here is the document which I am following:
https://aws.amazon.com/tr/blogs/aws/introducing-amazon-msk-connect-stream-data-to-and-from-your-apache-kafka-clusters-using-managed-connectors/
How Can I fix this problem ? Thanks!
I'm having the same issue in the last few days. I have created Kafka connectors in early Jan and December with no issues using the provided AWSServiceRoleForKafkaConnect role.
I tried creating the new connectors with the same config on the same cluster using the same role that worked before and now I'm also getting the same error:
Invalid parameter serviceExecutionRoleArn: A service linked role ARN cannot be provided as service execution role ARN.
I think this may be due to an update on the AWS side.
Update from AWS****
Hi there, there were some recent changes made to the use of Service Linked Roles (SLR) as an execution role for Amazon MSK Connect Connectors. SLRs are no longer allowed to be used as the execution role. This may not affect Connectors which do not interact with Amazon MSK clusters or other resources using IAM authentication, but will affect any interaction with IAM controlled resources. It is recommend that all connector execution roles use customer managed roles with Trust Relationships including kafkaconnect.amazonaws.com. See the documentation on Service Execution Roles (1) for more details.
Currently the console allows the selection of an SLR (AWSServiceRoleForKafkaConnect) as the execution role, this will be removed in future updates and should not be used when creating connectors.
(1) https://docs.aws.amazon.com/msk/latest/developerguide/msk-connect-service-execution-role.html
I had a similar problem while creating MSK Connector to S3 bucket. You can solve it using instructions from this tutorial:
https://docs.aws.amazon.com/msk/latest/developerguide/mkc-tutorial-setup.html
You have to create new IAM Role and edit Trust relationship:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "kafkaconnect.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
Context: I was going though Link to Amazon Web Services to create Swarms, in order to connect to my provider.
The role was created with success.
Then, while creating the policy, to associate to the role, a problem happened.
Problem:
An error occurred: Cannot exceed quota for PolicySize: 5120
As suggested by them, this is what I need to add in policy:
https://docs.docker.com/docker-for-aws/iam-permissions/
Did some research and people seem to like this solution:
https://github.com/docker/machine/issues/1655
How can I create the policy using the best method?
Noticing that the documentation in Docker is wrong - doesn't work in my case - what's the best method?
You are looking at the wrong instructions to connect docker-cloud to AWS follow these instructions: https://docs.docker.com/docker-cloud/infrastructure/link-aws/
It's the following 3 steps
Create AWS Policy for docker-cloud
Create a docker-cloud role and attache the policy from 1
Attach AWS role/account to docker-cloud
The policy in (1) above is pretty simple. It should be allowed to perform ec2 instances related actions (your screenshot of the policy looks like it doesn't provide ec2 permissions):
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ec2:*",
"iam:ListInstanceProfiles"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
The role must have the permissions to implement the policy.
For a detailed post on the deployment via docker-cloud see: https://blog.geografia.com.au/how-we-are-using-docker-cloud-for-automated-testing-and-deployments-of-applications-bb87ec3173e7
I am attempting to run a boto3 python script inside a docker container using AWS ECS. My script need access to SQS ( get & delete messages ) and Lambda ( permission to search and run ).
In order to get the docker container running on my local machine I was able to pass my aws credentials into the docker container using the following docker run command.
docker run -v ~/.aws:/root/.aws
Recently ECS has announced:
Amazon ECS now supports IAM roles for tasks. When you specify an IAM role for a task, its containers can then use the latest versions of the AWS CLI or SDKs to make API requests to authorized AWS services. Learn More
I attach a task IAM role to the task but upon running the task I get the following error:
Unable to run task
ECS was unable to assume the role that was provided for this task. Please verify that the role being passed has the proper trust relationship and permissions and that your IAM user has permissions to pass this role.
Any ideas would be appreciated.
It looks like IAM Task Roles are now supported in Boto, but regardless, that would be an issue when the Boto client was trying to make a request, not when trying to launch a task.
The issue here is defined in the error message. Either:
1) Your user does not have the iam:PassRole permission defined for the task role. This can be added by editing your user's policy to have a statement similar to the following:
{
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "arn:aws:iam::<account>:role/<role name>"
}
2) The Task role you are trying to assign to the task does not have the proper trust relationship. Add the following trust policy to the ECS task role to make sure that it can be assumed by the task.
{
"Version": "2008-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ecs-tasks.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
Boto3 uses the botocore library for its API functions, and it allows a range of botocore versions for each Boto3 version, so even if you have the latest Boto3 version, you may not have the latest botocore.
Botocore supports ECS IAM roles for tasks as of version 1.4.37, so if you update the underlying botocore in your environment to at least that version, you should be able to use the ECS IAM roles for tasks feature.
I am trying to setup code deployment using aws, but when I try to perform deployment, I am getting this error:
2016-06-08 23:57:11 ERROR [codedeploy-agent(1207)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller: Cannot reach InstanceService: Aws::CodeDeployCommand::Errors::AccessDeniedException -
2016-06-08 23:58:41 INFO [codedeploy-agent(1207)]: Version file found in /opt/codedeploy-agent/.version.
2016-06-08 23:58:41 INFO [codedeploy-agent(1207)]: [Aws::CodeDeployCommand::Client 400 0.055741 0 retries] poll_host_command(host_identifier:"IAM-user-ARN") Aws::CodeDeployCommand::Errors::AccessDeniedException
I have two IAM roles - one for EC2 instance, and one for deployment app.
S3 bucket have permission set for iam role which is used for deployment:
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "XXXXXXXX:role/TestRole"
},
"Action": [
"s3:Get*",
"s3:List*"
],
"Resource": "arn:aws:s3:::pmcdeploy/*"
}
]
}
What is going on?
Is the error consistent? On looking at the agent code, it seems like the agent might having trouble talking to EC2. If this is a persistent problem, you can share the EC2 instance profile.
Also starting the agent with verbose option enabled gives a lot more information about what's going on.
Thanks
This is actually something related to the order of credential loading. The host agent is running with root user by default and also uses instance profile.
The exception is got when you've setup a root credential which has priority over instance profile according to: http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#config-settings-and-precedence
Then the aws sdk used by host agent will use the credential configured for the root user instead of instance profile to configure the requests.
One of the workaround would be run the agent with a different user and don't configure any credential for that user.
We had what I think the same issue.
Our systems had a /root/.aws/credentials in place which CodeDeploy absolutely uses and I found no way of telling it to not do that.
Especially no documentation...
In the end, we rewrote everything on our end to ensure we'll no longer need a credentials file in place.
From that moment on, CodeDeploy used the instance profile and it was working fine.
I deleted /home/ubuntu/.aws and rebooted codedeploy agent service and it worked for me :-)