AWS MSK Connectors for Debezium IAM Role Problem - amazon-web-services

I want creat a connector for Debezium. I am unable to create a kafka connector using the AWSServiceRoleForKafkaConnect role. Here is the error I get when clicking Create Connector on the last page in the form:
Error creating connector There was a problem creating a connector. If the problem persists, contact AWS Support. API response Invalid parameter serviceExecutionRoleArn: A service linked role ARN cannot be provided as service execution role ARN.
Here is the document which I am following:
https://aws.amazon.com/tr/blogs/aws/introducing-amazon-msk-connect-stream-data-to-and-from-your-apache-kafka-clusters-using-managed-connectors/
How Can I fix this problem ? Thanks!

I'm having the same issue in the last few days. I have created Kafka connectors in early Jan and December with no issues using the provided AWSServiceRoleForKafkaConnect role.
I tried creating the new connectors with the same config on the same cluster using the same role that worked before and now I'm also getting the same error:
Invalid parameter serviceExecutionRoleArn: A service linked role ARN cannot be provided as service execution role ARN.
I think this may be due to an update on the AWS side.
Update from AWS****
Hi there, there were some recent changes made to the use of Service Linked Roles (SLR) as an execution role for Amazon MSK Connect Connectors. SLRs are no longer allowed to be used as the execution role. This may not affect Connectors which do not interact with Amazon MSK clusters or other resources using IAM authentication, but will affect any interaction with IAM controlled resources. It is recommend that all connector execution roles use customer managed roles with Trust Relationships including kafkaconnect.amazonaws.com. See the documentation on Service Execution Roles (1) for more details.
Currently the console allows the selection of an SLR (AWSServiceRoleForKafkaConnect) as the execution role, this will be removed in future updates and should not be used when creating connectors.
(1) https://docs.aws.amazon.com/msk/latest/developerguide/msk-connect-service-execution-role.html

I had a similar problem while creating MSK Connector to S3 bucket. You can solve it using instructions from this tutorial:
https://docs.aws.amazon.com/msk/latest/developerguide/mkc-tutorial-setup.html
You have to create new IAM Role and edit Trust relationship:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "kafkaconnect.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}

Related

AccessDeniedException on sagemaker:CreateDomain in AWS SageMaker Studio, despite having SageMakerFullAccess

I am trying to use the AWS SageMaker Studio > Get Started > Quick Start, as an IAM user with the AmazonSageMakerFullAccess policy attached, but I am getting the following error:
User: arn:aws:iam::<user-id>:user/<username> is not authorized to perform: sagemaker:CreateDomain on resource: arn:aws:sagemaker:us-west-1:<user-id>:domain/d-<domain-id>
I looked up some documentation on the CreateDomain command, and it looks like it involves EFS storage and VPC configuration, so I have also added the FullAccess policies for these services to my IAM user, but am still getting the same error.
I also tried adding a custom policy as shown here: https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html#sagemaker-roles-createdomain-perms which also seemed to have no effect.
What am I doing wrong here?
AmazonSageMakerFullAccess policy gives the user access to perform actions such as start training jobs, deploy endpoints, along with limited access on other services such as ECR, Glue etc. This is generally attached to a SageMaker notebook instance or Studio.
The user creating the SageMaker domain needs sagemaker:CreateDomain permission, i.e., to your IAM user, add:
{
"Sid": "AllowCreateDomain",
"Effect": "Allow",
"Action": "sagemaker:CreateDomain",
"Resource": "*"
}
I work at AWS and my opinions are my own.

What is the actual structure of ECS Service ARNs?

I'm trying to grant the ecs:UpdateService permission in an IAM policy like below:
{
"Sid": "AllowECS",
"Effect": "Allow",
"Action": [
"ecs:UpdateService"
],
"Resource": "arn:aws:ecs:ap-southeast-2:123456789012:service/my-service"
}
where the service ARN is as listed by the cli aws ecs list-services (and describe-services).
However the UpdateService API call fails with an AccessDenied error saying the process does not have permission to the service with ARN:
arn:aws:ecs:ap-southeast-2:123456789012:service/*my-cluster*/my-service
I cannot find any reference to the ARN for an ECS service including the the cluster name, and the documentation for the cli "ecs describe-services" explicitly says:
serviceArn -> (string)
The ARN that identifies the service. The ARN
contains the arn:aws:ecs namespace, followed by the Region of the
service, the AWS account ID of the service owner, the service
namespace, and then the service name. For example,
arn:aws:ecs:region:012345678910:service/my-service .
If I change the IAM policy to actually use the ARN shown in the error, which includes the cluster name, then the UpdateService completes successfully.
I'm not sure what I'm missing here, should an ECS service ARN include the cluster name? If so, how can I obtain that version of the ARN as the AWS APIs all seem to return the version that doesn't include the cluster name?
I had a similar issue lately as well.
Basically, there is a new ECS ARN as mentioned here. It's still under opt-in period till end of this year.
Please find below the old and new format comparison.
From your question, I think that your AWS account is also opted in for new format.
More details can be found FAQ

AWS API Gateway. Update existing API from github. add new route

I am going to Update existing Api gateway through aws cli commands (https://docs.aws.amazon.com/cli/latest/reference/apigateway/put-rest-api.html) from AWS CodePipeline and meet problem that CodeBuild has
An error occurred (AccessDeniedException) when calling the GetRestApis operation: User: arn:aws:sts:<skipped_text> is not authorized to perform: apigateway:GET on resource: arn:aws:apigateway:us-west-2::/restapis
Is it possible to update Api gateway through code using CodePipeline and aws cli?
Or What do you use as a tool for updating apigateway?
Make sure to add following policy to the IAM user in order to allow admin access (CREATE, READ, UPDATE, DELETE) to API gateway,
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"apigateway:*"
],
"Resource": "arn:aws:apigateway:*::/*"
}
]
}
which also exists under amazon managed policies, AmazonAPIGatewayAdministrator,
1) Please check your credential working properly or not ,It appears that the credentials used in your AWS SDK do not have valid permissions on the API Gateway API.
You will need to define a policy for the user/role similar to below.
http://docs.aws.amazon.com/apigateway/latest/developerguide/permissions.html
2) When using CloudFormation with CodePipeline, need to create a role that can be assumed by CloudFormation,CodePipeline moves releases through a pipeline using the role you specify for a pipeline. CloudFormation needs permission to assume a separate role that you create for the CloudFormation action (it's not enough to create a role with permission to access other resources).
References
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline-basic-walkthrough.html
https://s3.amazonaws.com/cloudformation-examples/user-guide/continuous-deployment/basic-pipeline.yml
aws forums

Amazon cloudwatch agent not working

I'm trying to add aws cloudwatch agent to see additional metrics using tutorial
A brief review of what I did:
Create AIM role and attach to EC2 instance doc (NOTE: I do not use Parameter Store just for communication between EC2 and cloudwatch)
Install Agent using s3 link
Create agent configuration file docs
Run agent using CLI dosc
But it still not working and in agent log, I see errors like
ec2tagger: Unable to initialize EC2 Instance Tags : +NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
While googling I found not much related to cloudwath just only that in AIM role in 'Trust Relationship' config ec2 should be mentioned in service section and it is:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
Any ideas, thanks!?
In my case the instance had an IAM role attached, but the role was missing the ec2:DescribeTags permission. Adding that fixed the problem.
"The first procedure creates the IAM role that you must attach to each Amazon EC2 instance that runs the CloudWatch agent. This role provides permissions for reading information from the instance and writing it to CloudWatch." in docs
please attach IAM role that you created to your ec2 instance first,it works for me
The cloudwatch agent process that runs in the ec2 should be able to describe the tags of ec2. The permission required for that is ec2:DescribeTags.
Attaching instance role with the managed policy arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy will resolve the problem.
Check to see if the CloudWatch Agent service is running (started)
I got the same issue, resolve by using below command, refresh routes
Import-Module C:\ProgramData\Amazon\EC2-Windows\Launch\Module\Ec2Launch.psm1; Add-Routes
Solved by running aws configure from inside the instance

How can I allow a single user to have access of an ec2 instance in AWs console using IAM

I am exploring IAM. I want to give access to a single ec2 instance to a user. I have created a policy for this as:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1392113879000",
"Effect": "Allow",
"Action": [
"ec2:*"
],
"Resource": [
"arn:aws:ec2:us-east-1:account:instance/instance_id"
]
}
]
}
But I am getting this error:
I have referred to this link
Any lead is appriciated.
The Resource-Level Permissions for EC2 and RDS Resources you are referring to are not yet available for all API actions, but AWS in gradually adding more, see this note from Amazon Resource Names for Amazon EC2:
Important Currently, not all API actions support individual ARNs; we'll add support for additional API actions and ARNs for additional
Amazon EC2 resources later. For information about which ARNs you can
use with which Amazon EC2 API actions, as well as supported condition
keys for each ARN, see Supported Resources and Conditions for Amazon
EC2 API Actions.
You will find that all ec2:Describe* actions are indeed absent still from Supported Resources and Conditions for Amazon EC2 API Actions at the time of this writing, and these are the ones required for listing resources e.g. in the AWS Management Console and triggering the errors you are seeing in turn ("You are not authorized to describe ...").
See also Granting IAM Users Required Permissions for Amazon EC2 Resources for a concise summary of the above and details on the ARNs and Amazon EC2 condition keys that you can use in an IAM policy statement to grant users permission to create or modify particular Amazon EC2 resources - this page also mentions that AWS will add support for additional actions, ARNs, and condition keys in 2014.