IAM SDK AmazonIdentityManagement; Status Code: 404; Error Code: NoSuchEntity; - amazon-web-services

AmazonIdentityManagementClient identityManagementClient = new AmazonIdentityManagementClient();
System.out.println("This is the role");
GetRoleResult roleResult = identityManagementClient.getRole(new GetRoleRequest().withRoleName("myRole"));
System.out.println(roleResult.getRole().toString());
System.out.println("This is the Policy");
GetRolePolicyResult rolePolicyResult = identityManagementClient.getRolePolicy(new GetRolePolicyRequest()
.withRoleName("myRole").withPolicyName("AmazonS3FullAccess"));
System.out.println(rolePolicyResult.getPolicyDocument());
System.out.println(rolePolicyResult.getPolicyName());
I have attached IAM role to my instance - myRole. I have attached it AmazonS3FullAccess policy to myRole. The above code I am using to fetch the policies associated with myRole using SDK but I am facing NoSuchEntity Exception when I try to fetch Policy. I am able to get Role name and my output is myRole but I am facing error when I try to fetch Policy details.
I don't know why.
What I am doing wrong here?

I only managed to get the policy document in an alternative way (AWS Java SDK v.1x) - by using getPolicyVersion method of the AmazonIdentityManagementClient.
The GetPolicyVersionRequest for this is for the policyArn (which you can get from the policy details of the listAttachedRolePolicies result for your role name) and the versionId is from calling the getDefaultVersionId() on the policy returned by the AmazonIdentityManagementClient 's getPolicy for your policyArn.

Related

How to use iam role when creating aws eks cluster with terraform?

I am trying to create aws eks cluster with terraform. I want to use an existing iam role. I won't be creating a new role. I tried this.
module 'eks' {
...
iam_role_arn = "arn:aws:iam::11111111:role/cluster-role"
...
}
But I get 403 permission denied. How can I this? Thanks.
Edit: added error.
failed creating IAM Role (green-eks-node-group): AccessDenied: User: arn:aws:iam::*******:user/****** is not authorized to perform: iam:TagRole on resource: arn:aws:iam::*******:role/green-eks-node
First for good measure: Please do not post pictures here of either code or error messages. Just copy paste the stuff and format it.
That said: I think the error tells you exactly what is wrong (or at least what the first error is, because in my experience it takes a few iterations of this to get your IAM permissions straight).
The role that you use you execute terraform does not have the permission:
iam:TagRole
on the resource:
arn:aws:iam:xxxxxxxxxxx:role/eks-cluster
So you will need to add it to it's policy.

Aws Emr Cluster creation with RunJobFlowResult unable to take specified config?

I am creating an Aws Emr cluster with AWS Java SDK. Below is the code snippet.
JobFlowInstancesConfig jobFlowInstanceConfig = new JobFlowInstancesConfig()
.withEc2SubnetId(config.getEc2SubnetId())
.withEc2KeyName(config.getEc2KeyName())
.withInstanceCount(config.getInstanceCount())
.withKeepJobFlowAliveWhenNoSteps(true)
.withMasterInstanceType(config.getMasterInstanceType())
.withSlaveInstanceType(config.getSlaveInstanceType());
RunJobFlowRequest request = new RunJobFlowRequest()
.withName(clusterName)
.withReleaseLabel(config.getReleaseLabel())
.withApplications(applications)
.withLogUri(config.getLogUri())
.withServiceRole(config.getServiceRole())
.withJobFlowRole(config.getJobFlowRole())
.withInstances(jobFlowInstanceConfig);
RunJobFlowResult runJobFlowResult = emrClient.runJobFlow(request);
As you can see I am setting "JobFlowRole" using .withJobFlowRole(config.getJobFlowRole()), but it is taking default values which does not have permission to create cluster.
I am getting following error:
com.amazonaws.services.elasticmapreduce.model.AmazonElasticMapReduceException: User: arn:aws:sts::6...0:assumed-role/default-role/i-0...4 is not authorized to perform: iam:PassRole on resource: arn:aws:iam::6...0:role/EMR_DefaultRole (Service: AmazonElasticMapReduce; Status Code: 400; Error Code: AccessDeniedException; Request ID: a...f)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1701)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1356)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1102)
Help please.
The JobFlowRole is the role of EMR service and this is not the role for creation EMR. See documentation.
You should have the right permission to create an EMR where you used to get the AWS credentials. The iam:PassRole is missing for your credentials.

Not able to attach policy to a role using Boto3

Using BOTO3 script,Created a Role and a Policy and trying to attached policy to that role. I am getting error while attaching but if i do attach manually then working fine.
Using BOTO3 i am doing followings:
Created a AWS role say "TEST"
Created a policy called "POL"
Both have been created and we can see on AWS console. Now attaching policy to Role with below command
response = client.attach_role_policy(
RoleName='TEST',
PolicyArn='arn:aws:iam::6929051012:policy/POL'
)
getting below error.
raise error_class(parsed_response, operation_name)
botocore.errorfactory.NoSuchEntityException: An error occurred (NoSuchEntity) when calling the AttachRolePolicy operation: Policy arn:aws:iam::6929051012:policy/POL does not exist or is not attachable.
Manually i can attached this policy to Role.
Your Help is highly appreciated. Thanks
To reproduce your situation, I did the following:
Created an IAM Role (stack-role) via the management console
Created an IAM Policy via the management console (arn:aws:iam::123456789012:policy/stack-policy)
I then ran:
import boto3
iam_client = boto3.client('iam')
response = iam_client.attach_role_policy(
RoleName='stack-role',
PolicyArn='arn:aws:iam::123456789012:policy/stack-policy'
)
print (response)
The call returned successfully. I then looked at the Role in the IAM management console and the stack-policy was attached.
So, seems to work fine!

Configure EMR to use s3a instead of s3 for spark.sql calls

All my calls to spark.sql("") fails with the error in the stacktrace (1) below
Update - 2
I have zeroed in on the problem, it is AccessDenied for sts:AssumeRule, any leads appreciated
User: arn:aws:sts::00000000000:assumed-role/EMR_EC2_XXXXX_XXXXXX_POLICY/i-3232131232131232 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::00000000000:role/EMR_XXXXXX_XXXXXX_POLICY
When the same location is accessed with
spark.read.parquet("s3a://xxx.xxx-xxx-xx.xxxxx-xxxxx/xxx/")
I was able to read the records.
But the same stacktrace (1) resurfaces when access with s3: instead of s3a: scheme
spark.read.parquet("s3://xxx.xxx-xxx-xx.xxxxx-xxxxx/xxx/")
So how can I configure Spark on EMR to use s3a: or have s3: running without the access denied which is presume because it may not be using the appropriate credential chain
(1)
Caused by: com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException: Access denied (Service: AWSSecurityTokenService; Status Code: 403; Error Code: AccessDenied; Request ID: xxxxx-xxxx-xxxx-xxxx-xxxxxxxx)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1658)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1322)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1072)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:745)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:719)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:701)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:669)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:651)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:515)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.doInvoke(AWSSecurityTokenServiceClient.java:1369)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1338)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1327)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.executeAssumeRole(AWSSecurityTokenServiceClient.java:488)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.assumeRole(AWSSecurityTokenServiceClient.java:460)
Update - 1
Tried setting secret and access key doesn't work
spark.sparkContext.hadoopConfiguration.set("fs.s3.awsAccessKeyId", "")
spark.sparkContext.hadoopConfiguration.set("fs.s3.awsSecretAccessKey", "")
this stack trace says "amazon EMR S3 client"; not the Apache ASF one, so different settings, and error messages.
That error message about "assumed role" hints that you are running in an EC2 VM (yes?), and that "assumed role" is actually the IAM role the EC2 VM is deployed as. In which case (a) no other credentials are being picked up and (b) that VM doesn't have permissions to access the role. Fixes: work out the setting to get the credentials in, increase EC2 IAM role rights, or create VMs with a different role

AccessDeniedException on CloudWatchEvents.putRule()

What is the right policy i have to attach to an iam user to use CloudWatchEvents.putRule() ?
i tryed CloudWatchFullAccess and CloudWatchEventsFullAccess, both give me an AccessDeniedException for my webclient.
Just ran into this. If your policy does not have a region set for the arn allowing PutRule, you will get this 400 Access Exception. Set region to '*' or your specific region.