I am creating an Aws Emr cluster with AWS Java SDK. Below is the code snippet.
JobFlowInstancesConfig jobFlowInstanceConfig = new JobFlowInstancesConfig()
.withEc2SubnetId(config.getEc2SubnetId())
.withEc2KeyName(config.getEc2KeyName())
.withInstanceCount(config.getInstanceCount())
.withKeepJobFlowAliveWhenNoSteps(true)
.withMasterInstanceType(config.getMasterInstanceType())
.withSlaveInstanceType(config.getSlaveInstanceType());
RunJobFlowRequest request = new RunJobFlowRequest()
.withName(clusterName)
.withReleaseLabel(config.getReleaseLabel())
.withApplications(applications)
.withLogUri(config.getLogUri())
.withServiceRole(config.getServiceRole())
.withJobFlowRole(config.getJobFlowRole())
.withInstances(jobFlowInstanceConfig);
RunJobFlowResult runJobFlowResult = emrClient.runJobFlow(request);
As you can see I am setting "JobFlowRole" using .withJobFlowRole(config.getJobFlowRole()), but it is taking default values which does not have permission to create cluster.
I am getting following error:
com.amazonaws.services.elasticmapreduce.model.AmazonElasticMapReduceException: User: arn:aws:sts::6...0:assumed-role/default-role/i-0...4 is not authorized to perform: iam:PassRole on resource: arn:aws:iam::6...0:role/EMR_DefaultRole (Service: AmazonElasticMapReduce; Status Code: 400; Error Code: AccessDeniedException; Request ID: a...f)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1701)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1356)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1102)
Help please.
The JobFlowRole is the role of EMR service and this is not the role for creation EMR. See documentation.
You should have the right permission to create an EMR where you used to get the AWS credentials. The iam:PassRole is missing for your credentials.
Related
I have a service that is deployed using Kubernetes and Docker.
To call Amazon service (SP-API) we have created a role (SellerRole) as per this document. https://developer-docs.amazon.com/sp-api/docs/creating-and-configuring-iam-policies-and-entities
We have one user who has this role assigned. Now, using the user if we do assume the above role (SellerRole) it gives us the temporary credentials and works fine.
Since the service is deployed using K8 I am trying to use the IRSA and role to do the same thing.
I have created an IRSA and given the K8 cluster a role (PODRole). The PODRole has access to assume SellerRole. Also, the pods have a token file that confirms that the pods are configured correctly.
Now the issue is when I ssh into POD and do
aws sts assume-role --role-name SellerRole --session-name piyush-session
it works correctly and gives back the temp credentials.
Hoowever when I try to do the same from code it gives an error. Below is the code.
StsClient.builder()
.region(region)
.credentialsProvider(WebIdentityTokenFileCredentialsProvider.create())
.build();
AssumeRoleRequest roleRequest =
AssumeRoleRequest.builder()
.roleArn("SellerRole")
.roleSessionName("SessionName")
.build();
AssumeRoleResponse roleResponse = stsClient.assumeRole(roleRequest);
Credentials credentials = roleResponse.credentials();
Below is the error.
Unable to assume role. Exception: software.amazon.awssdk.services.sts.model.StsException: User: arn:aws:sts::id:assumed-role/eks-qa01-PODRole/aws-sdk-java-1661372423393 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::12345678999:role/eks-qa01-PODRole (Service: Sts, Status Code: 403, Request ID: b6a8f294-52d8-450f-9698)
I am trying to use glueContext.purge_table function in my aws glue job. Whenever the job is executed it throws the following error:
An error occurred while calling o82.purgeTable.
: java.lang.RuntimeException: class com.amazonaws.services.gluejobexecutor.model.AccessDeniedException:User: arn:aws:sts::012345678:assumed-role/XYZ/GlueJobRunnerSession is not authorized to perform: lakeformation:GetDataAccess on resource: arn:aws:glue:us-east-1:MICHIGAN_DEFAULT_CATALOG_ID_RANDOMIZED:table/database/table (Service: AWSLakeFormation; Status Code: 400; Error Code: AccessDeniedException; Request ID: 25829fe6-2a10-430a-b050-023c13bcc8ce; Proxy: null) (Service: AWSGlueJobExecutor; Status Code: 400; Error Code: AccessDeniedException; Request ID: ed60ddfa-8263-486a-b9f6-1dd57cbfd9bd; Proxy: null)
The following policies have been attached with the role:
Any help would be highly appreciated.
Just to add some clarity on this, you need to add "AWSLakeFormationDataAdmin" policy to the IAM role that you are using to run your Glue job.
Also, on Lake Formation side, you need to make sure the above principle (IAM role) has data lake permission to access the Glue metadata tables of the data catalog.
You also need to provide full LakeFormation access to your job role, since it seems you have LakeFormation active.
All my calls to spark.sql("") fails with the error in the stacktrace (1) below
Update - 2
I have zeroed in on the problem, it is AccessDenied for sts:AssumeRule, any leads appreciated
User: arn:aws:sts::00000000000:assumed-role/EMR_EC2_XXXXX_XXXXXX_POLICY/i-3232131232131232 is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::00000000000:role/EMR_XXXXXX_XXXXXX_POLICY
When the same location is accessed with
spark.read.parquet("s3a://xxx.xxx-xxx-xx.xxxxx-xxxxx/xxx/")
I was able to read the records.
But the same stacktrace (1) resurfaces when access with s3: instead of s3a: scheme
spark.read.parquet("s3://xxx.xxx-xxx-xx.xxxxx-xxxxx/xxx/")
So how can I configure Spark on EMR to use s3a: or have s3: running without the access denied which is presume because it may not be using the appropriate credential chain
(1)
Caused by: com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.securitytoken.model.AWSSecurityTokenServiceException: Access denied (Service: AWSSecurityTokenService; Status Code: 403; Error Code: AccessDenied; Request ID: xxxxx-xxxx-xxxx-xxxx-xxxxxxxx)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1658)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1322)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1072)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:745)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:719)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:701)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:669)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:651)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:515)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.doInvoke(AWSSecurityTokenServiceClient.java:1369)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1338)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.invoke(AWSSecurityTokenServiceClient.java:1327)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.executeAssumeRole(AWSSecurityTokenServiceClient.java:488)
at com.amazon.ws.emr.hadoop.fs.shaded.com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.assumeRole(AWSSecurityTokenServiceClient.java:460)
Update - 1
Tried setting secret and access key doesn't work
spark.sparkContext.hadoopConfiguration.set("fs.s3.awsAccessKeyId", "")
spark.sparkContext.hadoopConfiguration.set("fs.s3.awsSecretAccessKey", "")
this stack trace says "amazon EMR S3 client"; not the Apache ASF one, so different settings, and error messages.
That error message about "assumed role" hints that you are running in an EC2 VM (yes?), and that "assumed role" is actually the IAM role the EC2 VM is deployed as. In which case (a) no other credentials are being picked up and (b) that VM doesn't have permissions to access the role. Fixes: work out the setting to get the credentials in, increase EC2 IAM role rights, or create VMs with a different role
I've been trying to use Codestar on AWS Ruby on Rails using Elastic Beanstalk. I tried applying and assigning auto scaling full access policies to no avail. Can anybody help me navigate around these errors? Also using root account. Tried assigning this under roles but no success.
Error messages:
Creating Auto Scaling group failed Reason: API: autoscaling:CreateAutoScalingGroup The default Service-Linked Role for Auto Scaling could not be created. com.amazonaws.services.identitymanagement.model.AmazonIdentityManagementException:
User: arn:aws:sts::**********:assumed-role/CodeStarWorker-phcnetworks-net-CloudFormation/AWSCloudFormation is not authorized to perform: iam:CreateServiceLinkedRole on resource: arn:aws:iam::**************:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling
(Service: AmazonIdentityManagement; Status Code: 403; Error Code: AccessDenied; Request ID: *******-******-*******)
Stack named 'awseb-e-*********-stack' aborted operation. Current state: 'CREATE_FAILED' Reason: The following resource(s) failed to create: [AWSEBAutoScalingGroup].
The reported error says that the role "CodeStarWorker-phcnetworks-net-CloudFormation" is not authorized to perform operation iam:CreateServiceLinkedRole on the set of resources defined in IAM policy.So action "CreateServiceLinkedRole" needs to be added to the role for your autoscaling policies to succeed.
This is error:
ActivityFailed:AmazonServiceException:AmazonElasticMapReduce:AccessDeniedException
User: arn:aws:iam::833376745199:user/data_analytics is not authorized to perform: elasticmapreduce:DescribeCluster (Service: AmazonElasticMapReduce; Status Code: 400; Error Code: AccessDeniedException; Request ID: 593d224c-7097-11e6-a574-fd5be6acde1b)
Make sure that the IAM user that you're using to start the task runner has the elasticmapreduce:DescribeCluster permission. Unless you want to apply a more restrictive policy to your task runner, an easy way to do this would be to attach the AmazonEC2RoleForDataPipeline to your IAM user.