What are the correct policies for EC2 Data Pipeline Default Resource Role. I am creating a datapipeline and struggling to give proper permissions to this role.
This is the error I get from the Datapipeline --> WARNING: Error occurred while validating resourceRole 'MyDatapipelineEC2Role'. Need iam:ListRolePolicies and iam:GetRolePolicy to validate. Error: User: arn:aws:sts::*********:assumed-role/MyDataPipelineRole/EDPSession is not authorized to perform: iam:ListRolePolicies on resource: role MyDatapipelineEC2Role because no identity-based policy allows the iam:ListRolePolicies action (Service: AmazonIdentityManagement; Status Code: 403; Error Code: AccessDenied; Request ID: f6566144-d887-4804-a041-cccc437933e2; Proxy: null)
Related
I am trying to use glueContext.purge_table function in my aws glue job. Whenever the job is executed it throws the following error:
An error occurred while calling o82.purgeTable.
: java.lang.RuntimeException: class com.amazonaws.services.gluejobexecutor.model.AccessDeniedException:User: arn:aws:sts::012345678:assumed-role/XYZ/GlueJobRunnerSession is not authorized to perform: lakeformation:GetDataAccess on resource: arn:aws:glue:us-east-1:MICHIGAN_DEFAULT_CATALOG_ID_RANDOMIZED:table/database/table (Service: AWSLakeFormation; Status Code: 400; Error Code: AccessDeniedException; Request ID: 25829fe6-2a10-430a-b050-023c13bcc8ce; Proxy: null) (Service: AWSGlueJobExecutor; Status Code: 400; Error Code: AccessDeniedException; Request ID: ed60ddfa-8263-486a-b9f6-1dd57cbfd9bd; Proxy: null)
The following policies have been attached with the role:
Any help would be highly appreciated.
Just to add some clarity on this, you need to add "AWSLakeFormationDataAdmin" policy to the IAM role that you are using to run your Glue job.
Also, on Lake Formation side, you need to make sure the above principle (IAM role) has data lake permission to access the Glue metadata tables of the data catalog.
You also need to provide full LakeFormation access to your job role, since it seems you have LakeFormation active.
I am creating an Aws Emr cluster with AWS Java SDK. Below is the code snippet.
JobFlowInstancesConfig jobFlowInstanceConfig = new JobFlowInstancesConfig()
.withEc2SubnetId(config.getEc2SubnetId())
.withEc2KeyName(config.getEc2KeyName())
.withInstanceCount(config.getInstanceCount())
.withKeepJobFlowAliveWhenNoSteps(true)
.withMasterInstanceType(config.getMasterInstanceType())
.withSlaveInstanceType(config.getSlaveInstanceType());
RunJobFlowRequest request = new RunJobFlowRequest()
.withName(clusterName)
.withReleaseLabel(config.getReleaseLabel())
.withApplications(applications)
.withLogUri(config.getLogUri())
.withServiceRole(config.getServiceRole())
.withJobFlowRole(config.getJobFlowRole())
.withInstances(jobFlowInstanceConfig);
RunJobFlowResult runJobFlowResult = emrClient.runJobFlow(request);
As you can see I am setting "JobFlowRole" using .withJobFlowRole(config.getJobFlowRole()), but it is taking default values which does not have permission to create cluster.
I am getting following error:
com.amazonaws.services.elasticmapreduce.model.AmazonElasticMapReduceException: User: arn:aws:sts::6...0:assumed-role/default-role/i-0...4 is not authorized to perform: iam:PassRole on resource: arn:aws:iam::6...0:role/EMR_DefaultRole (Service: AmazonElasticMapReduce; Status Code: 400; Error Code: AccessDeniedException; Request ID: a...f)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1701)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1356)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1102)
Help please.
The JobFlowRole is the role of EMR service and this is not the role for creation EMR. See documentation.
You should have the right permission to create an EMR where you used to get the AWS credentials. The iam:PassRole is missing for your credentials.
I've been trying to use Codestar on AWS Ruby on Rails using Elastic Beanstalk. I tried applying and assigning auto scaling full access policies to no avail. Can anybody help me navigate around these errors? Also using root account. Tried assigning this under roles but no success.
Error messages:
Creating Auto Scaling group failed Reason: API: autoscaling:CreateAutoScalingGroup The default Service-Linked Role for Auto Scaling could not be created. com.amazonaws.services.identitymanagement.model.AmazonIdentityManagementException:
User: arn:aws:sts::**********:assumed-role/CodeStarWorker-phcnetworks-net-CloudFormation/AWSCloudFormation is not authorized to perform: iam:CreateServiceLinkedRole on resource: arn:aws:iam::**************:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling
(Service: AmazonIdentityManagement; Status Code: 403; Error Code: AccessDenied; Request ID: *******-******-*******)
Stack named 'awseb-e-*********-stack' aborted operation. Current state: 'CREATE_FAILED' Reason: The following resource(s) failed to create: [AWSEBAutoScalingGroup].
The reported error says that the role "CodeStarWorker-phcnetworks-net-CloudFormation" is not authorized to perform operation iam:CreateServiceLinkedRole on the set of resources defined in IAM policy.So action "CreateServiceLinkedRole" needs to be added to the role for your autoscaling policies to succeed.
user/abcd is not authorized to perform: iam:CreateRole on resource: arn:aws:iam::453006003233:role/Elastic_Transcoder_Default_Role (Service: AmazonIdentityManagement; Status Code: 403; Error Code: AccessDenied;
We need to attach the policy ElasticTranscoderDefaultRole with the particular group to which the IAM is added.
This is error:
ActivityFailed:AmazonServiceException:AmazonElasticMapReduce:AccessDeniedException
User: arn:aws:iam::833376745199:user/data_analytics is not authorized to perform: elasticmapreduce:DescribeCluster (Service: AmazonElasticMapReduce; Status Code: 400; Error Code: AccessDeniedException; Request ID: 593d224c-7097-11e6-a574-fd5be6acde1b)
Make sure that the IAM user that you're using to start the task runner has the elasticmapreduce:DescribeCluster permission. Unless you want to apply a more restrictive policy to your task runner, an easy way to do this would be to attach the AmazonEC2RoleForDataPipeline to your IAM user.