While creating AWS EMR cluster, always i get the issue- Service role EMR_DefaultRole has insufficient EC2 permissions
And the cluster terminates automatically, have even done steps as per aws documentation of recreating emr specific roles, but no progress please guide how to resolve the issue- Service role EMR_DefaultRole has insufficient EC2 permissions
EMR needs two roles to start the cluster 1) EC2 Instance profile role 2)EMR Service role. The service role should have enough permissions to provision new resources to start the cluster, EC2 instances, their network etc. There could be many reasons for this common error:
Verify the resources and their actions. Refer https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-iam-role.html.
Check if you are passing the tag that signifies if cluster needs to use emr managed policy.
{
"Key": "for-use-with-amazon-emr-managed-policies",
"Value": "true"
}
At last try to find out the exact reason from cloud trail. Go to aws>cloud trail. From the event history configuration enable the error code so that you can see the exact error. If you find the error code something like 'You are not authorized to perform this operation. Encoded authorization failure message'. Then open the event history details, pick up the encrypted error message and decrypt using aws cli
aws sts decode-authorization-message message. This will show you the complete role details, event, resources, action. Compare it with AWS IAM permissions and you can find out the missing permission or parameter that you need to pass while creating the job flow.
Related
I'm working on an AWS account managed by another team which use it only for S3 storage. We have authorization to use sagemaker, and administartor said “AmazonSageMakerFullAccess” have been given to me.
I'm trying to access sagemaker studio, for that I'm ask to "Setup SageMaker Domain" by aws.
I then need a "Default execution role"
If I try to create one, I got error "User ... is not authorized to perform: iam:CreateRole on resource: ..."
There is an option to use a custom exiting one with the format
"arn:aws:iam::YourAccountID:role/yourRole"
but while I have an account Id, I don't know what role to use.
I don't have permission to create role, and the ones I see in IAM service doesn't seem to be related to sagemaker (also I don't have permission to see the details of those roles).
Should the sagemaker setup be done by the administrator who can create a new role ? Or is there a way for me to do it, and if so where can I find the role I need ?
If you don't attach any role to AWS SageMaker, and when you try to create SageMaker resource the very first time it will create a default execution role for the service. Either get the permission to create a role or ask your administrator to create a execution role for your SageMaker so that next time when you create one you can use the same role.
I'm deploying a service to amazon ecs through docker. I need to assign an IAM role to the cluster that allows it to communicate with secrets manager, as that's where some of the important configuration comes from. However, when trying to select the iam role, I get this error message:
Error message
So I tried following the amazon documentation and going to the account settings tab signed in as the root account, only to find that the settings were already enabled by default. So now I'm sort of stuck as to what to do at this point. Is there something I'm missing here?
resources already enabled
From your screenshot what I understand is that the IAM role you are using to deploy the container does not have "ecsInstanceRole" in your IAM roles. You could use the following steps to create it.
Open the IAM console at https://console.aws.amazon.com/iam/
In the navigation pane, choose Roles and then choose Create role.
Choose the AWS service role type, and then choose Elastic Container Service.
Choose the EC2 Role for Elastic Container Service use case and then Next: Permissions.
In the Attached permissions policy section, select AmazonEC2ContainerServiceforEC2Role and then choose Next: Review.
Important
The AmazonEC2ContainerServiceforEC2Role managed policy should be attached to the container instance IAM role, otherwise you will receive an error using the AWS Management Console to create clusters.
For Role name, type ecsInstanceRole and optionally you can enter a description.
Review your role information and then choose Create role to finish.
or you can follow the below document:
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/instance_IAM_role.html
is there a way to find out what all resources are using an IAM role. Because I want to modify that and wanted to check what all could affect my modification.
Not quite.
Services can "assume" a role. This happens when the activity is run (eg when an AWS Lambda function is invoked, or when an Amazon EC2 instance is launched). Thus, there is no permanent 'link' between roles and services. Therefore, it is not possible to say "list me everything that is using this IAM Role".
However, you could list services and see which roles they are configured to use. For example, you could describe EC2 instances and check what IAM Role they are configured to use. However, you would need to do this for all services that you know are potentially using the roles.
You can find where an IAM role is used from based on the past usage.
I can think of few ways.
method 1 - Access Advisor
click the "Access Advisor" tab section that appears when you click an IAM role
check last accessed time of each services
method 2 - Cloudtrail
the cli command will tell you which services/user assumed the role and also the action they performed.
aws cloudtrail lookup-events --max-results 20 --lookup-attributes AttributeKey=ResourceName,AttributeValue=arn:aws:iam::012345678901:role/lambdaRole --output json --query "Events[*].[CloudTrailEvent]"
we have a policy we are attaching to roles that's ensuring the ec2 provisioner has included the required tags defined by our finance department. sample here
I can picture an engineer getting frustrated when each time he tries to spin up an EC2 instance it's immediately shut down because he forgot to include required tags and hit a DENY in the iam policy, but he has no way of knowing.
I was hoping for a custom error description return by the api. It doesn't have to be iam, if there's a benefit to instead use lambda fired off the cloudwatch runinstances event, I'm open to that as well.
What can we do to inform the engineer his instance was shutdown due to missing required tags?
Would love to hear your suggestions!
AWS offers a base set of APIs. It's impossible to provide every feature that all users want, but using the base APIs anyone can build a service on top of AWS.
For example, instead of having your developers launch instances directly through AWS, you could have them use a custom interface (perhaps a page on an Intranet) where they can request certain services. This interface can then call AWS APIs on their behalf, including required elements, such as tags. It's like storage -- people don't write directly to the disk, they do it through their operating system.
If that's too low-level for you, an alternative is to use AWS CloudFormation, which launches services based upon a template. The template can collect the required information or automatically add it to instances when they are launched.
Then, throw in AWS Service Catalog and you can force users to launch services through CloudFormation templates. Service Catalog offers a list of services (effectively just CloudFormation templates) that users can launch -- even if the users don't have permission to launch the services themselves!
For example, let's say your developers do not have permission to create a an Amazon EC2 instance. You could provide a template via Service Catalog that launches an EC2 instance on their behalf but also enforces your standards, such as tagging, subnets, security groups, etc.
Bottom line: If you don't see something in AWS that specifically meets your needs, you can often build it on top of AWS either via your own code or via AWS Service Catalog.
tl;dr The access denied message does include the condition it failed on.
The sample linked to provides an IAM policy to deny RunInstances if tags were not also included in the RunInstances api call. Resource level permissions were provided in March of 2017 allowing users to include tags in the RunInstances api call as well as allowing IAM to enforce ec2 resource level permissions, in this case, enforcing users to include required tags.
Prior to March 2017, two api calls were required to create tags:
ec2 run-instances --image-id ami-6df1e514 --count 1 --instance-type t2.micro --subnet-id subnet-e25e29bb
ec2-create-tags <instanceid> --tag "Name=<value>" --tag "App=<value>" --tag "AppOwner=<value>" --tag "Environment=<value>"
After implementing this iam policy the workflow above would DENY on step 1.
Here’s the new workflow for provisioning an EC2 instance which includes tags:
ec2 run-instances --image-id ami-6df1e514 --count 1 --instance-type t2.micro --subnet-id subnet-e25e29bb --tag-specifications 'ResourceType=instance,Tags=[{Key=name,Value=required_tag_name_value},{Key=App,Value=required_tag_app_value},{Key=AppOwner,Value=required_tag_appowner_value},{Key=Environment,Value=required_tag_env_value}]'
Based on the sample iam policy linked to, if the user does not include the required tags, the returned error message is encoded and is displayed to the user as such:
An error occurred (UnauthorizedOperation) when calling the
RunInstances operation: You are not authorized to perform this
operation. Encoded authorization failure message:
zGetZzIIedikZSAbE4YGEGhy1ytjrXD8Ak-hr1UJvDkKW7wzDu27ZS0NfMGaOUBQGO1I3b3v6Us8BXO-41973SckcmEH17019Sheua16dmrTPYHYymw9pftYope_jmR6MgsvH1bMP0FE_gHnEvaJCIMNukOo-utK....
If the user's iam policy also includes the sts:DecodeAuthorizationMessage, they can decode the message with the following:
aws sts decode-authorization-message --encoded-message <encoded message here>
{
"DecodedMessage": "{\"allowed\":false,\"explicitDeny\":true,\"matchedStatements\":{\"items\":[{\"statementId\":\"\",\"effect\":\"DENY\",\"principals\":{\"items\":[{\"value\":\"AROAJVNFHTEF6I2STOU\"}]},\"principalGroups\":{\"items\":[]},\"actions\":{\"items\":[{\"value\":\"ec2:RunInstances\"}]},\"resources\":{\"items\":[{\"value\":\"arn:aws:ec2:::instance/\"}]},\"conditions\":{\"items\":[{\"key\":\"aws:RequestTag/AppOwner\",\"values\":{\"items\":[{\"value\":\"true\"}]}}]}}]},\"failures\":{\"items\":[]},\"context\":{\"principal\":{\"id\":\"AROAJVNFHTEF6I2STOU-CLI-session-1501883988\",\"arn\":\"arn:aws:sts:::assumed-role/_test_require_tags/AWS-CLI-session-1501883988\"},\"action\":\"ec2:RunInstances\",\"resource\":\"arn:aws:ec2:us-west-2::instance/\",\"conditions\":{\"items\":[{\"key\":\"ec2:Tenancy\",\"values\":{\"items\":[{\"value\":\"default\"}]}},{\"key\":\"ec2:AvailabilityZone\",\"values\":{\"items\":[{\"value\":\"us-west-2c\"}]}},{\"key\":\"ec2:Region\",\"values\":{\"items\":[{\"value\":\"us-west-2\"}]}},{\"key\":\"ec2:ebsOptimized\",\"values\":{\"items\":[{\"value\":\"false\"}]}},{\"key\":\"ec2:InstanceType\",\"values\":{\"items\":[{\"value\":\"t2.micro\"}]}},{\"key\":\"ec2:RootDeviceType\",\"values\":{\"items\":[{\"value\":\"ebs\"}]}}]}}}"
}
While a little difficult to read, we can see which condition the RunInstance calls failed on:
aws:RequestTag/AppOwner\",\"values\":{\"items\":[{\"value\":\"true\"}]}}]}}]},\"failures\":
When I try to launch an Elastic Beanstalk environment from the command line, I get this error during the process:
Environment health has transitioned from Pending to Warning. Access
denied while accessing Auto Scaling and Elastic Load Balancing using
role "arn:aws:iam::XXXXXXXXXX:role/aws-elasticbeanstalk-service-role".
Verify the role policy.
When I launch it from the console, I do not get this warning. I have tried to replicate the same AWS console configuration from the CLI , but I still get this error.
Any clue what's going on?
If anyone is still getting a similar error and is still unsure how to solve it
Navigate to IAM in the AWS Console.
Navigate to the aws-elasticbeanstalk-service-role (under Access Management > Roles).
Attach the following policies:
AutoScalingFullAccess
ElasticLoadBalancingFullAccess
I then rebuilt the environment, no more errors!
The following article helped me understand this better: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/concepts-roles-service.html
You can go to your IAM console. You will see the "aws-elasticbeanstalk-service-role", Under the permission tab, you can grant the specific you want to give the policy on your behalf. Refresh and you should have "Ok". Make sure the permission you grant the role can manage this " Auto Scaling and Elastic Load Balancing using role"
Elastic Beanstalk now uses a Service Role to call other AWS services on your behalf. The IAM role is created in your account and you give permissions to launch
Service role is optional but recommended for new environments.
Especially for enhanced health monitoring (which is what your using based on the error message), service role is mandatory. A misconfigured service role can lead to this error message.
The console experience makes it very easy for you to create/use the role with the correct permissions. This is because you just need to select the correct role from the dropdown (if the role already exists) when using the create environment wizard.
When using the CLI you need to pass the service role option setting. (namespace: aws:elasticbeanstalk:environment, option_name: ServiceRole). You can find the required permissions for a role configured in this documentation.
There are some more details about service role in my previous stack overflow answer here.
If anyone can not solve the error whit the previous actions what I had to do is:
Add to aws-elasticbeanstalk-service-role the following policies AutoScalingFullAccess | ElasticLoadBalancingFullAccess
Not re-start but build the environment again
(after a while it kept launching the error)
Then I added:
AdministratorAccess-AWSElasticBeanstalk (again to aws-elasticbeanstalk-service-role)
Re-build the environment again
It is working at the moment.