I am trying to launch an AWS EMR Cluster from the AWS Console, and am getting the following error:
Failed to provision ec2 instances because 'IAM Instance Profile "arn:aws:iam::553706642095:instance-profile/EMR_EC2_DefaultRole" has no associated IAM Roles
Any one know what this means and how to resolve it?
The following is the role policy:
{
"Statement": [
{
"Action": [
"cloudwatch:*",
"dynamodb:*",
"ec2:Describe*",
"elasticmapreduce:Describe*",
"rds:Describe*",
"s3:*",
"sdb:*",
"sns:*",
"sqs:*"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
Its trust policy document is:
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
I finally resolved this issue. This was confusing because the instance-profile and the role use the same name by default. Full steps outline below, but you may be able to skip various steps.
Create default roles (if error, downgrade to awscli version 1.10.30)
aws emr create-default-roles
Create instance profile if it doesn't already exist:
aws iam create-instance-profile --instance-profile-name EMR_EC2_DefaultRole
Verify that instance profile exists but doesn't have any roles:
aws iam get-instance-profile --instance-profile-name EMR_EC2_DefaultRole
Add the role using:
aws iam add-role-to-instance-profile --instance-profile-name EMR_EC2_DefaultRole --role-name EMR_EC2_DefaultRole
You have only readonly permission for EMR
"elasticmapreduce:Describe*",
You need to give full access to elastic map reduce so that you can launch cluster/terminate
once you give this access role policy will look like
"elasticmapreduce:*",
I tried around and could get it to work without the tool using my own Cloudformation stack.
The key you have to have a InstanceProfile for the flow role and both flow and service role have to be provided as ARN.
That's how I got it to work for me!
Hope that helps someone else as well.
I got the same issue. Instead of giving new cluster name, i just kept the same default cluster name 'My Cluster' and clicked on 'Create cluster' again. It created without this error.
Related
I am referring to document https://sagemaker-examples.readthedocs.io/en/latest/sagemaker-featurestore/sagemaker_featurestore_fraud_detection_python_sdk.html#Ingest-Data-into-FeatureStore to create featuregroup in ec2 instance.
getting below error while executing the script.
An error occurred (ValidationException) when calling the CreateFeatureGroup operation: The execution role ARN is invalid. Please ensure that the role exists and that its trust relationship policy allows the action 'sts:AssumeRole' for the service principal 'sagemaker.amazonaws.com'.
I have confusion regarding roles here.
In above script, get_execution_role() is returning ec2-role-123 which is attached to ec2 instance and has trust policy as "ec2.amazonaws.com".Also this ec2 role I am passing to other role sync-role-123 which has both "ec2.amazonaws.com", "sagemaker.amazonaws.com" as trusted entities. I have attached below permission to ec2 role
{
"Sid": "AllowIAMPassRoleAccess",
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": "arn:aws:iam::1212121:role/sync-role-123"
}
now my confusion is since we are running script inside EC2 does it uses ec2-role-123 as execution role for both ec2 and sagemaker as well
OR
ec2-role-123 execution role will be passed to sync-role-123 for sagemaker API calls?
I am not getting clear understanding on how sagemaker API works specially inside EC2?
can someone please guide.
so that I can apply the fix accordingly.
If you are using the get_execution_role() this will return the role attached to the ec2 instance and the same will be used with SageMaker API for feature store. You can fix it in 2 ways
Attach the below permissions under trust_relationship for your ec2_role_123 ( attached to your ec2 instance) {
"Effect": "Allow",
"Principal": {
"Service": "sagemaker.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
You can directly put a role arn for your sync-role-123 instead of the get_execution_role()and make sure sync-role-123 has necessary permissions.
I am trying to set up CI/CD with AWS + EC2 and am stuck when creating Deployment Group. The role of CodeDeploy has policies AWSCodeDeployRole and AWSCodeDeployRoleForECS but it throws an error. I tried giving it Admin rights but it is still not enough. Am I missing something? Thanks for any help!
You have a role that has the permissions required for the codedeploy to perform the deployment. What you are missing here is, You should have a trust policy defined in the role that allows codedeploy to assume the role.
Goto IAM console and select the role from the roles section
Click Trust relationships
Click Edit trust Relationships
Add the following trust policy to allow code deploy service to assume this role.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": [
"codedeploy.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}
Reference: Create a service role for CodeDeploy
I'm currently spinning in circles trying to restore from an AWS Backup and am running into permissions errors. I have administrator access to my AWS account. I've tried creating a new policy and attach it to my user account in IAM as follows:
The issue I can't seem to get around is that I need to add the permission iam:PassRole but I can't seem to find it anywhere within the AWS portal. How can I add this permission to my policy?!
EDIT: I've created a policy with all backup permissions allowed and including iam:PassRole however I am still receiving the error message You are not authorized to perform this operation. when trying to perform the backup. The policy I've created and attached to my user looks as follows:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"backup:*",
"iam:PassRole",
"iam:GetRole"
],
"Resource": "*"
}
]
}
“To successfully do a restore with the original instance profile, you will need to make changes to the restore policy. If you apply instance profile during the restore, you must update the operator role and add PassRole permissions of the underlying instance profile role to EC2. Otherwise, Amazon EC2 won’t be able to authorize the instance launch and it will fail.”
Here is the policy you can attach to the AWS default Backup role “AWSBackupDefaultServiceRole” to work around this issue:
{
"Version": "2012–10–17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "arn:aws:iam::<Account-ID>:role/*"
}]}
Source: https://medium.com/contino-engineering/new-aws-backup-features-for-quick-and-easy-ec2-instance-recovery-c8887365ca6a
I'm trying to lock down a user to a specific VPC in AWS and following How to Help Lock Down a User’s Amazon EC2 Capabilities to a Single VPC | AWS Security Blog.
It is mentioned that we need to create an IAM role with name VPCLockDown of type AWS Service
and add the services for which the role needs access to. like ec2, lambda etc.
I was trying to create this role programatically using boto3.
I checked the create_role documentation for creating a role using boto3.
However, they haven't mentioned anything to specify the type of role and the services that I can specify that the role should have access to.
Is there any way to specify these items while creation of the IAM role using boto3
Edit1:
I tried creating a service_linked_role as per Sudarshan Rampuria's answer like
response = iam.create_service_linked_role(
AWSServiceName='ec2.amazonaws.com',
)
But getting the following error:
An error occurred (AccessDenied) when calling the
CreateServiceLinkedRole operation: Cannot find Service Linked Role
template for ec2.amazonaws.com
You can use create_service_linked_role() function boto3 to link a role to a service.
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/iam.html#IAM.Client.create_service_linked_role
Here is a policy that allows a specific IAM User to launch an instance (RunInstances), but only in a given VPC:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EC2RunInstancesVPC",
"Effect": "Allow",
"Action": "ec2:RunInstances",
"Resource": "arn:aws:ec2:ap-southeast-2:111111111111:subnet/*",
"Condition": {
"StringEquals": {
"ec2:vpc": "arn:aws:ec2:ap-southeast-2:111111111111:vpc/vpc-abcd1234" <--- Change this
}
}
},
{
"Sid": "RemainingRunInstancePermissions",
"Effect": "Allow",
"Action": "ec2:RunInstances",
"Resource": [
"arn:aws:ec2:ap-southeast-2:111111111111:instance/*",
"arn:aws:ec2:ap-southeast-2:111111111111:volume/*",
"arn:aws:ec2:ap-southeast-2::image/*",
"arn:aws:ec2:ap-southeast-2::snapshot/*",
"arn:aws:ec2:ap-southeast-2:111111111111:network-interface/*",
"arn:aws:ec2:ap-southeast-2:111111111111:key-pair/*",
"arn:aws:ec2:ap-southeast-2:111111111111:security-group/*"
]
}
]
}
You might need to change the Region. (I tested it in the Sydney region.)
For anyone trying to do this for Lambda, we get the similar error mentioned by the question author under "Edit". Lambda doesn't have a service linked role. You can see from the AWS Lambda documentation that "create-role" is used for creating lambda execution role.
You can also see here that only Lambda#Edge has service linked role.
One just needs to use use boto3 create-role with a policy document
response = iam_client.create_role(
RoleName="some-role-name",
AssumeRolePolicyDocument='{"Version": "2012-10-17","Statement": [{ "Effect": "Allow", "Principal": {"Service": "lambda.amazonaws.com"}, "Action": "sts:AssumeRole"}]}',
Description='Lambda role'
)
I am trying to download AWS Codedeploy Agent file in my Amazon Linux. I followed instructions as mentioned in http://docs.aws.amazon.com/codedeploy/latest/userguide/how-to-run-agent.html, for Amazon Linux, have created appropriate instance profile, service role etc. Everything is latest (Amazon Linux, CLI Packages, it is a brand new instance and I have tried this with at least 3 more brand new instances with same result). All instances have full outbound internet access.
But following statement for downloading install from S3 always fails,
aws s3 cp s3://aws-codedeploy-us-east-1/latest/install . --region us-east-1
With Error,
A client error (403) occurred when calling the HeadObject operation: Forbidden
Completed 1 part(s) with ... file(s) remaining
Can anyone help me with this error?
I figured out the problem, According to Codedeploy documentation for IAM Instance profile
http://docs.aws.amazon.com/codedeploy/latest/userguide/how-to-create-iam-instance-profile.html
following permissions needs to be given to your IAM instance profile.
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:Get*",
"s3:List*"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
But I limited the resource to my code bucket since I don't want my instances to access other buckets directly. But turns out I also need to give additional permission for aws-codedeploy-us-east-1/* s3 resource for being able to download the agent. This is not very clear in the document for setting up IAM instance profile for Codedeploy.
More restrictive policy that works:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:Get*",
"s3:List*"
],
"Resource": [
"arn:aws:s3:::aws-codedeploy-us-east-1/*",
"arn:aws:s3:::aws-codedeploy-us-west-1/*",
"arn:aws:s3:::aws-codedeploy-us-west-2/*",
"arn:aws:s3:::aws-codedeploy-ap-south-1/*",
"arn:aws:s3:::aws-codedeploy-ap-northeast-2/*",
"arn:aws:s3:::aws-codedeploy-ap-southeast-1/*",
"arn:aws:s3:::aws-codedeploy-ap-southeast-2/*",
"arn:aws:s3:::aws-codedeploy-ap-northeast-1/*",
"arn:aws:s3:::aws-codedeploy-eu-central-1/*",
"arn:aws:s3:::aws-codedeploy-eu-west-1/*",
"arn:aws:s3:::aws-codedeploy-sa-east-1/*"
]
}
]
}