validation exception while featuregroup creation through sagemaker in ec2 - amazon-web-services

I am referring to document https://sagemaker-examples.readthedocs.io/en/latest/sagemaker-featurestore/sagemaker_featurestore_fraud_detection_python_sdk.html#Ingest-Data-into-FeatureStore to create featuregroup in ec2 instance.
getting below error while executing the script.
An error occurred (ValidationException) when calling the CreateFeatureGroup operation: The execution role ARN is invalid. Please ensure that the role exists and that its trust relationship policy allows the action 'sts:AssumeRole' for the service principal 'sagemaker.amazonaws.com'.
I have confusion regarding roles here.
In above script, get_execution_role() is returning ec2-role-123 which is attached to ec2 instance and has trust policy as "ec2.amazonaws.com".Also this ec2 role I am passing to other role sync-role-123 which has both "ec2.amazonaws.com", "sagemaker.amazonaws.com" as trusted entities. I have attached below permission to ec2 role
{
"Sid": "AllowIAMPassRoleAccess",
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": "arn:aws:iam::1212121:role/sync-role-123"
}
now my confusion is since we are running script inside EC2 does it uses ec2-role-123 as execution role for both ec2 and sagemaker as well
OR
ec2-role-123 execution role will be passed to sync-role-123 for sagemaker API calls?
I am not getting clear understanding on how sagemaker API works specially inside EC2?
can someone please guide.
so that I can apply the fix accordingly.

If you are using the get_execution_role() this will return the role attached to the ec2 instance and the same will be used with SageMaker API for feature store. You can fix it in 2 ways
Attach the below permissions under trust_relationship for your ec2_role_123 ( attached to your ec2 instance) {
"Effect": "Allow",
"Principal": {
"Service": "sagemaker.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
You can directly put a role arn for your sync-role-123 instead of the get_execution_role()and make sure sync-role-123 has necessary permissions.

Related

Does anyone know where this goes in the instances?

{
"Sid": "ElasticBeanstalkHealthAccess",
"Action": [
"elasticbeanstalk:PutInstanceStatistics"
],
"Effect": "Allow",
"Resource": [
"arn:aws:elasticbeanstalk:*:*:application/*",
"arn:aws:elasticbeanstalk:*:*:environment/*"
]
}
That's a part of the IAM profile for the elastic beanstalk instance.
If you choose AWSElasticBeanstalkWebTier or AWSElasticBeanstalkWorkerTier as IAM Instance profile, the ElasticBeanstalkHealthAccess permissions will be added already.
See https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/iam-instanceprofile.html
There are two IAM roles associated with an Elastic Beanstalk Environment:
Service role: used to manage the environment
Instance role: role assumed by the running application. It is used to provide access to other AWS services.
You need to find your instance role in IAM console and attach the permission that you see in the documentation. This will allow your application to send statistics.

SageMaker Studio domain creation fails due to KMS permissions

Question
Please help understand the cause and solution for the problem.
Problem
SageMaker Studio domain creation fails due to KMS permissions. The IAM Role specified to the SageMaker arn:aws:iam::316725000538:role/SageMaker has the permissions for KMS required as specified in https://docs.aws.amazon.com/sagemaker/latest/dg/api-permissions-reference.html.
Domain creation failed
Unable to create Amazon EFS for domain 'd-1dq5c9rpkswy' because you don't have permissions to use the KMS key 'arn:aws:kms:us-east-2:316725000538:key/1e2dbf9d-daa0-408d-a290-1633b615c54f'. See https://docs.aws.amazon.com/sagemaker/latest/dg/api-permissions-reference.html for required permissions for CreateDomain action.
tells the IAM permissions
IAM Permission for CreateDomain action
Amazon SageMaker API Permissions: Actions, Permissions, and Resources Reference
The IAM permission required for the CreateDomain action have been attached to the IAM role.
I had the same problem when trying to use the aws/s3 key. I created my own Customer Managed Key (CMK) and it worked just fine.
I think it's related to the AWS assigned policy on the aws/s3 key.
This part:
"Condition": {
"StringEquals": {
"kms:CallerAccount": "120455730103",
"kms:ViaService": "s3.us-east-1.amazonaws.com"
}
I don't think SageMaker meets the kms:ViaService condition.
Apart from SageMakerFullAccess we need to create a new policy and attach that to your user.
Create a new policy with below json -
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"sagemaker:CreateUserProfile",
"sagemaker:CreateModel",
"sagemaker:CreateLabelingJob",
"sagemaker:CreateFlowDefinition",
"sagemaker:CreateDomain",
"sagemaker:CreateAutoMLJob",
"sagemaker:CreateProcessingJob",
"sagemaker:CreateTrainingJob",
"sagemaker:CreateNotebookInstance",
"sagemaker:CreateCompilationJob",
"sagemaker:CreateImage",
"sagemaker:CreateMonitoringSchedule",
"sagemaker:RenderUiTemplate",
"sagemaker:UpdateImage",
"sagemaker:CreateHyperParameterTuningJob"
],
"Resource": "*"
}
]
}

Error when creating AWS Budget Actions to automatically stop EC2 and RDS

I use one EC2 instance and one RDS to run my Backend for my website. Now I saw that you can create an AWS Budget Action to automatically stop your EC2 or RDS instance. That wasn't possible before and I definitly wanna use it to be 100% sure not to get a fat bill from AWS. I already set up a billing alert, but I want the EC2 and RDS to also be automatically shut down, just to be really safe.
So when creating a new budget in AWS Budgets you can add a "Budget action", which requires you to attach an IAM role to AWS Budgets. I tried that with an existing EC2Role of mine, which has "AmazonEC2FullAccess" Policy attached to it. But this doesn't seem to be the right policy:
Just to clarify, after clicking Edit trust relationship, insert this policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
},
{
"Effect": "Allow",
"Principal": {
"Service": "budgets.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
Ok I just figured it out after posting the question, the Role also needed a Trust relationship to AWS Budgets, which had to be attached to the Role in IAM:
While creating the role, the first step is to select the trusted entity, where you need to select Budgets
And then select specific permissions related to EC2/RDS and create the role.
NOTE: you will have to create a new role to add this privilege for IAM in the budget.

How to create an IAM role of specific type using boto3?

I'm trying to lock down a user to a specific VPC in AWS and following How to Help Lock Down a User’s Amazon EC2 Capabilities to a Single VPC | AWS Security Blog.
It is mentioned that we need to create an IAM role with name VPCLockDown of type AWS Service
and add the services for which the role needs access to. like ec2, lambda etc.
I was trying to create this role programatically using boto3.
I checked the create_role documentation for creating a role using boto3.
However, they haven't mentioned anything to specify the type of role and the services that I can specify that the role should have access to.
Is there any way to specify these items while creation of the IAM role using boto3
Edit1:
I tried creating a service_linked_role as per Sudarshan Rampuria's answer like
response = iam.create_service_linked_role(
AWSServiceName='ec2.amazonaws.com',
)
But getting the following error:
An error occurred (AccessDenied) when calling the
CreateServiceLinkedRole operation: Cannot find Service Linked Role
template for ec2.amazonaws.com
You can use create_service_linked_role() function boto3 to link a role to a service.
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/iam.html#IAM.Client.create_service_linked_role
Here is a policy that allows a specific IAM User to launch an instance (RunInstances), but only in a given VPC:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EC2RunInstancesVPC",
"Effect": "Allow",
"Action": "ec2:RunInstances",
"Resource": "arn:aws:ec2:ap-southeast-2:111111111111:subnet/*",
"Condition": {
"StringEquals": {
"ec2:vpc": "arn:aws:ec2:ap-southeast-2:111111111111:vpc/vpc-abcd1234" <--- Change this
}
}
},
{
"Sid": "RemainingRunInstancePermissions",
"Effect": "Allow",
"Action": "ec2:RunInstances",
"Resource": [
"arn:aws:ec2:ap-southeast-2:111111111111:instance/*",
"arn:aws:ec2:ap-southeast-2:111111111111:volume/*",
"arn:aws:ec2:ap-southeast-2::image/*",
"arn:aws:ec2:ap-southeast-2::snapshot/*",
"arn:aws:ec2:ap-southeast-2:111111111111:network-interface/*",
"arn:aws:ec2:ap-southeast-2:111111111111:key-pair/*",
"arn:aws:ec2:ap-southeast-2:111111111111:security-group/*"
]
}
]
}
You might need to change the Region. (I tested it in the Sydney region.)
For anyone trying to do this for Lambda, we get the similar error mentioned by the question author under "Edit". Lambda doesn't have a service linked role. You can see from the AWS Lambda documentation that "create-role" is used for creating lambda execution role.
You can also see here that only Lambda#Edge has service linked role.
One just needs to use use boto3 create-role with a policy document
response = iam_client.create_role(
RoleName="some-role-name",
AssumeRolePolicyDocument='{"Version": "2012-10-17","Statement": [{ "Effect": "Allow", "Principal": {"Service": "lambda.amazonaws.com"}, "Action": "sts:AssumeRole"}]}',
Description='Lambda role'
)

AWS EMR Cluster fails to launch

I am trying to launch an AWS EMR Cluster from the AWS Console, and am getting the following error:
Failed to provision ec2 instances because 'IAM Instance Profile "arn:aws:iam::553706642095:instance-profile/EMR_EC2_DefaultRole" has no associated IAM Roles
Any one know what this means and how to resolve it?
The following is the role policy:
{
"Statement": [
{
"Action": [
"cloudwatch:*",
"dynamodb:*",
"ec2:Describe*",
"elasticmapreduce:Describe*",
"rds:Describe*",
"s3:*",
"sdb:*",
"sns:*",
"sqs:*"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
Its trust policy document is:
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
I finally resolved this issue. This was confusing because the instance-profile and the role use the same name by default. Full steps outline below, but you may be able to skip various steps.
Create default roles (if error, downgrade to awscli version 1.10.30)
aws emr create-default-roles
Create instance profile if it doesn't already exist:
aws iam create-instance-profile --instance-profile-name EMR_EC2_DefaultRole
Verify that instance profile exists but doesn't have any roles:
aws iam get-instance-profile --instance-profile-name EMR_EC2_DefaultRole
Add the role using:
aws iam add-role-to-instance-profile --instance-profile-name EMR_EC2_DefaultRole --role-name EMR_EC2_DefaultRole
You have only readonly permission for EMR
"elasticmapreduce:Describe*",
You need to give full access to elastic map reduce so that you can launch cluster/terminate
once you give this access role policy will look like
"elasticmapreduce:*",
I tried around and could get it to work without the tool using my own Cloudformation stack.
The key you have to have a InstanceProfile for the flow role and both flow and service role have to be provided as ARN.
That's how I got it to work for me!
Hope that helps someone else as well.
I got the same issue. Instead of giving new cluster name, i just kept the same default cluster name 'My Cluster' and clicked on 'Create cluster' again. It created without this error.