How do I make this IAM role error in aws sagemaker go away? - amazon-web-services

I suspect this has to more to do with IAM roles than Sagemaker.
I'm following the example here
Specifically, when it makes this call
tf_estimator.fit('s3://bucket/path/to/training/data')
I get this error
ClientError: An error occurred (AccessDenied) when calling the GetRole operation: User: arn:aws:sts::013772784144:assumed-role/AmazonSageMaker-ExecutionRole-20181022T195630/SageMaker is not authorized to perform: iam:GetRole on resource: role SageMakerRole
My notebook instance has an IAM role attached to it.
That role has the AmazonSageMakerFullAccess policy. It also has a custom policy that looks like this
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::*"
]
}
]
}
My input files and .py script is in an s3 bucket with the phrase sagemaker in it.
What else am I missing?


If you're running the example code on a SageMaker notebook instance, you can use the execution_role which has the AmazonSageMakerFullAccess attached.
from sagemaker import get_execution_role
sagemaker_session = sagemaker.Session()
role = get_execution_role()
And you can pass this role when initializing tf_estimator.
You can check out the example here for using execution_role with S3 on notebook instance.


This is not an issue with S3 Bucket policy but for IAM, The user role that you're choosing has a policy attached that doesn't give it permissions to manage other IAM roles. You'll need to make sure the role you're using can manage (create, read, update) IAM roles.
Hope this helps !


Try using aws configure and make sure you are the expected user. If not, change / update your credentials.This worked for me.

Related

How can I apply my local cli after changed IAM on AWS management console

Prerequisite
I usually use aws-mfa
I tried to terraform apply
I got a problem of insufficient IAM. But it might be resolved by: User: x is not authorized to perform: (sts:DecodeAuthorizationMessage)`
The answer makes sense.
What I did
Added the IAM Permission to my policy(this attached to my group)
Checked sts get-caller-identity on my local as my IAM User.
IAM Policy Simulator told me that the IAM Group includes my IAM User is permitted to perform sts: DecodeAuthorizationMessage...
Problem
The IAM I added seems to be not applied on my local.
it still shows <my IAM USER> is not authorized to perform: sts:DecodeAuthorizationMessage with an explicit deny in an identity-based policy
Ask
Anyone can resolve the problem?
And, please let me know that the timing gap of between aws cli and aws management console. How much does it takes?
Moreover, how can I immediately apply to aws cli after IAM changed on aws management console?

Add this as an inline policy to your IAM user,
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowStsDecode",
"Effect": "Allow",
"Action": "sts:DecodeAuthorizationMessage",
"Resource": "*"
}
]
}

SageMaker Studio domain creation fails due to KMS permissions

Question
Please help understand the cause and solution for the problem.
Problem
SageMaker Studio domain creation fails due to KMS permissions. The IAM Role specified to the SageMaker arn:aws:iam::316725000538:role/SageMaker has the permissions for KMS required as specified in https://docs.aws.amazon.com/sagemaker/latest/dg/api-permissions-reference.html.
Domain creation failed
Unable to create Amazon EFS for domain 'd-1dq5c9rpkswy' because you don't have permissions to use the KMS key 'arn:aws:kms:us-east-2:316725000538:key/1e2dbf9d-daa0-408d-a290-1633b615c54f'. See https://docs.aws.amazon.com/sagemaker/latest/dg/api-permissions-reference.html for required permissions for CreateDomain action.
tells the IAM permissions
IAM Permission for CreateDomain action
Amazon SageMaker API Permissions: Actions, Permissions, and Resources Reference
The IAM permission required for the CreateDomain action have been attached to the IAM role.

I had the same problem when trying to use the aws/s3 key. I created my own Customer Managed Key (CMK) and it worked just fine.
I think it's related to the AWS assigned policy on the aws/s3 key.
This part:
"Condition": {
"StringEquals": {
"kms:CallerAccount": "120455730103",
"kms:ViaService": "s3.us-east-1.amazonaws.com"
}
I don't think SageMaker meets the kms:ViaService condition.

Apart from SageMakerFullAccess we need to create a new policy and attach that to your user.
Create a new policy with below json -
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"sagemaker:CreateUserProfile",
"sagemaker:CreateModel",
"sagemaker:CreateLabelingJob",
"sagemaker:CreateFlowDefinition",
"sagemaker:CreateDomain",
"sagemaker:CreateAutoMLJob",
"sagemaker:CreateProcessingJob",
"sagemaker:CreateTrainingJob",
"sagemaker:CreateNotebookInstance",
"sagemaker:CreateCompilationJob",
"sagemaker:CreateImage",
"sagemaker:CreateMonitoringSchedule",
"sagemaker:RenderUiTemplate",
"sagemaker:UpdateImage",
"sagemaker:CreateHyperParameterTuningJob"
],
"Resource": "*"
}
]
}

trying to give a redshift user access to an IAM role, trusted entity list was updated but still getting the same error

Error message: User "arn:aws:redshift:us-west-2:123456789012:dbuser:my-cluster/user2" is not authorized to assume IAM Role "roleArn"
on the role I've updated the trust policy to this which should allow the assume role, what am I messing up here?
"Version": "2008-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "redshift.amazonaws.com"
},
"Action": "sts:AssumeRole"
},
code is valid JSON had to cut off the rest.
I'm interning and new to IAM roles. if the redshift account also needs the permission update, how do I give it to them? I've been on this issue for a while so thanks to any help you can give.

To be able to use an IAM role with LOAD or UNLOAD operations one has to:
create an IAM role with trust relationship with Redshift service
attach the role to the cluster
You described doing the first step. Have you also attached the role? You can see the attached role in the AWS UI or list them with CLI:
aws redshift describe-clusters --cluster-identifier my-cluster --query 'Clusters[].IamRoles'
[
[
{
"IamRoleArn": "arn:aws:iam::123456789012:role/my-redhift-role",
"ApplyStatus": "in-sync"
}
]
]
Looking at the error you're getting,
Error message: User "arn:aws:redshift:us-west-2:123456789012:dbuser:my-cluster/user2" is not authorized to assume IAM Role "roleArn"
looks like in the operation you're issuing the role is wrongly configured. To me the error suggests that you're instructing Redshift to assume roleArn role, which probably does not exist. You should put your role name there.

S3 PutObject operation gives Access Denied with IAM Role containing Policy granting access to S3

I have an IAM role with a custom policy attached to it allowing access to an S3 bucket we'll call foo-bar. I've tried granting access to that specific resource, with PutObject and a couple other actions. That IAM Role is attached to an EC2 instance yet that EC2 instance does not have access to upload files when I use aws s3 sync. s3://foo-bar.
To test if it was an issue with the policy, I just granted S3:* to * resources, and it still won't upload.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"cloudformation:ListExports",
"s3:*"
],
"Resource": "*"
}
]
}
The error I get at the CLI is:
upload failed: infrastructure\vpc.template to s3://foo-bar/infrastructure/vpc.template An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
Is there something else I need to do in order to give it access? Why isn't the Policy attached to the IAM Role working?

I tried running it with --debug to see what's going on.
This helped me discover that I have a local .aws/credentials file which overrode the IAMRole attached to the machine.
If you need the credentials file - you can have a different profile [some name] and use --profile to choose it.
HTH.

AWS ECR GetAuthorizationToken

I've tried to follow AWS instructions on setting ECR authorization to my user by giving the AmazonEC2ContainerRegistryFullAccess policy to my user.
However when I try to run on my PC the aws ecr get-login I get an error that I don't have permission.
An error occurred (AccessDeniedException) when calling the GetAuthorizationToken operation: User: arn:aws:iam::ACCOUNT_NUMBER:user/MY_USER is not authorized to perform: ecr:GetAuthorizationToken on resource: *
What have I done wrong ?

You must attach a policy to your IAM role.
I attached AmazonEC2ContainerRegistryFullAccess and it worked.

Here is a full answer, after I followed all steps - I was able to use ECR
The error can have a few meanings:
You are not authorized because you do not have ECR policy attached to your user
You are not authorized because you are using 2FA and using cli is not secure unless you set a temporary session token
You provided invalid credentials
Here is a list of all steps to get access (including handling 2FA)
First of all, you have to create a policy that gives you access to GetAuthorizationToken action in ECR.
Attach this policy either to a user or a group (groups/roles are IMHO always better approach, my vote to roles, e.g. DevOps)
Make sure you have AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY set in your environment. I recommend to use aws folder with credentials and profiles separated.
If you have 2FA enabled
You need to generate session token using this command
aws sts get-session-token --serial-number arn-of-the-mfa-device --token-code code-from-token. arn-of-the-mfa-device can be found in your profile, 2FA section. Token, is generated token from the device.
Update aws credentails with received AccessKeyId, SecretAccessKey, and SessionToken. AWS recommends having either cron job to refresh token, which means if you are doing it you are testing things, your prod resources most likely do not have 2FA enabled. You can increase session by providing --duration-seconds but only up to 36 hours. A good explanation can be found at authenticate-mfa-cli
This should do the job

I ended up using AmazonEC2ContainerRegistryPowerUser as seemed a better option than Full Access. Here are the policies I found as of June 2019:

The user must have GetAuthorizationToken for all resources on ECR. To make the policy tight, you can grant all actions only to the desired registry and only the ecr:GetAuthorizationToken to all resources. Here is an example policy to attach to your user:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ecr:BatchCheckLayerAvailability",
"ecr:BatchGetImage",
"ecr:CompleteLayerUpload",
"ecr:GetDownloadUrlForLayer",
"ecr:InitiateLayerUpload",
"ecr:PutImage",
"ecr:UploadLayerPart"
],
"Effect": "Allow",
"Resource": "<REPOSITORY_ARN_HERE>"
},
{
"Action": [
"ecr:GetAuthorizationToken",
],
"Effect": "Allow",
"Resource": "*"
}
]
}

Just as it appears in the error description, I have to allow action "GetAuthorizationToken" in my policy.
{
"Sid": "VisualEditor2",
"Effect": "Allow",
"Action": "ecr:GetAuthorizationToken",
"Resource": "*"
}
Note: This is not my full policy but a subsection of Statement.

I've found out that when 2FA is enabled there is no option to use the aws ecr get-login, once I've removed the 2FA from my account I got the authorization token

This was my guy EC2InstanceProfileForImageBuilderECRContainerBuilds

I had the same problem with ECS when I tried to push my container in the repository.
To solve it, I attached to my IAM role this : AmazonECS_FullAccess

In case you are trying to pull images from a PUBLIC AWS repository, you must add the following user permissions to your policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ecr-public:GetAuthorizationToken",
"sts:GetServiceBearerToken"
],
"Resource": "*"
}
]
}
Please see the full documentation here.

I have the same problem, but I have set the permission boundary only to s3 previously that causes the issue.
Removed the permission boundary ,it worked like a charm

For me:
- Effect: Allow
Sid: VisualEditor2
Action:
- ecr:GetAuthorizationToken
- ecr:BatchGetImage
- ecr:GetDownloadUrlForLayer
Resource: "*"