AWS Sagemaker + AWS Lambda - amazon-web-services

I try to use AWS SageMaker following documentation. I successfully loaded data, trained and deployed the model.
deployed-model
My next step have to be using AWS Lambda, connect it to this SageMaker endpoint.
I saw, that I need to give Lambda IAM execution role permission to invoke a model endpoint.
I add some data to IAM policy JSON and now it has this view
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "logs:CreateLogGroup",
"Resource": "arn:aws:logs:us-east-1:<my-account>:*"
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": [
"arn:aws:logs:us-east-1:<my-account>:log-group:/aws/lambda/test-sagemaker:*"
]
},
{
"Effect": "Allow",
"Action": "sagemaker:InvokeEndpoint",
"Resource": "*"
}
]
}
Problem that even with role that have permission for invoking SageMaker endpoint my Lambda function didn't see it
An error occurred (ValidationError) when calling the InvokeEndpoint operation: Endpoint xgboost-2020-10-02-12-15-36-097 of account <my-account> not found.: ValidationError

I found an error by myself. Problem was in different regions. For training and deploying model I used us-east-2 and for lambda I used us-east-1. Just creating all in same region fixed this issue!

Related

Cross account SQS - Lambda setup throws error execution role does not have permissions to call receiveMessage on SQS

I am trying to setup cross account communication from SQS queue to Lambda function. Both these resources are on eu-central-1 region but in 2 different AWS accounts.
My setup is below
AccountA has the Lambda function
AccountB has the SQS queue
I have created IAM role on Account A and it is attached to Lambda function (AccountA_LAMBDA_EXECUTION_ROLE). IAM role has following permissions
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"sqs:DeleteMessage",
"sqs:GetQueueUrl",
"sqs:ListDeadLetterSourceQueues",
"sqs:ChangeMessageVisibility",
"sqs:PurgeQueue",
"sqs:ReceiveMessage",
"sqs:DeleteQueue",
"sqs:SendMessage",
"sqs:GetQueueAttributes",
"sqs:ListQueueTags",
"sqs:CreateQueue",
"sqs:SetQueueAttributes"
],
"Resource": "<AccountB_SQS_QUEUE_ARN>"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "sqs:ListQueues",
"Resource": "*"
}
]
}
SQS Queue has following access policy
{
"Version": "2012-10-17",
"Id": "Queue1_Policy_UUID",
"Statement": [
{
"Sid": "Queue1_AllActions",
"Effect": "Allow",
"Principal": {
"AWS": "<AccountA_LAMBDA_EXECUTION_ROLE>"
},
"Action": "sqs:*",
"Resource": "<AccountB_SQS_QUEUE_ARN>"
}
]
}
I am using AWS CLI to add Lambda trigger, so that AccountB_SQS_QUEUE can be added as a trigger to AccountA_LAMBDA_FUNCTION. Following is the AWS CLI command
aws lambda create-event-source-mapping --function-name AccountA_LAMBDA_FUNCTION_NAME --event-source-arn AccountB_SQS_QUEUE_ARN --profile AccountA_PROFILE --region eu-central-1
But this command failed with an error
An error occurred (InvalidParameterValueException) when calling the CreateEventSourceMapping operation: The provided execution role does not have permissions to call ReceiveMessage on SQS
I am following this tutorial from AWS, but it is not success. What went wrong here

AWS Tutorial Code - Lambda Access Denied, incorrect server location showing in error message

I am very new to AWS and have only just started learning it. I am following AWS's full-stack tutorial, however, when I test module 4, my lambda function is not authorized to perform dynamodb:PutItem. In the error message, I can see the ARN has us-east-1 in it, however, the ARN I passed into the JSON for the IAM policy is eu-west-2. I have set everything up on eu-west-2 servers.
Here is the JSON used in the IAM policy, I have replaced my ID with xxxxx, but it is the same as what's listed in the table details on the DynamoDB dashboard.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"dynamodb:PutItem",
"dynamodb:DeleteItem",
"dynamodb:GetItem",
"dynamodb:Scan",
"dynamodb:Query",
"dynamodb:UpdateItem"
],
"Resource": "arn:aws:dynamodb:eu-west-2:xxxxxxxxx:table/HelloWorldDatabase/*"
}
]
}
Is there anything I should be checking elsewhere they could be wrong?
EDIT:
Having changed some JSON from comments, JSON now looks like this:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ListAndDescribe",
"Effect": "Allow",
"Action": [
"dynamodb:List*",
"dynamodb:DescribeReservedCapacity*",
"dynamodb:DescribeLimits",
"dynamodb:DescribeTimeToLive"
],
"Resource": "*"
},
{
"Sid": "SpecificTable",
"Effect": "Allow",
"Action": [
"dynamodb:BatchGet*",
"dynamodb:DescribeStream",
"dynamodb:DescribeTable",
"dynamodb:Get*",
"dynamodb:Query",
"dynamodb:Scan",
"dynamodb:BatchWrite*",
"dynamodb:CreateTable",
"dynamodb:Delete*",
"dynamodb:Update*",
"dynamodb:PutItem"
],
"Resource": "arn:aws:dynamodb:*:*:table/HelloWorldDatabase"
}
]
}
This is the full stack trace I am now getting:
Requested resource not found (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ResourceNotFoundException; Request ID: S62KLPBAGKNLA66SSI77RC1AC7VV4KQNSO5AEMVJF66Q9ASUAAJG)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1799)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1383)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1359)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1139)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:796)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:764)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:738)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:698)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:680)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:544)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:524)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:5110)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:5077)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.executePutItem(AmazonDynamoDBClient.java:2721)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.putItem(AmazonDynamoDBClient.java:2687)
at com.amazonaws.services.dynamodbv2.document.internal.PutItemImpl.doPutItem(PutItemImpl.java:85)
at com.amazonaws.services.dynamodbv2.document.internal.PutItemImpl.putItem(PutItemImpl.java:63)
at com.amazonaws.services.dynamodbv2.document.Table.putItem(Table.java:168)
at com.example.app.SavePersonHandler.persistData(SavePersonHandler.java:38)
at com.example.app.SavePersonHandler.handleRequest(SavePersonHandler.java:27)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
From DynamoDB this is the table details:
Region EU (London)
Amazon Resource Name (ARN) arn:aws:dynamodb:eu-west-2:xxxxxxxxx:table/HelloWorldDatabase
The Problem is the region name 3rd module step named Create a WebApp With Amplify Console
quoting from the above step:
In a new browser window, log into the Amplify Console. NOTE: We will be using the Oregon (us-west-2) region for this tutorial.
Please use the Amazon DynamoDB: Allows access to a specific table
Below policy shows how you might create a policy that allows full access to the HelloWorldDatabase DynamoDB table. This policy grants the permissions necessary to complete this action from the AWS API or AWS CLI only.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ListAndDescribe",
"Effect": "Allow",
"Action": [
"dynamodb:List*",
"dynamodb:DescribeReservedCapacity*",
"dynamodb:DescribeLimits",
"dynamodb:DescribeTimeToLive"
],
"Resource": "*"
},
{
"Sid": "SpecificTable",
"Effect": "Allow",
"Action": [
"dynamodb:BatchGet*",
"dynamodb:DescribeStream",
"dynamodb:DescribeTable",
"dynamodb:Get*",
"dynamodb:Query",
"dynamodb:Scan",
"dynamodb:BatchWrite*",
"dynamodb:CreateTable",
"dynamodb:Delete*",
"dynamodb:Update*",
"dynamodb:PutItem"
],
"Resource": "arn:aws:dynamodb:eu-west-2:xxxxxx:table/HelloWorldDatabase"
}
]
}
If you want to learn how to build Lambda functions that interact with AWS Services, such as Amazon DynamoDB, you can use the Lambda Java runtime API. This gives you full control exactly what you want the Lambda function to perform.
To interact with the AWS Services, you have to use an IAM role (as discussed in this tutorial). For example, to use DynamoDB, the IAM role has to have a policy that lets it use Amazon DynamoDB.
All of these concepts are covered in this API development tutorial. In addition, this tutorial shows you how to schedule the Lambda function using scheduled events:
Creating scheduled events to invoke Lambda functions
I met the same issue. To solve the issue, please use only usa-east-1 server along the way when doing the tutorial. The jar file seems to hard-code the server address.

AWS CloudWatch and EC2 Role + Policy

I've followed a great tutorial by Martin Thwaites outlining the process of logging to AWS CloudWatch using Serilog and .Net Core.
I've got the logging portion working well to text and console, but just can't figure out the best way to authenticate to AWS CloudWatch from my application. He talks about inbuilt AWS authentication by setting up an IAM policy which is great and supplies the JSON to do so but I feel like something is missing. I've created the IAM Policy as per the example with a LogGroup matching my appsettings.json, but nothing comes though on the CloudWatch screen.
My application is hosted on an EC2 instance. Are there more straight forward ways to authenticate, and/or is there a step missing where the EC2 and CloudWatch services are "joined" together?
More Info:
Policy EC2CloudWatch attached to role EC2Role.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
ALL EC2 READ ACTIONS HERE
],
"Resource": "*"
},
{
"Sid": "LogStreams",
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:DescribeLogStreams",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:*:*:log-group:cloudwatch-analytics-staging
:log-stream:*"
},
{
"Sid": "LogGroups",
"Effect": "Allow",
"Action": [
"logs:DescribeLogGroups"
],
"Resource": "arn:aws:logs:*:*:log-group:cloudwatch-analytics-staging"
}
]
}
In order to effectively apply the permissions, you need to assign the role to the EC2 instance.

Unable to add trigger to AWS Lambda

I am trying to add SQS as a trigger to my Lambda function running in AWS-VPC but it throws error as :
An error occurred when creating the trigger: The provided execution role does not have permissions to call ReceiveMessage on SQS (Service: AWSLambda; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: d34b7525-5c69-4434-a015-112e8e74f447; Proxy: null)
Tried via adding AWSLambdaVPCAccessExecutionRole to the policy for
the role as well via IAM. But no luck!
I am unable to figure where I am making a mistake? Please help me out, if anyone had similar experience in past or knows how to resolve it. Thanks you in advance!
Please attach managed policy AWSLambdaSQSQueueExecutionRole in your lambda execution role.
If your lambda function is working with any other aws services, you can try creating custom role and add specific permissions.
In aws if any service want to access any another service you need those specific permission in role.
for more information on lambda permission please check Managed lambda permissions
You need to add the following actions to the IAM Role attached to your lambda:
sqs:ReceiveMessage
sqs:DeleteMessage
sqs:GetQueueAttributes
Otherwise your lambda will not be able to receive any message from the queue. DeleteMessage action allows to remove a message from queue once its successfully processed. As a resource set the ARN of your SQS queue. Policy should look like this:
{
"Action": [
"sqs:DeleteMessage",
"sqs:ReceiveMessage",
"sqs:GetQueueAttributes"
],
"Resource": "arn:aws:sqs:region:accountid:queuename",
"Effect": "Allow"
}
If you're looking for a managed policy, have a look at AWSLambdaSQSQueueExecutionRole.
Attach a policy for a lambda role you might have to change account_number to your account no if you need to invoke another lambda form this lambda
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:**eu-west-1**:**account_number:function**:*"
},
{
"Sid": "",
"Effect": "Allow",
"Action": [
"logs:PutLogEvents",
"logs:CreateLogStream",
"logs:CreateLogGroup"
],
"Resource": "*"
},
{
"Sid": "",
"Effect": "Allow",
"Action": [
"sqs:*"
],
"Resource": "*"
}
]
}

Missing required field Principal - Amazon S3 - Bucket Policy

I'm trying to resize hosted images in amazon s3 using AWS Lamba. I followed the following tutorial which is given by Amazon.
Tutorial
however, when I tried to update my bucket policy it always gives an error as "Missing required field Principal"
This is my policy code:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:*:*:*",
},
{
"Effect": "Allow",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::yyyy",
"Principal":{"AWS":"arn:aws:iam::123456789:user/xxxxx"}
}
]
}
I Couldn't understand why I'm getting the error. What am I doing wrong?
This is actually not an S3 bucket policy, but a policy document that grants permissions to your Lambda function to write logs to CloudWatch and put the objects to your S3 bucket.
Please see how to set up that: https://www.screencast.com/t/SjD72va1Zso