How can i call sagemaker inference endpoint using API gateway - amazon-web-services

I am trying to call sagemaker inference endpoint from api gateway with AWS Integration.I don't want to use lamdba in between of API gateway and sagemaker runtime. I followed this doc to setup api gateway method but it fails.
How can i call sagemaker inference endpoint from API gateway?
Web Browser ----> API Gateway ----> Sagemaker endpoint

API Gateway supports integration with AWS services directly (without the Lambda). You can follow the instructions at https://docs.aws.amazon.com/apigateway/latest/developerguide/getting-started-aws-proxy.html.
When you go to Step 4 in the instructions above, for the AWS Service option, you can choose 'SageMaker Runtime' to target the invoke endpoints.

API Gateway can be used to front an Amazon SageMaker inference endpoint as a REST API, by making use of an API Gateway feature called mapping templates. This feature makes it possible for the REST API to be integrated directly with an Amazon SageMaker runtime endpoint, thereby avoiding the use of any intermediate compute resource (such as AWS Lambda or Amazon ECS containers) to invoke the endpoint. The result is a solution that is simpler, faster, and cheaper to run. See this blog post for more detail on how to configure the API Gateway mapping templates against the Sagemaker runtime endpoint.

it's a long shot since it's an old question but somebody might end up here.
Reading the first section of the documentation about calling the inference endpoint in sagemaker, you'll find that you can only call it with a POST and pass your input data in the body.
https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html
So it might be that you created a GET method in API Gateway and that you need to map your request parameters to a body payload or simply set up a POST method instead.

Related

AWS HTTP API Gateway as a proxy to private S3 bucket

I have a private S3 bucket with lots of small files. I'd like to expose the contents of the bucket (only read-only access) using AWS API Gateway as a proxy. Both S3 bucket and AWS API Gateway belong to the same AWS account and are in the same VPC and Availability Zone.
AWS API Gateway comes in two types: HTTP API, REST API. The configuration options of REST API are more advanced, additionally, REST API supports much more AWS services integrations than the HTTP API. In fact, the use case I described above is fully covered in one of the documentation tabs of REST API. However, REST API has one huge disadvantage - it's about 70% more expensive than the HTTP API, the price comes with more configuration options but as for now, I need only one - integration with the S3 service that's why I believe this type of service is not well suited for my use case. I started searching if HTTP API can be integrated with S3, and so far I haven't found any way to achieve it.
I tried creating/editing service-linked roles associated with the HTTP API Gateway instance, but those roles can't be edited (only read-only access). As for now, I don't have any idea where I should search next, or if my goal is even achievable using HTTP API.
I am a fan of AWSs HTTP APIs.
I work daily with an API that serves a very similar purpose. The way I have done it is by using AWS Lambda functions integrated with the APIs paths.
What works for me is this:
Define your API paths, and integrate them with AWS Lambda functions.
Have your integrated Lambda function return a signed URL for any objects you want to provide access to through API calls.
There are several different ways to pass the name of the object(s) you want to the Lambda function servicing the API call.
This is the short answer. I plan to give a longer answer at a later time. But this has worked for me.

How to use aws lambda without HTTP api?

Since there is aditional costs for using HTTP and REST apis on AWS lambda, i would like to know if i could make AWS Lambda receive gets and posts without the need of these HTTP API services.
In this example it seems to be possible:
https://github.com/serverless/examples/tree/master/aws-node-simple-http-endpoint
You will need to use the API Gateway to expose your lambda. Your example is actually using an API Gateway, because the endpoint is execute-api.us-east-1.amazonaws.com and that is the Amazon API Gateway Data Plane.
Just to be clear; if you need to expose the Lambda externally you need to use the API Gateway. If the Lambda needs to be invoked internally then you don't need the API GW.
Best regards
Lambda also exposes a client API in all languages. Therefore, you can invoke a Lambda function by using the client API (not use API Gateway if you prefer). For example, assume you want the ability to invoke a Lambda function from a Java web app. In this situation, you can use the LambdaClient object to do so. You can find an example here:
https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/javav2/example_code/lambda/src/main/java/com/example/lambda/LambdaInvoke.java

Create an api in api gateway to invoke a lambda function

I am working on a project and trying to use API Gateway to invoke a lambda function. The lambda function is used to update a DynamoDB item. The DynamoDB table is used to keep a running count of visitors to a web page. I need to create an API to invoke the lambda function but I'm not sure how to create the API. Any assistance is appreciated.
General steps would be:
Create AWS_PROXY integration between API Gateway and your Lambda function. The example of this is in the AWS tutorials: Set up Lambda proxy integrations in API Gatewa and in Tutorial: Build a REST API with HTTP proxy integration
Add/amend execution role to your function allowing it to access DynamoDB. This is exemplified in the AWS tutorial: Using AWS Lambda with Amazon DynamoDB.
Test the API. It can be done directly in API gateway console, or using external tools such as curl or Postman.
I figured out my issue. In my lamdba function, I needed to change the output to a JSON object. Once I made the change, I was able to get my API working. Here is a link to the fix.

API Gateway - How Deploy API works?

After creating API Gateway with two API names and integrating with lambda function,
AWS documentation recommends to deploy this API, as shown below:
1) What does deploy API mean? How creating API gateway different from deploying API?
2) Does deploy API option internally create Cloud formation template? that creates a stack and deploy
1) What does deploy API mean? How creating an API gateway different from deploying API?
Let' say you have created your API but how about making it public so that it can be used.
That's where deploy comes. Once you are done with writing your API, deploy it to make it callable by your users. When you deploy, then you get the link from API Gateway which can be accessed by everyone.
It is described here
2) Does deploy API option internally create Cloud formation template? that creates a stack and deploy
No, Like you said you have integrated a lambda function with your API then API Gateway will simply redirect all the calls to your lambda function which is serverless.
An API Gateway is a Proxy that manages the endpoints ,It acts as the single entryway into a system allowing multiple APIs or microservices to act cohesively and provide a uniform experience to the user.
The most important role the API gateway plays is ensuring reliable processing of every API call. In addition, the API gateway provides the ability to design API specs, help provide enterprise-grade security, and manage APIs centrally.
An API Gateway is a server that is the single entry point into the
system. It is similar to the Facade pattern from object‑oriented
design. The API Gateway encapsulates the internal system architecture
and provides an API that is tailored to each client. It might have
other responsibilities such as authentication, monitoring, load
balancing, caching, request shaping and static response handling.
https://learn.microsoft.com/en-us/azure/architecture/microservices/design/gateway
https://microservices.io/patterns/apigateway.html
Deploying a REST API in Amazon API Gateway:
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-tutorials.html
https://auth0.com/docs/integrations/aws-api-gateway/custom-authorizers/part-1
https://auth0.com/docs/integrations/aws-api-gateway/custom-authorizers/part-2
https://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-deploy-api.html

How can I invoke AWS SageMaker endpoint to get inferences?

I want to get real time predictions using my machine learning model with the help of SageMaker. I want to directly get inferences on my website. How can I use the deployed model for predictions?
Sagemaker endpoints are not publicly exposed to the Internet. So, you'll need some way of creating a public HTTP endpoint that can route requests to your Sagemaker endpoint. One way you can do this is with an AWS Lambda function fronted by API gateway.
I created an example web app that takes webcam images and passes them on to a Sagemaker endpoint for classification. This uses the API Gateway -> Lambda -> Sagemaker endpoint strategy that I described above. You can see the whole example, including instructions for how to set up the Lambda (and the code to put in the lambda) at this GitHub repository: https://github.com/gabehollombe-aws/webcam-sagemaker-inference/
You can invoke the SageMaker endpoint using API Gateway or Lambda.
Lambda:
Use sagemaker aws sdk and invoke the endpoint with lambda.
API Gateway:
Use API Gateway and pass parameters to the endpoint with AWS service proxy.
Documentation with example:
https://aws.amazon.com/blogs/machine-learning/call-an-amazon-sagemaker-model-endpoint-using-amazon-api-gateway-and-aws-lambda/
Hope it helps.
Use the CLI like this:
aws sagemaker-runtime invoke-endpoint \
--endpoint-name <endpoint-name> \
--body '{"instances": [{"in0":[863],"in1":[882]}]}' \
--content-type application/json \
--accept application/json \
results
I found it over here in a tutorial about accessing Sagemaker via API Gateway.
As other answers have mentioned, your best option is fronting the SageMaker endpoint with a REST API in API Gateway. The API then lets you control authorisation and 'hides' the backend SageMaker endpoint from API clients, lowering the coupling between API clients (your website) and your backend. (By the way, you don't need a Lambda function there, you can directly integrate the REST API with SageMaker as a backend).
However, if you are simply testing the endpoint after deploying it and you want to quickly get some inferences using Python, there's two options:
After deploying your endpoint with predictor = model.deploy(...), if you still have the predictor object available in your Python scope, you can simply run predictor.predict(), as documented here. However, it's rather likely that you've deployed the endpoint a while ago and you can no longer access the predictor object, and naturally one doesn't want to re-deploy the entire endpoint just to get the predictor.
If your endpoint already exists, you can invoke it using boto3 as follows, as documented here:
import boto3
payload = "string payload"
endpoint_name = "your-endpoint-name"
sm_runtime = boto3.client("runtime.sagemaker")
response = sm_runtime.invoke_endpoint(
EndpointName=endpoint_name,
ContentType="text/csv",
Body=payload
)
response_str = response["Body"].read().decode()
Naturally, you can adjust the above invocation according to your content type, to send JSON data for example. Then just be aware of the (de)serializer the endpoint uses, as well as the ContentType in the argument to invoke_endpoint.