I am trying to learn to use AWS Lambda, DynamoDB, and API Gateway.
I am able to set up a run a successful test of the Lambda that would take querystring parameters and returns the data from the database and I know I have the ability to access the Lambda from my API as I have created one to just return hello world. However, I am stuck as to how to send the parameters I need to the Lambda. These work fine from the Lambda test feature within Lambda.
{
"queryStringParameters": {
"TableName": "Exams",
"Key": {
"ExamName": "MyFirstExam"
}
}
}
However I can not seem to create a html query http://apiurl/route?parameters
as I have no idea how to send the Key.
I have tried everything I can think of here is just a sample of what I have tried
http://apiurl/route?TableName=Exams&Key=ExamName&ExamName=MyFirstExam
http://apiurl/route?TableName=Exams&key=ExamName&Keyvalue=MyFirstExam
http://apiurl/route?TableName=Exams&key=ExamName&value=MyFirstExam
http://apiurl/route?TableName=Exams&key=ExamName:MyFirstExam
but nothing has worked so I am wondering how do you send key = ExamName : MyfirstExam so that it is a part of the querystring parameters?
UPDATE:
I found that the rest api gave me more feed back and it looks like I need to use
````http://apiURL/Route?TableName= Exams&Key= { ExamName: MyFirstExam } ```
as this populates the querystring parameters.
"queryStringParameters":{"TableName":"Exams","Key":"{ ExamName: MyFirstExam }"}
However, I am still getting an error. That leads me to believe that I still may not have the format right. The problem is that when I used the same data in the test from within the lambda console it works and returns the data. When I use the querystring above which appears to be the same data as used in the lambda test it throws an error.
Lambda execution failed with status 200 due to customer function error: The provided key element does not match the schema. So my Key is getting there as I get a status 200 and if I leave the key off of the querystring it throws and error that the key is missing but it appears to not be formatted correctly apparently or is it something else that I am missing. It appears to be the same as the format that is working in the Lambda test.
This works from the lambda test
{ "queryStringParameters": { "TableName": "Exams", "Key": { "ExamName": "MyFirstExam" } } }
this does not work from the API test
"queryStringParameters":{"TableName":"Exams","Key":"{ ExamName: MyFirstExam }"}
This is how I am formatting the querystring
http://apiURL/Route?TableName= Exams&Key= { ExamName: MyFirstExam }
How do I format a querystring so that it works from the API gateway?
The easiest and most commonly used method of joining API Gateway and lambda is through AWS_PROXY integration between your API and your function.
With this type of integration:
API Gateway applies a default mapping template to send the entire request to the Lambda function and transforms the output from the Lambda function to HTTP responses.
This means that every query parameter you provide through your API endpoint is going to be delivered directly into lambda function:
This request data includes the request headers, query string parameters, URL path variables, payload, and API configuration data.
The event format your lambda function gets is described here.
I resolved this by having the key part already in the code and just now only need to pass a normal variable ExamName=Exam.
var queryParameters ={
TableName: request.TableName,
Key:{
"ExamName": request.ExamName
},
};```
Related
I'm trying to get my API Gateway api to:
Run an authorizer
Pass authorizer context to a Step Function execution
Respond to client with Step Function output
I already have #1 and #3 done, but passing the response of the attached authorizer lambda to the step function is proving to be impossible.
I found this page and this page with reference sheets on what interpolation values you can use for your parameter mapping (Create Integration -> Step Function: StartSyncExecution -> Advanced Settings -> Input) but any time I try to use anything related to $context like $context.authorizer.email, API Gateway just responds with an HTTP 400 and gives me this CloudWatch output:
"Unable to resolve property Input from source {\"lambdaName\": \"arn:aws:lambda:us-east-1:xxxxxxx\", \"reqBody\": $request.body.Input, \"authContext\": $context.apiId }. Please make sure that the request to API Gateway contains all the necessary fields specified in request parameters."
These are the JSON objects I've tried using for the Input text box and all of them either give me an errors when trying to save or throw an HTTP 400 and log the above errors when I visit the route:
{"lambdaName": "xxx", "reqBody": $request.body.Input, "authContext": $context.authorizer.email }
{"lambdaName": "xxx", "reqBody": $request.body.Input, "authContext": "$context.authorizer.email" }
{"lambdaName": "xxx", "reqBody": $request.body.Input, "authContext": $context.apiId }
{"lambdaName": "xxx", "reqBody": $request.body.Input, "authContext": $context }
{"lambdaName": "xxx", "reqBody": $request.body.Input, "authContext": $event.requestContext.authorizer.email }
It seems like the only way to have authorization code to work with step functions is to wrap my step function called by API Gateway in another step function that authorizes the request and then invokes the endpoint step function. I've researched this for hours and I'm not getting anywhere. Any help at all is appreciated.
I ended up solving this by using API Gateway v1 and a REST API instead of a HTTP API. For some reason v2's input field currently doesn't work for anything other than $request.body.Input. From there, I hooked up all of my endpoints to a step function that runs the authorization lambda on their Authorization header in the request.
I have a step function that allows me to chain together step function and lambda actions so for most requests I just chain together the authorizer lambda and the endpoint's action (can be lambda or another step function).
The main takeaway here is that if you're using API Gateway and Step Functions, it looks like passing custom-formatted input into your step function isn't very easy to do without using the v1 of API Gateway in a REST API, not an HTTP api. Hopefully this will be fixed in the future.
Another solution could be to use the Mapping Template in API Gateway Integration Request.
Example: Consider the response from the Lambda Authorizer as this:
{
principalId: 'myuser',
context: {
customKey: 'CustomValue'
},
policyDocument: {
Version: '2012-10-17',
Statement: [
{
Action: ['execute-api:Invoke'],
Effect: 'Allow',
Resource:
'arn:aws:execute-api:_region_:111111111111:0a0a0a0a0a/default/POST/my-endpoint'
}
]
}
}
In the Mapping Template for application/json, map whatever property you want in the input field (quote escaped). Each property sent from the Lambda Authorizer in the context field will be available as $context.authorizer.property.
One possible Template Mapping could be:
{
"input": "{\"origin\":\"$input.json('$.requestvalue').replaceAll('\"','')\", \"customPropertyFromLambdaAuthorizer\" : \"$context.authorizer.customKey\" }",
"name": "DemoStateMachineRequest",
"stateMachineArn": "arn:aws:states:_region_:11111111111:stateMachine:MyStateMachine"
}
Reference $context.authorizer.property
I am using serverless to deploy API gateway and lambda using lambda integration. Below is the infra code:
myLambda:
...
events:
- http:
path: customer/{id}
method: delete
integration: lambda
request:
parameters:
paths:
id: true
template:
text/plain: "$input.params('id')"
It creates a delete API to delete customer based on the id specified in the path parameter. I'd like to map the path id to myLambda's event parameter in its handler. But the above code doesn't work the lambda receives the whole http request from API gateway. How can I map the id parameter to the event parameter in lambda?
When you are using API Gateway and Lambda functions, you have 2 options:
Lambda proxy
Lambda integration
By default, the Serverless framework is using Lambda proxy. This way of connecting to a Lambda function is a lot quicker. It passes the whole HTTP request to a Lambda function and it lets you handle the request completely on the Lambda function side.
There is another way, as I mentioned, it's Lambda integration, without using Lambda proxy. If you want to use this method, you need to set up mappings on your request and your response. This way is "cleaner" on the LAmbda code side, because Lambda received mapped parameters in the event, compared to the whole HTTP request as in proxy. On the other hand, this kind of integration is more complicated to set up, as you need to make all the mappings, and it uses VTL (Apache Velocity Template Language) to do that.
In order to set this up using Serverless, take a look at the "Lambda Integration" section in Serverless framework API Gateway documentation:
I believe that ApiGateway event itself should contain that query / path parameter key value pair and you can directly query your parameters from the event, if you'd go with Lambda Proxy method which #Caldazar mentioned above.
For eg let's say if you have a URL
https://example.execute-api.us-west-2.amazonaws.com/prod/confirmReg?token=12345&uid=5
you could simply retrieve the values as below from your handler:
token = event['queryStringParameters']['token']
uid = event["queryStringParameters"]['uid']
EDIT:
Please follow the below steps for Lambda Integration:
Go to your respective API resource method and click on Integration Request section
Find the Mapping Templates area of the Integration request and open it up. Add a new mapping template for the application/json Content-Type. You'll need to change the type in the combo box from "Input passthrough" to "Mapping Template".
Then paste the following piece of JSON inside the template:
{
"id" : "$input.params('id')"
}
This way you'll not end up receiving the entire event object inside your lambda but only your path param which is sent in your request.
Some references: 1 & 2
I am trying to create a simple step function comprising of two lambda functions. The first lambda function takes two numbers as input from the user (say num1=2, num2 =5), and passes on the sum of the two numbers to the second lambda function. The second lambda function multiplies the output of the first lambda function with a constant (say 10).
I want to be able to get the final answer (which is (2+5)*10 = 70) in this case) as the output of an API invocation.
I am able to execute the Step Function successfully (from the AWS Step Function Console). But the output I get when I invoke the API (integrated with the Step Function) is not a number but as follows:
{
"executionArn": "arn:aws:states:ap-south-1:123456789012:execution:Test_Math:xxx",
"startDate": 1560344276.117
}
How do I get the API call to return the answer (which is the number 70 in this case)?
I have gone already gone through the AWS Documentation and AWS API Gateway with Step Function this question but I am not still not clear.
How can I make a new lambda (or use any of the above two) function which invokes/executes this step function to return the answer?
Are there any other ways to return the answer of a step function via an API call?
My question is somewhat similar to this one Api gateway get output results from step function?.
I tried adding a second method in API Gateway which will call the Step Function with the DescribeExecution action but it did not work.
The mapping template (application/json) for the POST Method with StartExecution action is as follows:
#set($inputRoot = $input.path('$'))
#set($data = $util.escapeJavaScript($input.json('$')))
{
"input": "{ \"num1\": $inputRoot.num1, \"num2\": $inputRoot.num2 }",
"stateMachineArn": "arn:aws:states:ap-south 1:998338******:stateMachine:Test_Math"
}
I created a new GET Method with DescribeExecution action with the following Integration Request details:
Integration type: AWS Service
AWS Service: Step Functions
HTTP method: GET
Action: Describe Execution
Content Handling: Passthrough
Mapping template : application/json
#set($inputRoot = $input.path('$'))
#set($data = $util.escapeJavaScript($input.json('$')))
{
"executionArn": "arn:aws:states:ap-south-1:998338321653:execution:Test_Math:3981114a-da51-411d-9533-8571dc976e2d",
"input": "{ \"num1\": $inputRoot.num1, \"num2\": $inputRoot.num2 }"
}
Please let me know what changes do I need to make in the above to be able to return the answer (70) with an API call.
P.S: I want to deploy my machine learning model using step functions and this is just a simple test which I was trying out.
you can use describe_execution method to fetch the final result of your stepfunction. you have to pass the execution arn as an input to this method.
Find more details here, https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/stepfunctions.html#SFN.Client.describe_execution
Below python code will print the output given the execution arn.
import boto3
client = boto3.client('stepfunctions')
executionArn = ''
response = client.describe_execution(
executionArn=executionArn
)
#print(response)
print(response.get('output'))
Using API Gateway, I am trying to define a POST end point that accepts application/json to do the following:
Trigger a Lambda asynchronously
Respond with a JSON payload composed of elements from the request body
I have #1 working. I think it's by the book.
It's #2 I'm getting tripped up on. It looks like I don't have access to the request body in the context of the response mapping template. I have access to the original query params with $input.params but I cannot find any property that will give me the original request body, and I need it to get the data that I want to respond with. It's either that or I need to figure out how to get the asynchronous launch of a Lambda to somehow provide the original request body.
Does anyone know if this is possible?
My goal is to ensure that my API responds as fast as possible without incurring a cold start of a Lambda to respond AND simultaneously triggering an asynchronous workflow by starting a Lambda. I'd also be willing to integrate with SNS instead of Lambda directly and have Lambda subscribe to the topic but I don't know if that will get me access to the data I need in the response mapping template.
From https://stackoverflow.com/a/61482410/3221253:
Save the original request body in the integration mapping template:
#set($context.requestOverride.path.body = $input.body)
Retrieve it in the integration mapping response:
#set($body = $context.requestOverride.path.body)
{
"statusCode": 200,
"body": $body,
}
You can also access specific attributes:
#set($object = $util.parseJson($body))
{
"id": "$object.id"
}
To access the original request directly, you should use a Proxy Integration for Lambda rather than mapping things via a normal integration. You'll be able to access the entire request context, such as headers, path params, etc.
I have determined that it is not possible to do what I want to do.
I'm using AWS API Gateway integrated with Lambda.
Note: I am NOT using Lambda Proxy.
I need to return a binary response from API Gateway. I have successfully set this up as follows:
Encoded my binary data as base64 UTF-8 string and returning ONLY that from my lambda function return "base64 encoded binary data"
Enabled CONVERT_TO_BINARY on the API Gateway Integration Response
Mapped the Content-Type header on the API Gateway Method Response to the binary media type of my binary content
Added the media type of my binary content to API Gateway's list of Binary Media Types
The issue is that as well as sending the binary data (which I can do successfully from the above steps), I need to include a x-my-header custom header in the API Response.
I know how to set-up header mapping in the API Gateway, but the header has to be calculated from database data, and therefore this value needs to also be returned from lambda.
My understanding of lambda integration (remember, I'm not using lambda proxy here) is that API gateway makes a HTTP request to trigger lambda. Lambda then returns a HTTP response to API Gateway, adding the functions output to the body, and also adding internal aws headers to the response.
Now it is possible to map a header to the Method Response using:
integration.response.header.header-name
My question is...
Can I tell lambda to add my custom header to a binary response, when I'm
using custom lambda integration (not proxy) ?
Note: IF i was using lambda proxy, I know that the return object looks as below, and then I would be able to send custom headers. But for reasons out of my control, I cant use lambda proxy.
Lambda return object solution IF i was using lambda proxy:
return {
'body': "base64 encoded binary data",
'headers': 'x-my-header': 'my-value',
'isBase64Encoded': True
}
For Lambda Integration (not proxy) I have tried modifying my lambda output...
return {
"base64-data": "base64 encoded binary data",
"x-my-header: "some value"
}
And setting up a mapping template in the integration response...
$input.json("$.base64-data")
And setting up a header mapping using...
integration.response.body.x-my-header
But API Gateway returns an error:
Execution failed due to configuration error: Unable to transform response
I believe this error occurs because there cannot be a mapping template when you have CONVERT_TO_BINARY enable. From AWS docs:
When converting a text payload to a binary blob, API Gateway assumes that the text data is a Base64-encoded string and outputs the binary data as a Base64-decoded blob. If the conversion fails, it returns a 500 response indicating an API configuration error. You do not provide a mapping template for such a conversion, although you must enable the passthrough behaviors on the API.
I realize this is an old question, but I ran into a similar header mapping issue recently, even though I'm not using binary data.
To my mind the return from Lambda could look like this
{
"base64-data": "base64 encoded binary data",
"x-my-header: "some value"
}
Based on that Lambda Response, you could apply the following mapping (which I modified based on an AWS example)
$input.json("$.base64-data")
#set($context.responseOverride.header.x-my-header = "$input.json('$.x-my-header')")
$input.json("$") references your response and $.[key] is the right way to reference your sub-keys.
You also have to preconfigure your header "x-my-header" in your method response.
In the integration Response the header mapping can just be an empty value - e.g.
""
The real value will be provided by the override ($context.responseOverride.header) which is set by the mapping template.