I've been searching for some sort of official documentation for the data types sent to AWS Lambda from integrations such as, AWS API Gateway. I've been able to find some "Examples" in the API Gateway documentation like here and here. It is also relatively easy to create a Lambda that just echoes the input event as output and check the output. For example (using a REST type API with LAMBDA_PROXY integration) you get something like:
{
"resource": "/another/{parameter}",
"path": "/another/some-parameter",
"httpMethod": "GET",
"headers": {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate, br",
...
},
"multiValueHeaders": {
"Accept": [
"*/*"
],
"Accept-Encoding": [
"gzip, deflate, br"
],
...
},
"queryStringParameters": null,
"multiValueQueryStringParameters": null,
"pathParameters": {
"parameter": "some-parameter"
},
"stageVariables": null,
"requestContext": {
"resourceId": "some-id",
"resourcePath": "/another/{parameter}",
"httpMethod": "GET",
...
},
"body": null,
"isBase64Encoded": false
}
But this doesn't tell me what fields I can always, or just sometimes, expect to be in the payload or the types of the field.
The closest I have to a bit of typing of the fields is the aws-lambda TypeScript typings from the Definitely Typed project.
Is there anything more detailed, or perhaps more general, and official source I can go to for the structure and type of Lambda event payloads for API Gateway and other AWS service integrations?
This is the closest I've come to an answer, so far. If anyone has more detailed (or general information) I'd gladly take it. In the Lambda documentation you can read that (my emphasis):
The runtime passes three arguments to the handler method. The first
argument is the event object, which contains information from the
invoker. The invoker passes this information as a JSON-formatted
string when it calls Invoke, and the runtime converts it to an object.
When an AWS service invokes your function, the event structure varies
by service.
Under "Working with other services" in the lambda documentation we only find the examples presented in the question. Instead we can look in the documentation for the service in question, e.g. API Gateway. For API Gateway, under "Working with [HTTP/REST] APIs", you will find descriptions of the event fields for REST and HTTP type APIs.
With that said, the documentation provided still only refer to the described payload structure as "examples". It only provides you with the names of the fields and some basic structure. Data types would have to be inferred from the examples, see the snippets below.
The story seems to be similar for other services you might want to integrate with Lambda like:
S3
DynamoDB
SQS
All that is to be found is examples of payloads, some on the services' own developer's guide other under Lambda's developer's guide.
HTTP v2.0
The following is an example of the payload format for version 2.0 using HTTP type API. For version 1 see the HTTP type integration docs.
{
version: '2.0',
routeKey: '$default',
rawPath: '/my/path',
rawQueryString: 'parameter1=value1¶meter1=value2¶meter2=value',
cookies: [ 'cookie1', 'cookie2' ],
headers: {
'Header1': 'value1',
'Header2': 'value2'
},
queryStringParameters: { parameter1: 'value1,value2', parameter2: 'value' },
requestContext: {
accountId: '123456789012',
apiId: 'api-id',
authorizer: { jwt: {
claims: {'claim1': 'value1', 'claim2': 'value2'},
scopes: ['scope1', 'scope2']
}
},
domainName: 'id.execute-api.us-east-1.amazonaws.com',
domainPrefix: 'id',
http: {
method: 'POST',
path: '/my/path',
protocol: 'HTTP/1.1',
sourceIp: 'IP',
userAgent: 'agent'
},
requestId: 'id',
routeKey: '$default',
stage: '$default',
time: '12/Mar/2020:19:03:58 +0000',
timeEpoch: 1583348638390
},
body: 'Hello from Lambda',
pathParameters: {'parameter1': 'value1'},
isBase64Encoded: false,
stageVariables: {'stageVariable1': 'value1', 'stageVariable2': 'value2'}
}
REST v3.0
The following is an example of the payload format for version 3.0 using REST type API. For version 2.0 see the REST type integration docs.
{
"openapi": "3.0.0",
"info": {
"version": "2016-09-12T17:50:37Z",
"title": "ProxyIntegrationWithLambda"
},
"paths": {
"/{proxy+}": {
"x-amazon-apigateway-any-method": {
"parameters": [
{
"name": "proxy",
"in": "path",
"required": true,
"schema": {
"type": "string"
}
}
],
"responses": {},
"x-amazon-apigateway-integration": {
"responses": {
"default": {
"statusCode": "200"
}
},
"uri": "arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-1:123456789012:function:SimpleLambda4ProxyResource/invocations",
"passthroughBehavior": "when_no_match",
"httpMethod": "POST",
"cacheNamespace": "roq9wj",
"cacheKeyParameters": [
"method.request.path.proxy"
],
"type": "aws_proxy"
}
}
}
},
"servers": [
{
"url": "https://gy415nuibc.execute-api.us-east-1.amazonaws.com/{basePath}",
"variables": {
"basePath": {
"default": "/testStage"
}
}
}
]
}
Related
I'm in the process of learning AWS Lambda. I have created a lambda that will act as a REST API (APIEvent in CloudFormation terms) and want to debug that Lambda locally using an event.
If I understand correctly, running sam local generate-event apigateway aws-proxy generates an event that is suitable for locally running/debugging my Lambda. This produces the following event (some nested values are abbreviated):
{
"body": "eyJ0ZXN0IjoiYm9keSJ9",
"resource": "/{proxy+}",
"path": "/path/to/resource",
"httpMethod": "POST",
"isBase64Encoded": true,
"queryStringParameters": {
"foo": "bar"
},
"multiValueQueryStringParameters": {
"foo": [
"bar"
]
},
"pathParameters": {
"proxy": "/path/to/resource"
},
"stageVariables": {
"baz": "qux"
},
"headers": {
...
},
"multiValueHeaders": {
...
},
"requestContext": {
"accountId": "123456789012",
"resourceId": "123456",
"stage": "prod",
"requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef",
"requestTime": "09/Apr/2015:12:34:56 +0000",
"requestTimeEpoch": 1428582896000,
"identity": {
...
},
"path": "/prod/path/to/resource",
"resourcePath": "/{proxy+}",
"httpMethod": "POST",
"apiId": "1234567890",
"protocol": "HTTP/1.1"
}
}
However, I do not understand the relation between resource (which is copied in requestContext.resourcePath) and path (which is copied in requestContext.path).
What am I supposed to fill in for these values?
The resource is the API resource you defined in the API gateway. For example in your case /{proxy+}.
The path is the actual path from the request.
So, when you make a request GET https://yourdomain.com/v1/pets :
the path is /v1/pets
the resource is /{proxy+}
I have an AWS API Gateway acting as a proxy to a backend service:
{
"apiKeySource": "HEADER",
"name": "-",
"createdDate": 1513820260,
"binaryMediaTypes": [
"application/zip",
"application/octet-stream"
],
"endpointConfiguration": {
"types": [
"EDGE"
]
},
"id": "-"
}
The integration definition is here:
{
"integrationResponses": {
"200": {
"responseTemplates": {
"application/json": null
},
"statusCode": "200"
}
},
"passthroughBehavior": "WHEN_NO_MATCH",
"timeoutInMillis": 29000,
"uri": "http://${stageVariables.backend}:7000/{proxy}",
"connectionType": "INTERNET",
"httpMethod": "ANY",
"cacheNamespace": "iv06s3",
"type": "HTTP_PROXY",
"requestParameters": {
"integration.request.path.proxy": "method.request.path.proxy",
"integration.request.header.X-Source-IP": "context.identity.sourceIp"
},
"cacheKeyParameters": [
"method.request.path.proxy"
]
}
I have an endpoint that generates a Zip file on the fly and returns it to the requester.
When I access the endpoint directly, the file is fine. When I access it via the API Gateway, it gets corrupted.
The corruption takes the form of bytes in the original file being converted to 0xEFBFBD. This is the UTF-8 'replacement character'.
My request has Accept set to application/zip and the response has Content-Type: application/zip.
My expectation is that the API Gateway should recognize this as a binary media type and leave the file alone, but it seems pretty clear that it's processing it as text content.
What am I doing wrong?
Setting the "Binary Media Type" to "multipart/form-data" resolved a similar issue for me.
See here: AWS Api Gateway as a HTTP Proxy is currupting binary uploaded image files
If I set a header X-Amz-Invocation-Type: 'Event', the call is done asynchronously but as the Amazon documentation states (https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-set-up-simple-proxy.html#api-gateway-simple-proxy-for-lambda-output-format), when one uses Proxy Lambda Integration, the lambda function must return a well formatted response of this kind:
callback(null, {"statusCode": 200, "body": "results"})
As the lambda function is called asynchronously, the API Gateway never get an answer and then return a 502 Bad Gateway error instead of a 200 OK status.
Below an extract of the swagger configuration:
"/myFunc": {
"post": {
"parameters": [
{
"name": "myparam",
"in": "query",
"required": true,
"type": "string"
}
],
"responses": {
"200": {
"description": "200 response"
}
},
"x-amazon-apigateway-request-validator": "Validate query string parameters and headers",
"x-amazon-apigateway-integration": {
"responses": {
"default": {
"statusCode": "200"
}
},
"uri": "arn:aws:apigateway:ap-northeast-1:lambda:path/2015-03-31/functions/arn:aws:lambda:ap-northeast-1:idAccount:function:myFunc/invocations",
"passthroughBehavior": "when_no_match",
"httpMethod": "POST",
"type": "aws_proxy",
"requestParameters": {
"integration.request.header.X-Amz-Invocation-Type": "'Event'"
}
}
}
}
Is there a way to have it worked?
You can setup a custom lambda integration (without proxy flag). You will need to configure the mapping templates to transform the request/response to your desired format.
http://docs.aws.amazon.com/apigateway/latest/developerguide/getting-started-lambda-non-proxy-integration.html#getting-started-new-lambda
As I understand it, if I create a ressource as /{proxy+} with an ANY method it should catch /, /pages and /pages/123/image?
Although it doesn't seem so. What could I be doing wrong?
The integration is a Lambda function, and when I test it everything looks fine.
I have deployed the API.
"/{proxy+}": {
"x-amazon-apigateway-any-method": {
"produces": [
"application/json"
],
"parameters": [
{
"name": "proxy",
"in": "path",
"required": true,
"type": "string"
}
],
"responses": {},
"security": []
}
}
I thought a /{proxy+} ressource could be used to catch requests to other ressources and do validation etc, but if another ressource matches the request, it takes precedence over {proxy+}.
I searched on AWS Lambda documentation but couldn't find an answer to my problem.
Is there a way I can access the entire request body from a Lambda function (written in node.js)?
The event parameter only seems to contain parsed JSON properties.
Your request body needs to be in XML or JSON in order to be able to access it in your Lambda function. You need to specify how it's processed/mapped and passed through in Integration Request section of the API Gateway dashboard.
You can access the request body in AWS Lambda once you expose it in a Body Mapping Template.
Open up your method in API Gateway console
Open Integration Request
In Integration Request, open the Body Mapping Templates panel
Add a Content Type of application/json
Click on your freshly created application/json item
Add the following template:
{
"body" : $input.json('$')
}
After that you can access the request body as event.body in your Node.js Lambda Function.
As an alternative, you might consider setting up lambda as a simple proxy if it works better for your use case. I've found more people using this technique recently.
http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-set-up-simple-proxy.html
A request like the following:
POST /testStage/hello/world?name=me HTTP/1.1
Host: gy415nuibc.execute-api.us-east-1.amazonaws.com
Content-Type: application/json
headerName: headerValue
{
"a": 1
}
Will wind up sending the following event data to your AWS Lambda function:
{
"message": "Hello me!",
"input": {
"resource": "/{proxy+}",
"path": "/hello/world",
"httpMethod": "POST",
"headers": {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate",
"cache-control": "no-cache",
"CloudFront-Forwarded-Proto": "https",
"CloudFront-Is-Desktop-Viewer": "true",
"CloudFront-Is-Mobile-Viewer": "false",
"CloudFront-Is-SmartTV-Viewer": "false",
"CloudFront-Is-Tablet-Viewer": "false",
"CloudFront-Viewer-Country": "US",
"Content-Type": "application/json",
"headerName": "headerValue",
"Host": "gy415nuibc.execute-api.us-east-1.amazonaws.com",
"Postman-Token": "9f583ef0-ed83-4a38-aef3-eb9ce3f7a57f",
"User-Agent": "PostmanRuntime/2.4.5",
"Via": "1.1 d98420743a69852491bbdea73f7680bd.cloudfront.net (CloudFront)",
"X-Amz-Cf-Id": "pn-PWIJc6thYnZm5P0NMgOUglL1DYtl0gdeJky8tqsg8iS_sgsKD1A==",
"X-Forwarded-For": "54.240.196.186, 54.182.214.83",
"X-Forwarded-Port": "443",
"X-Forwarded-Proto": "https"
},
"queryStringParameters": {
"name": "me"
},
"pathParameters": {
"proxy": "hello/world"
},
"stageVariables": {
"stageVariableName": "stageVariableValue"
},
"requestContext": {
"accountId": "12345678912",
"resourceId": "roq9wj",
"stage": "testStage",
"requestId": "deef4878-7910-11e6-8f14-25afc3e9ae33",
"identity": {
"cognitoIdentityPoolId": null,
"accountId": null,
"cognitoIdentityId": null,
"caller": null,
"apiKey": null,
"sourceIp": "192.168.196.186",
"cognitoAuthenticationType": null,
"cognitoAuthenticationProvider": null,
"userArn": null,
"userAgent": "PostmanRuntime/2.4.5",
"user": null
},
"resourcePath": "/{proxy+}",
"httpMethod": "POST",
"apiId": "gy415nuibc"
},
"body": "{\r\n\t\"a\": 1\r\n}",
"isBase64Encoded": false
}
}
Now you have access to all headers, url params, body etc. in each request.