API gateway and DynamoDB integration - SerializationException - amazon-web-services

I have setup an API endpoint integration to DynamoDB.
I want to query a table using query string parameter as query filter.
Unfortunatley I ended up getting error:
Method response body after transformations: {"__type":"com.amazon.coral.service#SerializationException"}
with no clue what is the root cause.
To do the stated above I have taken the following steps:
1) Setup a get method in API gateway
2) In the Method Request I added a parameter to URL Query String Parameters
3) In the integration request I added the same parameter from step 2 in URL Query String Parameters
4)In DynamoDB table I decalred a new index for the column that I'd like to query
5)In body mapping template:
{
"TableName": "tjusers",
"IndexName" "badge-index",
"KeyConditionExpression": "badge = :v1",
"ExpressionAttributeValues": {
":v1": {
"S": "$input.params('badge')"
}
}
}
THe exception:
xecution log for request f0081606-b521-11e8-adb2-cd8092221b33
Mon Sep 10 17:50:00 UTC 2018 : Starting execution for request: f0081606-b521-11e8-adb2-cd8092221b33
Mon Sep 10 17:50:00 UTC 2018 : HTTP Method: GET, Resource Path: /db/users
Mon Sep 10 17:50:00 UTC 2018 : Method request path: {}
Mon Sep 10 17:50:00 UTC 2018 : Method request query string: {badge=18323}
Mon Sep 10 17:50:00 UTC 2018 : Method request headers: {}
Mon Sep 10 17:50:00 UTC 2018 : Method request body before transformations:
Mon Sep 10 17:50:00 UTC 2018 : Endpoint request URI: https://dynamodb.us-west-2.amazonaws.com/?Action=Query&badge=11111
Mon Sep 10 17:50:00 UTC 2018 : Endpoint request headers: {Authorization=****************************************************************************************************************************************************************************************************************************************************************************************feb2ac, X-Amz-Date=20180910T175000Z, x-amzn-apigateway-api-id=53zgcztitj, Accept=application/json, User-Agent=AmazonAPIGateway_53zgcztitj, X-Amz-Security-Token=AgoGb3JpZ2l....
Mon Sep 10 17:50:00 UTC 2018 : Endpoint request body after transformations: {
"TableName": "tjusers",
"IndexName" "badge-index",
"KeyConditionExpression": "badge = :v1",
"ExpressionAttributeValues": {
":v1": {
"S": "11111"
}
}
}
Mon Sep 10 17:50:00 UTC 2018 : Sending request to https://dynamodb.us-west-2.amazonaws.com/?Action=Query&badge=18323
Mon Sep 10 17:50:00 UTC 2018 : Received response. Integration latency: 11 ms
Mon Sep 10 17:50:00 UTC 2018 : Endpoint response body before transformations: {"__type":"com.amazon.coral.service#SerializationException"}
Mon Sep 10 17:50:00 UTC 2018 : Endpoint response headers: {Server=Server, Date=Mon, 10 Sep 2018 17:49:59 GMT, Content-Type=application/x-amz-json-1.0, Content-Length=60, Connection=keep-alive, x-amzn-RequestId=ODMJ814D1GBN875T6MTR7BUHGVVV4KQNSO5AEMVJF66Q9ASUAAJG, x-amz-crc32=3948637019}
Mon Sep 10 17:50:00 UTC 2018 : Method response body after transformations: {"__type":"com.amazon.coral.service#SerializationException"}
Mon Sep 10 17:50:00 UTC 2018 : Method response headers: {X-Amzn-Trace-Id=Root=1-5b96aec8-addec09261723342f53a179f, Content-Type=application/json}
Mon Sep 10 17:50:00 UTC 2018 : Successfully completed execution
Mon Sep 10 17:50:00 UTC 2018 : Method completed with status: 200

You are missing : after "IndexName", that cause SerializationException

What is the schema of your tjusers DynamoDB Table?
In particular, what is the datatype of badge? If it is Number, for example, you need to be using "N" instead of "S".
(In general, here is a recent new tutorial that meets your requirements and has more details/screenshots than I've seen in others: https://medium.com/#likhita507/using-api-gateway-to-get-data-from-dynamo-db-using-without-using-aws-lambda-e51434a4f5a0)

Related

AWS APIGateway Integration Response Mapping Templates do not switch correctly based on Content-Type

I am experimenting with an AWS API-Gateway integration with an S3 backend. I have noticed the switch between different mapping-templates in the integration-response does not appear to work.
In integration-response, I have the following mapping templates:
application/json:
{
type: "JSON",
body: "$input.body"
}
text/plain:
PlainText:
$input.body
Nothing I can do appears to be able to make the text/plain mapping template to be used - it always uses application/json.
I would expect that the text/plain mapping template to be used based on one of the following being true:
S3 returns content with Content-Type: text/plain.
The initial request to API-Gateway passes an Accept: text/plain header.
As per the example below, S3 returns a Content-Type: text/plain AND I request with Accept: text/plain. API-Gateway correctly responds with Content-Type: text/plain also.
However the application/json template is still used to transform the body.
I have even removed application/json as a valid response type from the method-response entirely, but still nothing.
Any thoughts why this is happening?
FYI I am using a classic v1 ApiGateway (Rest).
Execution log for request c24cfce3-2cf3-4693-ad72-4fdf44f4fdcd
Wed May 19 17:13:38 UTC 2021 : Starting execution for request: c24cfce3-2cf3-4693-ad72-4fdf44f4fdcd
Wed May 19 17:13:38 UTC 2021 : HTTP Method: GET, Resource Path: /feeds-poc/test.txt
Wed May 19 17:13:38 UTC 2021 : Method request path: {filename=test.txt}
Wed May 19 17:13:38 UTC 2021 : Method request query string: {}
Wed May 19 17:13:38 UTC 2021 : Method request headers: {Accept=text/plain}
Wed May 19 17:13:38 UTC 2021 : Method request body before transformations:
Wed May 19 17:13:38 UTC 2021 : Endpoint request URI: https://my-bucket-id.s3.eu-west-1.amazonaws.com/test.txt
Wed May 19 17:13:38 UTC 2021 : Endpoint request headers: {Authorization=****57173a, X-Amz-Date=20210519T171338Z, x-amzn-apigateway-api-id=123456789a, Accept=application/json, User-Agent=AmazonAPIGateway_123456789a, X-Amz-Security-Token=**** [TRUNCATED]
Wed May 19 17:13:38 UTC 2021 : Endpoint request body after transformations:
Wed May 19 17:13:38 UTC 2021 : Sending request to https://my-bucket-id.s3.eu-west-1.amazonaws.com/test.txt
Wed May 19 17:13:38 UTC 2021 : Received response. Status: 200, Integration latency: 43 ms
Wed May 19 17:13:38 UTC 2021 : Endpoint response headers: {x-amz-id-2=****, x-amz-request-id=****, Date=Wed, 19 May 2021 17:13:39 GMT, Last-Modified=Mon, 17 May 2021 16:45:13 GMT, ETag="420f804aa21220bf0db57bb4b9799c8a", Accept-Ranges=bytes, Content-Type=text/plain, Content-Length=13, Server=AmazonS3}
Wed May 19 17:13:38 UTC 2021 : Endpoint response body before transformations: It's working
Wed May 19 17:13:38 UTC 2021 : Method response body after transformations: {
type: "JSON",
body: "It's working
"
}
Wed May 19 17:13:38 UTC 2021 : Method response headers: {X-Amzn-Trace-Id=Root=****, Content-Type=text/plain}
Wed May 19 17:13:38 UTC 2021 : Successfully completed execution
Wed May 19 17:13:38 UTC 2021 : Method completed with status: 200
AWS Support confirmed to me that the switch is based on the original request Accept header. Testing this using Postman confirms this functionality.
The test tools built into the AWS console for API-Gateway however do not appear to cater for testing response mapping templates - from AWS support: the purpose of API Gateway test console is only to test the Integration, it doesn't work for end-to-end request.

Creating a REST API from Lambda Function

I have a Java based Lambda function that is running correctly via the Lambda test event with the following JSON:
{
"married": "true",
"wages": "200000",
"homeInterest": "15000",
"propertyTaxes": "15000",
"stateTaxes": "13000",
"otherDeductions": "4000",
"postalCode": "11762"
}
I then created an API via the Amazon API Gateway. When I paste the same JSON as the body of the generated URL none of fields map correctly.
The Lambda handler is using POJOs for the request and response:
public class TaxHandler implements RequestHandler<TaxRequest, TaxResponse>{
public TaxResponse handleRequest(TaxRequest request, Context context){
When I test via the API gateway test I see the following info:
Execution log for request test-request
Tue Dec 19 02:01:16 UTC 2017 : Starting execution for request: test-invoke-request
Tue Dec 19 02:01:16 UTC 2017 : HTTP Method: POST, Resource Path: /TaxCalculation
Tue Dec 19 02:01:16 UTC 2017 : Method request path: {}
Tue Dec 19 02:01:16 UTC 2017 : Method request query string: {}
Tue Dec 19 02:01:16 UTC 2017 : Method request headers: {}
Tue Dec 19 02:01:16 UTC 2017 : Method request body before transformations: {
"married": "true",
"wages": "200000",
"homeInterest": "15000",
"propertyTaxes": "15000",
"stateTaxes": "13000",
"otherDeductions": "4000",
"postalCode": "11762"
}
Tue Dec 19 02:01:16 UTC 2017 : Endpoint request URI: https://lambda.us-east-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-1:896795400074:function:TaxCalculation/invocations
Tue Dec 19 02:01:16 UTC 2017 : Endpoint request headers: {x-amzn-lambda-integration-tag=test-request, Authorization=************************************************************************************************************************************************************************************************************************************************************************************************************************7fcbc9, X-Amz-Date=20171219T020116Z, x-amzn-apigateway-api-id=07yp3njqzk, X-Amz-Source-Arn=arn:aws:execute-api:us-east-1:896795400074:07yp3njqzk/null/POST/TaxCalculation, Accept=application/json, User-Agent=AmazonAPIGateway_07yp3njqzk, X-Amz-Security-Token= [TRUNCATED]
Tue Dec 19 02:01:16 UTC 2017 : Endpoint request body after transformations: {"resource":"/TaxCalculation","path":"/TaxCalculation","httpMethod":"POST","headers":null,"queryStringParameters":null,"pathParameters":null,"stageVariables":null,"requestContext":{"path":"/TaxCalculation","accountId":"xxxxxxxx","resourceId":"v8358d","stage":"test-invoke-stage","requestId":"test-invoke-request","identity":{"cognitoIdentityPoolId":null,"cognitoIdentityId":null,"apiKey":"test-invoke-api-key","cognitoAuthenticationType":null,"userArn":"arn:aws:iam::xxxxx:user/tvfoodmaps_aws","apiKeyId":"test-invoke-api-key-id","userAgent":"Apache-HttpClient/4.5.x (Java/1.8.0_144)","accountId":"896795400074","caller":"AIDAINMSXKH5AWAQ7NX36","sourceIp":"test-invoke-source-ip","accessKey":"ASIAIHXWW4BOHGXESRNQ","cognitoAuthenticationProvider":null,"user":"AIDAINMSXKH5AWAQ7NX36"},"resourcePath":"/TaxCalculation","httpMethod":"POST","apiId":"07yp3njqzk"},"body":"{\n \"married\": \"true\",\n \"wages\": \"200000\",\n \"homeInterest\": \"15000\",\n \"propertyTa [TRUNCATED]
Tue Dec 19 02:01:16 UTC 2017 : Sending request to https://lambda.us-east-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-1:896795400074:function:TaxCalculation/invocations
Tue Dec 19 02:01:16 UTC 2017 : Received response. Integration latency: 30 ms
Tue Dec 19 02:01:16 UTC 2017 : Endpoint response body before transformations: {"savings":565.0,"owedTaxes17":-635.0,"owedTaxes18":-1200.0,"effectiveRate17":0.0,"effectiveRate18":0.0}
Tue Dec 19 02:01:16 UTC 2017 : Endpoint response headers: {X-Amz-Executed-Version=$LATEST, x-amzn-Remapped-Content-Length=0, Connection=keep-alive, x-amzn-RequestId=7f8a1b13-e460-11e7-84cf-d1c3e8d3eaf5, Content-Length=104, Date=Tue, 19 Dec 2017 02:01:16 GMT, X-Amzn-Trace-Id=root=1-5a3872ec-1b9d875d8cc2fded5c30da46;sampled=0, Content-Type=application/json}
Tue Dec 19 02:01:16 UTC 2017 : Execution failed due to configuration error: Malformed Lambda proxy response
Tue Dec 19 02:01:16 UTC 2017 : Method completed with status: 502
How can I further debug why the function is not executing the same when called via REST instead of directly via the Lambda tester? Note: I know the error talks about the response but the issue is that the first line of my code reading the fields that should be mapped to the pojo aren't working (again, only when using the API).
Looks like you have your integration with ANY and not returning the proxy response. Instead you are returning the JSON object response.
Similar problem discussed here,
https://forums.aws.amazon.com/thread.jspa?threadID=255561
And the solution to configure is documented here,
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-set-up-simple-proxy.html#api-gateway-simple-proxy-for-lambda-output-format
Hope it helps.
You can include logging in your code so that they will be available in AWS CloudWatch to investigate the issue. Following libraries and approaches are recommended by AWS according to the documentation for Java.
Custom Appender for Log4j™ 2
LambdaLogger.log(): Example 2: Writing Logs Using LambdaLogger (Java)
System.out() and System.err()
Note: Also setup API Gateway logging so that it would be easier to trace any issue end to end.

AWS API Gateway Cannot GET / when function sleeped for long time

My current stack is AWS API Gateway --> AWS Lambda --> swagger-node + swagger-express-mw + aws-serverless-express.
So my Swagger API is hosted as one node.js Lambda Function and Invoked with aws_proxy from API Gateway. This works quite good. The only thing is that when the function sleeped for too long (cold start?) I get a Cannot GET / as Output from every URL I am calling first. From 2nd Request on, it runs very fast. Any ideas on that?
I don't think that it comes from API Gateway Integration Timeout as that are 30 seconds. The slowest invocation time of the function directly via lambda is around 2,5s and when it is called more often it is normally not more than 150ms. I also increased the Time of Lambda Timeout for that function to 10s so from there should also not come an error.
Logs from Test Request via API Gateway first Invocation
Response Body
Cannot GET /hello
Response Headers
{
"x-powered-by": "Express",
"x-content-type-options": "nosniff",
"content-type": "text/html; charset=utf-8",
"content-length": "18",
"date": "Sun, 19 Feb 2017 15:00:11 GMT",
"connection": "close",
"X-Amzn-Trace-Id": "<TRACE-ID>"
}
Logs
Execution log for request test-request
Sun Feb 19 15:00:07 UTC 2017 : Starting execution for request: test-invoke-request
Sun Feb 19 15:00:07 UTC 2017 : HTTP Method: GET, Resource Path: /hello
Sun Feb 19 15:00:07 UTC 2017 : Method request path: {}
Sun Feb 19 15:00:07 UTC 2017 : Method request query string: {}
Sun Feb 19 15:00:07 UTC 2017 : Method request headers: {}
Sun Feb 19 15:00:07 UTC 2017 : Method request body before transformations:
Sun Feb 19 15:00:07 UTC 2017 : Endpoint request URI: https://lambda.eu-central-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:eu-central-1:<ACCOUNT-ID>:function:api/invocations
Sun Feb 19 15:00:07 UTC 2017 : Endpoint request headers: {x-amzn-lambda-integration-tag=test-request, Authorization=**************************************************************************************************************************************************************************************************************************************************************************************************************************4b0637, X-Amz-Date=20170219T150007Z, x-amzn-apigateway-api-id=965h04axki, Accept=application/json, User-Agent=AmazonAPIGateway_965h04axki, X-Amz-Security-Token=<SECURITY-TOKEN>
Sun Feb 19 15:00:07 UTC 2017 : Endpoint request body after transformations: {"resource":"/hello","path":"/hello","httpMethod":"GET","headers":null,"queryStringParameters":null,"pathParameters":null,"stageVariables":null,"requestContext":{"accountId":"<ACCOUNT-ID>","resourceId":"ll6gw8","stage":"test-invoke-stage","requestId":"test-invoke-request","identity":{"cognitoIdentityPoolId":null,"accountId":"<ACCOUNT-ID>","cognitoIdentityId":null,"caller":"<ACCOUNT-ID>","apiKey":"test-invoke-api-key","sourceIp":"test-invoke-source-ip","accessKey":"<ACCESS-ID>","cognitoAuthenticationType":null,"cognitoAuthenticationProvider":null,"userArn":"arn:aws:iam::<ACCOUNT-ID>:root","userAgent":"Apache-HttpClient/4.5.x (Java/1.8.0_102)","user":"<ACCOUNT-ID>"},"resourcePath":"/hello","httpMethod":"GET","apiId":"965h04axki"},"body":null,"isBase64Encoded":false}
Sun Feb 19 15:00:11 UTC 2017 : Endpoint response body before transformations: {"statusCode":404,"body":"Cannot GET /hello\n","headers":{"x-powered-by":"Express","x-content-type-options":"nosniff","content-type":"text/html; charset=utf-8","content-length":"18","date":"Sun, 19 Feb 2017 15:00:11 GMT","connection":"close"},"isBase64Encoded":false}
Sun Feb 19 15:00:11 UTC 2017 : Endpoint response headers: {x-amzn-Remapped-Content-Length=0, x-amzn-RequestId=19f8554e-f6b4-11e6-8184-d3ccf0ccf643, Connection=keep-alive, Content-Length=267, Date=Sun, 19 Feb 2017 15:00:11 GMT, Content-Type=application/json}
Sun Feb 19 15:00:11 UTC 2017 : Method response body after transformations: Cannot GET /hello
Sun Feb 19 15:00:11 UTC 2017 : Method response headers: {x-powered-by=Express, x-content-type-options=nosniff, content-type=text/html; charset=utf-8, content-length=18, date=Sun, 19 Feb 2017 15:00:11 GMT, connection=close, X-Amzn-Trace-Id=Root=1-58a9b2f7-91fc7371e41d6ae9c2fbf64d}
Sun Feb 19 15:00:11 UTC 2017 : Successfully completed execution
Sun Feb 19 15:00:11 UTC 2017 : Method completed with status: 404
Logs from Test Request via API Gateway second Invocation
Response Body
"Hello, stranger!"
Response Headers
{
"x-powered-by": "Express",
"access-control-allow-origin": "*",
"content-type": "application/json; charset=utf-8",
"content-length": "18",
"etag": "W/\"12-E1p7iNXxJ4trMdmFBhlU9Q\"",
"date": "Mon, 13 Feb 2017 20:12:36 GMT",
"connection": "close",
"X-Amzn-Trace-Id": "<Trace-ID>"
}
Logs
Execution log for request test-request
Mon Feb 13 20:12:36 UTC 2017 : Starting execution for request: test-invoke-request
Mon Feb 13 20:12:36 UTC 2017 : HTTP Method: GET, Resource Path: /hello
Mon Feb 13 20:12:36 UTC 2017 : Method request path: {}
Mon Feb 13 20:12:36 UTC 2017 : Method request query string: {}
Mon Feb 13 20:12:36 UTC 2017 : Method request headers: {}
Mon Feb 13 20:12:36 UTC 2017 : Method request body before transformations:
Mon Feb 13 20:12:36 UTC 2017 : Endpoint request URI: https://lambda.eu-central-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:eu-central-1:<LAMBDA-FUNCTION-ID>:function:api/invocations
Mon Feb 13 20:12:36 UTC 2017 : Endpoint request headers: {x-amzn-lambda-integration-tag=test-request, Authorization=*******************************************************************************************************************************************************************************************************************************************************************************************************************************************3e1b18, X-Amz-Date=20170213T201236Z, x-amzn-apigateway-api-id=965h04axki, X-Amz-Source-Arn=arn:aws:execute-api:eu-central-1:<ACCOUNT-ID>:965h04axki/null/GET/hello, Accept=application/json, User-Agent=AmazonAPIGateway_965h04axki, X-Amz-Security-Token=<TOKEN>
Mon Feb 13 20:12:36 UTC 2017 : Endpoint request body after transformations: {"resource":"/hello","path":"/hello","httpMethod":"GET","headers":null,"queryStringParameters":null,"pathParameters":null,"stageVariables":null,"requestContext":{"accountId":"<ACCOUNT-ID>","resourceId":"ll6gw8","stage":"test-invoke-stage","requestId":"test-invoke-request","identity":{"cognitoIdentityPoolId":null,"accountId":"<ACCOUNT-ID>","cognitoIdentityId":null,"caller":"427402682812","apiKey":"test-invoke-api-key","sourceIp":"test-invoke-source-ip","accessKey":"<ACCESS-KEY>","cognitoAuthenticationType":null,"cognitoAuthenticationProvider":null,"userArn":"arn:aws:iam::<ACCOUNT-ID>:root","userAgent":"Apache-HttpClient/4.5.x (Java/1.8.0_102)","user":"<ACCOUNT-ID>"},"resourcePath":"/hello","httpMethod":"GET","apiId":"965h04axki"},"body":null,"isBase64Encoded":false}
Mon Feb 13 20:12:36 UTC 2017 : Endpoint response body before transformations: {"statusCode":200,"body":"\"Hello, stranger!\"","headers":{"x-powered-by":"Express","access-control-allow-origin":"*","content-type":"application/json; charset=utf-8","content-length":"18","etag":"W/\"12-E1p7iNXxJ4trMdmFBhlU9Q\"","date":"Mon, 13 Feb 2017 20:12:36 GMT","connection":"close"},"isBase64Encoded":false}
Mon Feb 13 20:12:36 UTC 2017 : Endpoint response headers: {x-amzn-Remapped-Content-Length=0, x-amzn-RequestId=c3354327-f228-11e6-8c1d-ed11cc413770, Connection=keep-alive, Content-Length=315, Date=Mon, 13 Feb 2017 20:12:36 GMT, Content-Type=application/json}
Mon Feb 13 20:12:36 UTC 2017 : Method response body after transformations: "Hello, stranger!"
Mon Feb 13 20:12:36 UTC 2017 : Method response headers: {x-powered-by=Express, access-control-allow-origin=*, content-type=application/json; charset=utf-8, content-length=18, etag=W/"12-E1p7iNXxJ4trMdmFBhlU9Q", date=Mon, 13 Feb 2017 20:12:36 GMT, connection=close, X-Amzn-Trace-Id=Root=1-58a21334-8ea6c4b5944eebb873bc7d2e}
Mon Feb 13 20:12:36 UTC 2017 : Successfully completed execution
Mon Feb 13 20:12:36 UTC 2017 : Method completed with status: 200
I think the response "Cannot GET /" is coming from your Lambda function itself. Can you check API Gateway CW logs (or Test Invoke feature in console) to see what's different in the integration request and response in the first call?
I didn't see any real documentation about it (just this Medium post) but I also experienced the fact that a Lambda can be frozen until the first invocation, or in the case it's not called for a long time.
A solution is to schedule a regular invocation to wake up your lambda, with Amazon CloudWatch Events
I know that is an old question, but if you use TypeORM (or more in general, if you wrap all your Express middlewares within a .then() callback of a Promise), and you use context.callbackWaitsForEmptyEventLoop = false in your lambda handler, maybe this could help you: https://github.com/typeorm/typeorm/issues/5894
Long story short: avoid to set that flag to false, if possible, otherwise avoid to wrap the Express middlewares within the .then() callback and, for instance, initialize your db connection in the first Express middleware.

HTTP request body not getting to AWS lambda function via AWS API Gateway

I have a very basic lambda function written in Scala deployed to AWS Lambda. The function works just fine when I test it via the AWS Lambda console.
Here's the function with some additional logging added for debug purposes.
package com.spacecorpshandbook.ostium.lambda.handler
import java.util
import com.google.gson.Gson
import temp.{ApiGatewayProxyResponse, Appointment, CancelResponse}
/**
* Amazon Lambda handler adapter for the Cancellation application
*/
class CancellationHandler {
def cancelAppointment(appointment: Appointment): ApiGatewayProxyResponse = {
System.out.println("++++ appointmentId is: " + appointment.getAppointmentId)
val apiGatewayProxyResponse = new ApiGatewayProxyResponse
val cancelResponse = new CancelResponse
cancelResponse.setMessage("Cancelled appointment with id " + appointment.getAppointmentId)
val gson: Gson = new Gson
apiGatewayProxyResponse.setBody(gson.toJson(cancelResponse))
apiGatewayProxyResponse.setStatusCode("200")
val headerValues = new util.HashMap[String, String]
headerValues put("Content-Type", "application/json")
apiGatewayProxyResponse.setHeaders(headerValues)
System.out.println("+++++ message before returning: " + apiGatewayProxyResponse.getBody)
apiGatewayProxyResponse
}
}
I was concerned that the POJO input/outputs being Scala beans might have been causing issues so I implemented Java versions temporarily just to rule that out.
The Integration Request on AWS API gateway is setup by default as Resources ANY with Lambda Proxy Integration enabled. Note that in this configuration when I test from the AWS API Gateway console the data is transformed and goes in but does't make it all the way to the lambda function
Execution log for request test-request
Fri Dec 09 11:14:40 UTC 2016 : Starting execution for request: test-invoke-request
Fri Dec 09 11:14:40 UTC 2016 : HTTP Method: PUT, Resource Path: /cancel-appointment
Fri Dec 09 11:14:40 UTC 2016 : Method request path: {}
Fri Dec 09 11:14:40 UTC 2016 : Method request query string: {}
Fri Dec 09 11:14:40 UTC 2016 : Method request headers: {Content-Type= application/json}
Fri Dec 09 11:14:40 UTC 2016 : Method request body before transformations: {
"applicationId": "asdfsfa"
}
Fri Dec 09 11:14:40 UTC 2016 : Endpoint request headers: {x-amzn-lambda-integration-tag=test-request, Authorization=****************************************************************************************************************************************************************************************************************************************************************************************************************************************5c044d, X-Amz-Date=20161209T111440Z, x-amzn-apigateway-api-id=l5tcmj0vlk, Accept=application/json, User-Agent=AmazonAPIGateway_l5tcmj0vlk, Host=lambda.us-east-1.amazonaws.com, X-Amz-Content-Sha256=857a062940a7fbb8134bad1c007e9975a10bd8323c39f6040e797a98e87ea1f6, X-Amzn-Trace-Id=Root=1-584a9220-9cd537954952cca7daee32bf, Content-Type=application/json}
Fri Dec 09 11:14:40 UTC 2016 : Endpoint request body after transformations: {"resource":"/cancel-appointment","path":"/cancel-appointment","httpMethod":"PUT","headers":{"Content-Type":" application/json"},"queryStringParameters":null,"pathParameters":null,"stageVariables":null,"requestContext":{"accountId":"456204981758","resourceId":"xznq3u","stage":"test-invoke-stage","requestId":"test-invoke-request","identity":{"cognitoIdentityPoolId":null,"accountId":"456204981758","cognitoIdentityId":null,"caller":"456204981758","apiKey":"test-invoke-api-key","sourceIp":"test-invoke-source-ip","accessKey":"ASIAJ5D7KU524H7CTTTQ","cognitoAuthenticationType":null,"cognitoAuthenticationProvider":null,"userArn":"arn:aws:iam::456204981758:root","userAgent":"Apache-HttpClient/4.5.x (Java/1.8.0_102)","user":"456204981758"},"resourcePath":"/cancel-appointment","httpMethod":"PUT","apiId":"l5tcmj0vlk"},"body":"{\n \"applicationId\": \"asdfsfa\"\n}","isBase64Encoded":false}
Fri Dec 09 11:14:40 UTC 2016 : Endpoint response body before transformations: {"statusCode":"200","headers":{"Content-Type":"application/json"},"body":"{\"message\":\"Cancelled appointment with id null\"}"}
Fri Dec 09 11:14:40 UTC 2016 : Endpoint response headers: {x-amzn-Remapped-Content-Length=0, x-amzn-RequestId=adcadf25-be00-11e6-8855-75e96d772946, Connection=keep-alive, Content-Length=128, Date=Fri, 09 Dec 2016 11:14:39 GMT, Content-Type=application/json}
Fri Dec 09 11:14:40 UTC 2016 : Method response body after transformations: {"message":"Cancelled appointment with id null"}
Fri Dec 09 11:14:40 UTC 2016 : Method response headers: {Content-Type=application/json, X-Amzn-Trace-Id=Root=1-584a9220-9cd537954952cca7daee32bf}
Fri Dec 09 11:14:40 UTC 2016 : Successfully completed execution
Fri Dec 09 11:14:40 UTC 2016 : Method completed with status: 200
If I add a specific method, like POST and do not set it as a Lambda Proxy Integration I do indeed see the that the provide request body data makes it to the lambda function, is correctly de-serialized into my POJO and is returned
Execution log for request test-request
Fri Dec 09 11:22:02 UTC 2016 : Starting execution for request: test-invoke-request
Fri Dec 09 11:22:02 UTC 2016 : HTTP Method: POST, Resource Path: /cancel-appointment
Fri Dec 09 11:22:02 UTC 2016 : Method request path: {}
Fri Dec 09 11:22:02 UTC 2016 : Method request query string: {}
Fri Dec 09 11:22:02 UTC 2016 : Method request headers: {}
Fri Dec 09 11:22:02 UTC 2016 : Method request body before transformations: {
"appointmentId" : "sfssdf"
}
Fri Dec 09 11:22:02 UTC 2016 : Endpoint request headers: {x-amzn-lambda-integration-tag=test-request, Authorization=****************************************************************************************************************************************************************************************************************************************************************************************************************************************a8dc41, X-Amz-Date=20161209T112202Z, x-amzn-apigateway-api-id=l5tcmj0vlk, Accept=application/json, User-Agent=AmazonAPIGateway_l5tcmj0vlk, Host=lambda.us-east-1.amazonaws.com, X-Amz-Content-Sha256=875dad4d4e05f8c12a7ca8aeaf69046d4153fc7f910e1eff1959cb011e8313a0, X-Amzn-Trace-Id=Root=1-584a93da-f841704d9feb371b31e41cb9, Content-Type=application/json}
Fri Dec 09 11:22:02 UTC 2016 : Endpoint request body after transformations: {
"appointmentId" : "sfssdf"
}
Fri Dec 09 11:22:02 UTC 2016 : Endpoint response body before transformations: {"statusCode":"200","headers":{"Content-Type":"application/json"},"body":"{\"message\":\"Cancelled appointment with id sfssdf\"}"}
Fri Dec 09 11:22:02 UTC 2016 : Endpoint response headers: {x-amzn-Remapped-Content-Length=0, x-amzn-RequestId=b4f5efce-be01-11e6-91c3-5b1e06f831e2, Connection=keep-alive, Content-Length=130, Date=Fri, 09 Dec 2016 11:22:02 GMT, Content-Type=application/json}
Fri Dec 09 11:22:02 UTC 2016 : Method response body after transformations: {"statusCode":"200","headers":{"Content-Type":"application/json"},"body":"{\"message\":\"Cancelled appointment with id sfssdf\"}"}
Fri Dec 09 11:22:02 UTC 2016 : Method response headers: {X-Amzn-Trace-Id=Root=1-584a93da-f841704d9feb371b31e41cb9, Content-Type=application/json}
Fri Dec 09 11:22:02 UTC 2016 : Successfully completed execution
Fri Dec 09 11:22:02 UTC 2016 : Method completed with status: 200
So everything looks great now, however when I actually do a real test against the AWS API URL from PostMan using HTTP method POST I get the response with null as the appointment Id and I can see in the CloudWatch logs that the appointmentId did not get set on the input Appointment object.
I feel like I'm missing something basic here. Any help would be greatly appreciated.
source code can be found here
Update
Resolved this problem by switching the lambda handler function to use Stream rather than attempting to serialize/deserialize JSON into POJOs. When using the API Gateway Lambda Proxy the input to handler is a complicated JSON structure that I didn't want to try and replicate as a Java/Scala class. It was easier to process the input as a stream, parse it into a JsonObject, and then convert the body of the message into my POJO using Gson or equivalent library. Sample handler below, you can also see a larger example here
class CancellationHandler {
def cancelAppointment(request: InputStream, response: OutputStream, context: Context): Unit = {
val logger = context.getLogger
val parser: JsonParser = new JsonParser
var inputObj: JsonObject = null
val gson: Gson = new Gson
try {
inputObj = parser.parse(IOUtils.toString(request, "UTF-8")).getAsJsonObject
} catch {
case e: IOException =>
logger.log("Error while reading request\n" + e.getMessage)
throw new RuntimeException(e.getMessage)
}
val body: String = inputObj.get("body").getAsString
val appointment: Appointment = gson.fromJson(body, classOf[Appointment])
val apiGatewayProxyResponse = new ApiGatewayProxyResponse
val cancelResponse = new CancelResponse
cancelResponse.setMessage("Cancelled appointment with id " + appointment.getAppointmentId)
apiGatewayProxyResponse.setBody(gson.toJson(cancelResponse))
apiGatewayProxyResponse.setStatusCode("200")
val headerValues = new util.HashMap[String, String]
headerValues put("Content-Type", "application/json")
apiGatewayProxyResponse.setHeaders(headerValues)
val output: String = gson.toJson(apiGatewayProxyResponse)
IOUtils.write(output, response, "UTF-8")
}
}
The input shape for the Lambda proxy will be different than the shape for the regular non-proxy Lambda integration. This is important for your use case of course because you're using Java/Scala where you have to explicitly structure the input POJO.
Here's what the proxy input will look like:
{
"resource": "\/pets",
"path": "\/pets",
"httpMethod": "POST",
"headers": null,
"queryStringParameters": null,
"pathParameters": null,
"stageVariables": null,
"requestContext": {
...
"stage": "test-invoke-stage",
"requestId": "test-invoke-request",
"identity": {
...
},
"resourcePath": "\/pets",
"httpMethod": "POST"
},
"body": "{\n \"foo\":\"bar\"\n}", <---- here's what you're looking for
"isBase64Encoded": false
}
Docs: http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-create-api-as-simple-proxy-for-lambda.html

AWS ApiGateway request path mapping + lambda

I know there are a lot of questions about mapping the request data, but neither one helped me.
So, what i am trying to achieve is an API endpoint mapped to a lambda. The request to that endpoint is forwarded when a bucket triggers a 404, and the parameters are passed to the lambda via the request path, like: /{image_name}/{width}/{height}.
My lambda's code simply calls context.succeed(event, context);
In the Method request configuration the request path's parameters were automatically created, .
In the integration request I have created three mapping templates: plain/text, plain/html, application/json with the same definition as bellow:
#set($inputRoot = $input.path('$'))
{
"name": $input.params('name'),
"width" : $input.params('width'),
"height" : $input.params('height'),
"params": $input.params(),
"resourcePath": $context.resourcePath,
}
When calling form an chrome rest client i get:
When calling the test from the console, i get the following response:
{"Type":"User","message":"Could not parse request body into json."}
The same response i get when I call curl or when I simply open the URL in the browser.
But in the logs from the console's test call I see:
Execution log for request test-request
Tue Sep 08 09:10:20 UTC 2015 : Starting execution for request: test-invoke-request
Tue Sep 08 09:10:20 UTC 2015 : API Key: test-invoke-api-key
Tue Sep 08 09:10:20 UTC 2015 : Method request path: {name=name, width=100, height=100}
Tue Sep 08 09:10:20 UTC 2015 : Method request query string: {}
Tue Sep 08 09:10:20 UTC 2015 : Method request headers: {}
Tue Sep 08 09:10:20 UTC 2015 : Method request body before transformations: null
Tue Sep 08 09:10:20 UTC 2015 : Endpoint request URI: <endpoint>:function:Magic/invocations
Tue Sep 08 09:10:20 UTC 2015 : Endpoint request headers: {
Authorization=<authorization>
Credential=<credential>,
SignedHeaders=accept;content-type;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-source-arn,
Signature=<signature>,
X-Amz-Date=20150908T091020Z,
X-Amz-Source-Arn=<ARN>/null/GET/image/{name}/{width}/{height},
Accept=application/json,
User-Agent=AmazonAPIGateway_ebkkwbbpo0,
Host=lambda.us-east-1.amazonaws.com,
X-Amz-Content-Sha256=<key>,
Content-Type=application/json
}
Tue Sep 08 09:10:20 UTC 2015 : Endpoint request body after transformations: {
"name": name,
"width" : 100,
"height" : 100,
"params": {path={name=name, width=100, height=100}, querystring={}, header={}},
"resourcePath": /image/{name}/{width}/{height},
}
Tue Sep 08 09:10:20 UTC 2015 : Endpoint response body before transformations: {"Type":"User","message":"Could not parse request body into json."}
Tue Sep 08 09:10:20 UTC 2015 : Endpoint response headers: {
x-amzn-ErrorType=InvalidRequestContentException:http://internal.amazon.com/coral/com.amazonaws.awsgirapi/,
x-amzn-RequestId=<RequestId>,
Connection=keep-alive,
Content-Length=68,
Date=Tue, 08 Sep 2015 09:10:20 GMT,
Content-Type=application/json}
Tue Sep 08 09:10:20 UTC 2015 : Method response body after transformations: {"Type":"User","message":"Could not parse request body into json."}
Tue Sep 08 09:10:20 UTC 2015 : Method response headers: {Content-Type=application/json}
Tue Sep 08 09:10:20 UTC 2015 : Successfully completed execution
As I see at some point, the URL path is parsed correctly, but I do not know what goes wrong.
Also, I don't know why there is in the X-Amz-Source-Arn a null value in the path.
Thank you.
The problem is the integration request mapping template. You should double quote the fields that are string type, so they can later be converted to JSON.
So in this example you should write:
#set($inputRoot = $input.path('$'))
{
"name": "$input.params('name')",
"width" : $input.params('width'),
"height" : $input.params('height'),
"params": "$input.params()",
"resourcePath": "$context.resourcePath",
}
It seemed odd to me, but this is the solution.
Also you don't need to write three mapping templates for this case, you should leave only the application/json
In case of lambda integration with path parameters, the path parameters should be mapped in the Integration request as follows.
Go to Integration Response -> Mapping Templates and add the following mapping of the path parameter to input values:
{ "itemId": "$input.params('catalogitemid')"}