I was trying the integration of Amazon API gateway with Lambda function. I was successfully able to achieve though but when I tried with curl it fails.
Lambda method, API gateway integration along with template mapping under integration request are setup.
When I run "test" from console, it works fine
Execution log for request test-request
Wed Nov 04 07:27:30 UTC 2015 : Starting execution for request: test-invoke-request
Wed Nov 04 07:27:30 UTC 2015 : API Key: test-invoke-api-key
Wed Nov 04 07:27:30 UTC 2015 : Method request path: {service=xml}
Wed Nov 04 07:27:30 UTC 2015 : Method request query string: {}
Wed Nov 04 07:27:30 UTC 2015 : Method request headers: {}
Wed Nov 04 07:27:30 UTC 2015 : Method request body before transformations: Articletext
Wed Nov 04 07:27:30 UTC 2015 : Endpoint request body after transformations: {
"prog" : "xml",
"content" : "Articletext",
"test" : "{path={service=xml}, querystring={}, header={}}"
}
but when I tries to access it through curl it is not able to create proper body after transformation
curl -X POST
https://aaaaaaa.execute-api.us-west-2.amazonaws.com/beta/apitest/xml
-H "Content-Type: application/x-www-form-urlencoded; charset=UTF-8" --data-binary "Articletext"
Starting execution for request:
Method request path: {service=xml}
Method request query string:
{}
Method request body before transformations: Articletext
Endpoint request body after transformations: Articletext
Endpoint response body before transformations:
{
"Type": "User",
"message": "Could not parse request body into json."
}
Can any one suggest me about missing part?
In my case the problem was that I did not Deploy the API, so it was not updated for external use.
See Amazon API Gateway : response body is not transformed when the API is called via Postman?
Please make sure your content-type what you defined in integration request template.
You can try this curl command.
curl -X POST https://aaaaaaa.execute-api.us-west-2.amazonaws.com/beta/apitest/xml -H "Content-Type: application/json" --data-binary "Articletext"
Related
I am experimenting with an AWS API-Gateway integration with an S3 backend. I have noticed the switch between different mapping-templates in the integration-response does not appear to work.
In integration-response, I have the following mapping templates:
application/json:
{
type: "JSON",
body: "$input.body"
}
text/plain:
PlainText:
$input.body
Nothing I can do appears to be able to make the text/plain mapping template to be used - it always uses application/json.
I would expect that the text/plain mapping template to be used based on one of the following being true:
S3 returns content with Content-Type: text/plain.
The initial request to API-Gateway passes an Accept: text/plain header.
As per the example below, S3 returns a Content-Type: text/plain AND I request with Accept: text/plain. API-Gateway correctly responds with Content-Type: text/plain also.
However the application/json template is still used to transform the body.
I have even removed application/json as a valid response type from the method-response entirely, but still nothing.
Any thoughts why this is happening?
FYI I am using a classic v1 ApiGateway (Rest).
Execution log for request c24cfce3-2cf3-4693-ad72-4fdf44f4fdcd
Wed May 19 17:13:38 UTC 2021 : Starting execution for request: c24cfce3-2cf3-4693-ad72-4fdf44f4fdcd
Wed May 19 17:13:38 UTC 2021 : HTTP Method: GET, Resource Path: /feeds-poc/test.txt
Wed May 19 17:13:38 UTC 2021 : Method request path: {filename=test.txt}
Wed May 19 17:13:38 UTC 2021 : Method request query string: {}
Wed May 19 17:13:38 UTC 2021 : Method request headers: {Accept=text/plain}
Wed May 19 17:13:38 UTC 2021 : Method request body before transformations:
Wed May 19 17:13:38 UTC 2021 : Endpoint request URI: https://my-bucket-id.s3.eu-west-1.amazonaws.com/test.txt
Wed May 19 17:13:38 UTC 2021 : Endpoint request headers: {Authorization=****57173a, X-Amz-Date=20210519T171338Z, x-amzn-apigateway-api-id=123456789a, Accept=application/json, User-Agent=AmazonAPIGateway_123456789a, X-Amz-Security-Token=**** [TRUNCATED]
Wed May 19 17:13:38 UTC 2021 : Endpoint request body after transformations:
Wed May 19 17:13:38 UTC 2021 : Sending request to https://my-bucket-id.s3.eu-west-1.amazonaws.com/test.txt
Wed May 19 17:13:38 UTC 2021 : Received response. Status: 200, Integration latency: 43 ms
Wed May 19 17:13:38 UTC 2021 : Endpoint response headers: {x-amz-id-2=****, x-amz-request-id=****, Date=Wed, 19 May 2021 17:13:39 GMT, Last-Modified=Mon, 17 May 2021 16:45:13 GMT, ETag="420f804aa21220bf0db57bb4b9799c8a", Accept-Ranges=bytes, Content-Type=text/plain, Content-Length=13, Server=AmazonS3}
Wed May 19 17:13:38 UTC 2021 : Endpoint response body before transformations: It's working
Wed May 19 17:13:38 UTC 2021 : Method response body after transformations: {
type: "JSON",
body: "It's working
"
}
Wed May 19 17:13:38 UTC 2021 : Method response headers: {X-Amzn-Trace-Id=Root=****, Content-Type=text/plain}
Wed May 19 17:13:38 UTC 2021 : Successfully completed execution
Wed May 19 17:13:38 UTC 2021 : Method completed with status: 200
AWS Support confirmed to me that the switch is based on the original request Accept header. Testing this using Postman confirms this functionality.
The test tools built into the AWS console for API-Gateway however do not appear to cater for testing response mapping templates - from AWS support: the purpose of API Gateway test console is only to test the Integration, it doesn't work for end-to-end request.
I would expect it to send my request to http://<removed>/search/qweqweqweqwe, not to http://<removed>/search/{searchString}. What am I doing wrong?
Mon May 06 13:18:59 UTC 2019 : HTTP Method: GET, Resource Path: /search/qweqweqweqwe
Mon May 06 13:18:59 UTC 2019 : Method request path: {searchString=qweqweqweqwe}
Mon May 06 13:18:59 UTC 2019 : Method request query string: {}
Mon May 06 13:18:59 UTC 2019 : Method request headers: {}
Mon May 06 13:18:59 UTC 2019 : Method request body before transformations:
Mon May 06 13:19:00 UTC 2019 : Endpoint request URI: http://<removed>/search/{searchString}
Mon May 06 13:19:00 UTC 2019 : Endpoint request headers: {x-amzn-apigateway-api-id=bx0tw55ts0, User-Agent=AmazonAPIGateway_bx0tw55ts0}
Mon May 06 13:19:00 UTC 2019 : Endpoint request body after transformations:
Mon May 06 13:19:00 UTC 2019 : Sending request to http://<removed>/search/{searchString}
Apparently if you are using http_proxy on API Gateway, you have to create a URL Path Parameter in the Integration Request section. Once I added searchString as the variable, and method.request.path.searchString, it worked as expected.
Now I just have to figure out how I can define that in the swagger api doc and have it work on import.
I have a Java based Lambda function that is running correctly via the Lambda test event with the following JSON:
{
"married": "true",
"wages": "200000",
"homeInterest": "15000",
"propertyTaxes": "15000",
"stateTaxes": "13000",
"otherDeductions": "4000",
"postalCode": "11762"
}
I then created an API via the Amazon API Gateway. When I paste the same JSON as the body of the generated URL none of fields map correctly.
The Lambda handler is using POJOs for the request and response:
public class TaxHandler implements RequestHandler<TaxRequest, TaxResponse>{
public TaxResponse handleRequest(TaxRequest request, Context context){
When I test via the API gateway test I see the following info:
Execution log for request test-request
Tue Dec 19 02:01:16 UTC 2017 : Starting execution for request: test-invoke-request
Tue Dec 19 02:01:16 UTC 2017 : HTTP Method: POST, Resource Path: /TaxCalculation
Tue Dec 19 02:01:16 UTC 2017 : Method request path: {}
Tue Dec 19 02:01:16 UTC 2017 : Method request query string: {}
Tue Dec 19 02:01:16 UTC 2017 : Method request headers: {}
Tue Dec 19 02:01:16 UTC 2017 : Method request body before transformations: {
"married": "true",
"wages": "200000",
"homeInterest": "15000",
"propertyTaxes": "15000",
"stateTaxes": "13000",
"otherDeductions": "4000",
"postalCode": "11762"
}
Tue Dec 19 02:01:16 UTC 2017 : Endpoint request URI: https://lambda.us-east-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-1:896795400074:function:TaxCalculation/invocations
Tue Dec 19 02:01:16 UTC 2017 : Endpoint request headers: {x-amzn-lambda-integration-tag=test-request, Authorization=************************************************************************************************************************************************************************************************************************************************************************************************************************7fcbc9, X-Amz-Date=20171219T020116Z, x-amzn-apigateway-api-id=07yp3njqzk, X-Amz-Source-Arn=arn:aws:execute-api:us-east-1:896795400074:07yp3njqzk/null/POST/TaxCalculation, Accept=application/json, User-Agent=AmazonAPIGateway_07yp3njqzk, X-Amz-Security-Token= [TRUNCATED]
Tue Dec 19 02:01:16 UTC 2017 : Endpoint request body after transformations: {"resource":"/TaxCalculation","path":"/TaxCalculation","httpMethod":"POST","headers":null,"queryStringParameters":null,"pathParameters":null,"stageVariables":null,"requestContext":{"path":"/TaxCalculation","accountId":"xxxxxxxx","resourceId":"v8358d","stage":"test-invoke-stage","requestId":"test-invoke-request","identity":{"cognitoIdentityPoolId":null,"cognitoIdentityId":null,"apiKey":"test-invoke-api-key","cognitoAuthenticationType":null,"userArn":"arn:aws:iam::xxxxx:user/tvfoodmaps_aws","apiKeyId":"test-invoke-api-key-id","userAgent":"Apache-HttpClient/4.5.x (Java/1.8.0_144)","accountId":"896795400074","caller":"AIDAINMSXKH5AWAQ7NX36","sourceIp":"test-invoke-source-ip","accessKey":"ASIAIHXWW4BOHGXESRNQ","cognitoAuthenticationProvider":null,"user":"AIDAINMSXKH5AWAQ7NX36"},"resourcePath":"/TaxCalculation","httpMethod":"POST","apiId":"07yp3njqzk"},"body":"{\n \"married\": \"true\",\n \"wages\": \"200000\",\n \"homeInterest\": \"15000\",\n \"propertyTa [TRUNCATED]
Tue Dec 19 02:01:16 UTC 2017 : Sending request to https://lambda.us-east-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-1:896795400074:function:TaxCalculation/invocations
Tue Dec 19 02:01:16 UTC 2017 : Received response. Integration latency: 30 ms
Tue Dec 19 02:01:16 UTC 2017 : Endpoint response body before transformations: {"savings":565.0,"owedTaxes17":-635.0,"owedTaxes18":-1200.0,"effectiveRate17":0.0,"effectiveRate18":0.0}
Tue Dec 19 02:01:16 UTC 2017 : Endpoint response headers: {X-Amz-Executed-Version=$LATEST, x-amzn-Remapped-Content-Length=0, Connection=keep-alive, x-amzn-RequestId=7f8a1b13-e460-11e7-84cf-d1c3e8d3eaf5, Content-Length=104, Date=Tue, 19 Dec 2017 02:01:16 GMT, X-Amzn-Trace-Id=root=1-5a3872ec-1b9d875d8cc2fded5c30da46;sampled=0, Content-Type=application/json}
Tue Dec 19 02:01:16 UTC 2017 : Execution failed due to configuration error: Malformed Lambda proxy response
Tue Dec 19 02:01:16 UTC 2017 : Method completed with status: 502
How can I further debug why the function is not executing the same when called via REST instead of directly via the Lambda tester? Note: I know the error talks about the response but the issue is that the first line of my code reading the fields that should be mapped to the pojo aren't working (again, only when using the API).
Looks like you have your integration with ANY and not returning the proxy response. Instead you are returning the JSON object response.
Similar problem discussed here,
https://forums.aws.amazon.com/thread.jspa?threadID=255561
And the solution to configure is documented here,
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-set-up-simple-proxy.html#api-gateway-simple-proxy-for-lambda-output-format
Hope it helps.
You can include logging in your code so that they will be available in AWS CloudWatch to investigate the issue. Following libraries and approaches are recommended by AWS according to the documentation for Java.
Custom Appender for Log4j™ 2
LambdaLogger.log(): Example 2: Writing Logs Using LambdaLogger (Java)
System.out() and System.err()
Note: Also setup API Gateway logging so that it would be easier to trace any issue end to end.
I'm moving some scala code to AWS Lambda and I intend to have it exposed via AWS API Gateway, but I've been struggling to make the whole thing work as soon as I have one parameter.
My (very simple) code looks like this:
class HelloService {
def hello(name: String) = {
"hello there, " + name
}
}
I uploaded the built jar to Lambda and tested it in the AWS console by creating a test event. It returns the right response, as expected.
However, I want this Lambda to be invoked by the API Gateway. I've used both Lambda Proxy Integration and also defined my own Body Mapping Templates. I can't seem to make it work and I keep getting:
{
"message": "Internal server error"
}
with logs:
Execution log for request test-request Mon Jul 03 16:23:21 UTC 2017 :
Starting execution for request: test-invoke-request Mon Jul 03
16:23:21 UTC 2017 : HTTP Method: GET, Resource Path: /car/aaa Mon Jul
03 16:23:21 UTC 2017 : Method request path: {carReg=aaa} Mon Jul 03
16:23:21 UTC 2017 : Method request query string: {} Mon Jul 03
16:23:21 UTC 2017 : Method request headers: {} Mon Jul 03 16:23:21 UTC
2017 : Method request body before transformations: Mon Jul 03
16:23:21 UTC 2017 : Endpoint request URI:
https://lambda.eu-west-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:eu-west-1:879461422967:function:getCarData/invocations
Mon Jul 03 16:23:21 UTC 2017 : Endpoint request headers:
{x-amzn-lambda-integration-tag=test-request,
Authorization=****************************************************************************************************************************************************************************************************************************************************************************************************************************************f8c749, X-Amz-Date=20170703T162321Z, x-amzn-apigateway-api-id=9dwaaf2mdg,
X-Amz-Source-Arn=arn:aws:execute-api:eu-west-1:879461422967:9dwaaf2mdg/null/GET/car/{carReg+},
Accept=application/json, User-Agent=AmazonAPIGateway_9dwaaf2mdg,
X-Amz-Security-Token=FQoDYXdzENn//////////wEaDMO73KD0CHVmggvYvSK3A8H1fpDgYiNK3HDD3ESe1aKYbv1HlGSQ85at3gRGA3kunmxVCxWbXNqR4ojBCn4hvBzdv1/iWD9xRzZQEtnQeDoO9NTuiBdYaXKgwjGozPKF/46X71f0sCt/Mm9i8EDtt3igEezJIhAF3OvYcdv2NBF3L0mRMMQKp4Vy+aC0mKu4ggadyLe+KYvmch8/AiZPlrxC1AtqwNGyWpSe1JqxeEXQGXIA5JsfwGpnpAB5IUec2r3Bd09zUFk/DCC80l9d4BLnhYAUn7xzrKYzisSEQitmhnTR3HijEYE6AJzJjFR+z2PqqVKvtgKQ
[TRUNCATED] Mon Jul 03 16:23:21 UTC 2017 : Endpoint request body after
transformations: { "message" : "foo" } Mon Jul 03 16:23:21 UTC 2017
: Endpoint response body before transformations: {"errorMessage":"An
error occurred during JSON
parsing","errorType":"java.lang.RuntimeException","stackTrace":[],"cause":{"errorMessage":"com.fasterxml.jackson.databind.JsonMappingException:
Can not deserialize instance of java.lang.String out of START_OBJECT
token\n at [Source:
lambdainternal.util.NativeMemoryAsInputStream#e720b71; line: 1,
column:
1]","errorType":"java.io.UncheckedIOException","stackTrace":[],"cause":{"errorMessage":"Can
not deserialize instance of java.lang.String out of START_OBJECT
token\n at [Source:
lambdainternal.util.NativeMemoryAsInputStream#e720b71; line: 1,
column:
1]","errorType":"com.fasterxml.jackson.databind.JsonMappingException","stackTrace":["com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)","com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:857)","com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java
[TRUNCATED] Mon Jul 03 16:23:21 UTC 2017 : Endpoint response headers:
{x-amzn-Remapped-Content-Length=0,
x-amzn-RequestId=ede9aaed-600b-11e7-834e-47baf0a4e23f,
Connection=keep-alive, Content-Length=1252,
X-Amz-Function-Error=Unhandled, Date=Mon, 03 Jul 2017 16:23:20 GMT,
X-Amzn-Trace-Id=root=1-595a6f79-c065d6038ba3209743378112;sampled=0,
Content-Type=application/json} Mon Jul 03 16:23:21 UTC 2017 :
Execution failed due to configuration error: Output mapping refers to
an invalid method response: 200 Mon Jul 03 16:23:21 UTC 2017 : Method
completed with status: 500
There's a null in the path of my ARN, but I guess that's because I have no authentication set, which is what's intended at this point. I don't think this would be the cause of the error.
Other than that, I've tried defining the Content-Type for the body both as application/json and text/plain. None seems to work and even with text/plain, AWS seems to be expecting json. I'd expected a string to be valid json anyway.
What am I doing wrong? What's the full expression I should put in my Body Mapping template? And how should the schema definition in my Model look like? I don't seem to be able to define a proper model for plain text.
I'm sure this is something really simple and I'm just missing something...
The response you send back to the server depends on whether you're using the Lambda proxy integration or not. Using proxy integration is easier to set up on the API Gateway side, but your Lambda needs to do a little more work because the gateway is going to send you a bunch of stuff and demand a specific format in the response. For proxy integration the response format needs to look like this:
{
"isBase64Encoded": true|false,
"statusCode": httpStatusCode,
"headers": { "headerName": "headerValue", ... },
"body": "..."
}
There's more here: http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-set-up-simple-proxy.html#api-gateway-set-up-lambda-proxy-integration-on-proxy-resource
If you are not using proxy integration you will need to setup body mapping for your parameters in the API Gateway integration for the HTTP verb in question to match the api parameters to the lambda parameters. There's a good explanation in the accepted answer here: How to pass a querystring or route parameter to AWS Lambda from Amazon API Gateway
With Lambda Proxy, you have to return a stringified JSON.
With Lambda, you can return JSON from your Lambda and then have your API Gateway body mapping template stringify it for you.
For additional context and example, see this page from Serverless docs.
I know there are a lot of questions about mapping the request data, but neither one helped me.
So, what i am trying to achieve is an API endpoint mapped to a lambda. The request to that endpoint is forwarded when a bucket triggers a 404, and the parameters are passed to the lambda via the request path, like: /{image_name}/{width}/{height}.
My lambda's code simply calls context.succeed(event, context);
In the Method request configuration the request path's parameters were automatically created, .
In the integration request I have created three mapping templates: plain/text, plain/html, application/json with the same definition as bellow:
#set($inputRoot = $input.path('$'))
{
"name": $input.params('name'),
"width" : $input.params('width'),
"height" : $input.params('height'),
"params": $input.params(),
"resourcePath": $context.resourcePath,
}
When calling form an chrome rest client i get:
When calling the test from the console, i get the following response:
{"Type":"User","message":"Could not parse request body into json."}
The same response i get when I call curl or when I simply open the URL in the browser.
But in the logs from the console's test call I see:
Execution log for request test-request
Tue Sep 08 09:10:20 UTC 2015 : Starting execution for request: test-invoke-request
Tue Sep 08 09:10:20 UTC 2015 : API Key: test-invoke-api-key
Tue Sep 08 09:10:20 UTC 2015 : Method request path: {name=name, width=100, height=100}
Tue Sep 08 09:10:20 UTC 2015 : Method request query string: {}
Tue Sep 08 09:10:20 UTC 2015 : Method request headers: {}
Tue Sep 08 09:10:20 UTC 2015 : Method request body before transformations: null
Tue Sep 08 09:10:20 UTC 2015 : Endpoint request URI: <endpoint>:function:Magic/invocations
Tue Sep 08 09:10:20 UTC 2015 : Endpoint request headers: {
Authorization=<authorization>
Credential=<credential>,
SignedHeaders=accept;content-type;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-source-arn,
Signature=<signature>,
X-Amz-Date=20150908T091020Z,
X-Amz-Source-Arn=<ARN>/null/GET/image/{name}/{width}/{height},
Accept=application/json,
User-Agent=AmazonAPIGateway_ebkkwbbpo0,
Host=lambda.us-east-1.amazonaws.com,
X-Amz-Content-Sha256=<key>,
Content-Type=application/json
}
Tue Sep 08 09:10:20 UTC 2015 : Endpoint request body after transformations: {
"name": name,
"width" : 100,
"height" : 100,
"params": {path={name=name, width=100, height=100}, querystring={}, header={}},
"resourcePath": /image/{name}/{width}/{height},
}
Tue Sep 08 09:10:20 UTC 2015 : Endpoint response body before transformations: {"Type":"User","message":"Could not parse request body into json."}
Tue Sep 08 09:10:20 UTC 2015 : Endpoint response headers: {
x-amzn-ErrorType=InvalidRequestContentException:http://internal.amazon.com/coral/com.amazonaws.awsgirapi/,
x-amzn-RequestId=<RequestId>,
Connection=keep-alive,
Content-Length=68,
Date=Tue, 08 Sep 2015 09:10:20 GMT,
Content-Type=application/json}
Tue Sep 08 09:10:20 UTC 2015 : Method response body after transformations: {"Type":"User","message":"Could not parse request body into json."}
Tue Sep 08 09:10:20 UTC 2015 : Method response headers: {Content-Type=application/json}
Tue Sep 08 09:10:20 UTC 2015 : Successfully completed execution
As I see at some point, the URL path is parsed correctly, but I do not know what goes wrong.
Also, I don't know why there is in the X-Amz-Source-Arn a null value in the path.
Thank you.
The problem is the integration request mapping template. You should double quote the fields that are string type, so they can later be converted to JSON.
So in this example you should write:
#set($inputRoot = $input.path('$'))
{
"name": "$input.params('name')",
"width" : $input.params('width'),
"height" : $input.params('height'),
"params": "$input.params()",
"resourcePath": "$context.resourcePath",
}
It seemed odd to me, but this is the solution.
Also you don't need to write three mapping templates for this case, you should leave only the application/json
In case of lambda integration with path parameters, the path parameters should be mapped in the Integration request as follows.
Go to Integration Response -> Mapping Templates and add the following mapping of the path parameter to input values:
{ "itemId": "$input.params('catalogitemid')"}