AWS cloudwatch is truncating logs - amazon-web-services

I have created an API using AWS api gateway. Under stages the "Log full requests/responses data" checked and "Enable CloudWatch Logs" is also checked.
When i look at the logs in cloudwatch i see some of the logs are "TRUNCATED". In-fact all of the logs are truncating request and response body. Is there any way to view the entire request/response.
Since there will be multiple integration points it make sense to see the entire logs.

Looks like it's one of the known issues in AWS API Gateway.
API Gateway currently limits log events to 1024 bytes. Log events
larger than 1024 bytes, such as request and response bodies, will be
truncated by API Gateway before submission to CloudWatch Logs.

API Gateway limits log events to 1024 bytes and cannot be increased. The log events larger than 1024 bytes, such as request and response bodies, will be truncated by API Gateway before submission to CloudWatch Logs.
A workaround can be using Lambda proxy integration with API Gateway.
With Lambda proxy integration, API Gateway passes the request as is to the integrated Lambda function with only exception that the order of the request parameters is not preserved.
This request data includes the request headers, query string parameters, URL path variables, payload, body and request context. As Lambda does not truncate log entry, all headers and query string parameters are logged in Lambda's CloudWatch log and can be seen.
Downside of this approach is that lambda would add to the cost.
Read about API Gateway CloudWatch logs here : https://cloudnamaste.com/api-gateway-cloudwatch-logs/

Related

How to handle lambda retries invoked from api gateway?

The api gateway url is triggered/called by a form submit. Now, as this is a synchronous invocation, how do I handle a lambda retries and handle throttles?
Note: I am deploying api gateway along with lambda and use the url generated as webhook for form submit. So basically, it would be a one-way communication.
Flow:
form(payload)-> (api gateway)-> lambda-> lambda-> sqs-> lambda-> dynamoDb
There won't be an automatic Lambda retry when it's invoked from API Gateway afaik. If you are throttled or the request fails (perhaps because of a Lambda function error), your client is responsible for deciding how to recover and whether or not to retry, likely based on the HTTP response code.
Also worth reading:
A Detailed Overview of AWS API Gateway
How AWS Lambda Retry really works

Getting '502 Bad Gateway' from AWS api gateway

I am getting below response when I hit AWS API gateway.
502: Bad Gateway
{
"message": "Internal server error"
}
I know that API gateway request payload limit is 10MB and I haven't exceeded that, but response size might be beyond 10 MB. Can response size affect?
Could find nothing in CloudWatch, not even getting logs related to this invocation.
Tried to follow this, but could find nothing.
First, Try to test your lambda function by invoking it in the api gateway Console or in the test part in the lambda console.
If the lambda function works, then maybe its a problem related to the parameters, content ( Integration request / response) or the permissions of the API Gateway + lambda.
Check also the time out of the lambda /!\

Schedule API Gateway requests with CloudWatch?

I have an API (through API Gateway) where each resource's method is routed to a single lambda function via lambda proxy integration, where each request is processed internally.
How can I automate a scheduled invocation of one resource's method with a CloudWatch event? This preferably will not be done by directly invoking the lambda function, and would allow a specified (constant) input.
Since CloudWatch Events does not support invoking an HTTP endpoint directly, you should be able to accomplish this by using SNS with a HTTP subscription. The workflow would be the following:
Scheduled CloudWatch Event -> SNS endpoint with constant JSON payload -> HTTP subscription, where the HTTP subscription is your API Gateway endpoint.

Trace full-request / response bodies in AWS ApiGateway (not truncated)

I am using AWS Api Gateway and I'd like to trace full request and response. Some of my integrations are lambdas and some other http endpoints.
I enabled stage > Logs > "Log full requests/responses data" and I can see logs on CloudWatch.
This seemed to be perfect until I discovered bodies were being TRUNCATED... There is a limitation of 1024 bytes in ApiGateway sending logs to cloudwatch.
Is there any solution to this?
I am considering using a lambda as a proxy (with http-proxy) as my last option...
As you stated, there is no current known solution to this problem and AWS is aware of the problem.
API Gateway currently limits log events to 1024 bytes. Log events larger than 1024 bytes, such as request and response bodies, will be truncated by API Gateway before submission to CloudWatch Logs.
You can see additional known issues at the AWS documentation page for API Gateway here.
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-known-issues.html
According to the AWS documentation, full logging should not be turned on in AWS API Gateway, in production, because of PII (source).
This is purely for development troubleshooting purposes. So the log limitation 1024 bytes continue to exists, because of this.

AWS API Gateway default response and Trigger AWS Lambda

I have been experimenting with AWS API Gateway and AWS Lambda to try out a serverless architecture. Have been going through blogs and AWS documentation. Have tried out sample GET/POST. But, I have the following requirement w.r.t tracking user events from my custom application
Events are posted from my application to API end point
I wanted the API to respond back with a custom response (Say {'fine'})
(acknowledging that the request has been received)
After the response is sent, hand over the event payload to a AWS Lambda function
As per the documentation, I understand,
a) I can post events to API end point
b) On GET/POST trigger an AWS Lambda Function
- Respond back from AWS Lambda function to API request
I wanted to change the above and modify it to
a) Post events to API end point
a.0) Respond back acknowledging that request is received [Say {'fine'} ]
b) Trigger AWS Lambda function to process the event payload
Please share across suggestions on how to achieve the same.
Another asynchronous model many customers have used:
Set up an API configured to send requests to Amazon Kinesis. This API could acknowledge the request.
Set up AWS Lambda to consume your Kinesis stream.
This setup has some advantages for high workload APIs as fetches from the Kinesis stream can be batched and don't require a 1-to-1 scaling of both your API Gateway limits and Lambda limits.
Update
To answer your questions about scalability:
Kinesis
Kinesis scales by adding what it calls "shards" to the stream. Each shard handles a portion of your traffic, based on a partition key. Each shard scales up to 1000 rps or 1MBps (see limits). Even with the lower default 25 shards, this would support up to 25,000 rps or 25MBps with an evenly distributed partition key.
API Gateway
API Gateway has a default account level limit of 500 rps, but this can easily be extended by requesting a limit increase. We have customers in production that are using the service at limits above your current suggested scale.
If you want a fast response from the API and not have to wait for the processing of data, you could:
post an event to an API Gateway endpoint
trigger an AWS Lambda Function A
call asynchronously a Lambda Function B using the AWS SDK in the Lambda Function A
Call context.succeed() or context.done() or the callback function in the Lambda Function A so it respond back to API Gateway
the Lambda Function B can process the data while API Gateway already received a response
You should first run some tests to see what type of real world response times you are getting from having your lambda function complete all the logic. If the times are above what you feel are acceptable for your use case, here is another asynchronous solution utilizing an SNS Topic to trigger a secondary Lambda function.
Client Request to API Gateway -> Calls Lambda function A
Lambda A verifies payload and then publishes to SNS Topic X
Lambda A returns {fine} success message -> API Gateway -> client
SNS Topic X triggers Lambda function B
Lambda function B implements given logic