I have custom resolvers on appsync that are handled by lambda functions, I can keep track of errors logs created by the lambda functions by using cloudwatch and creating alarms based on these logs. But when client applications tries to request fields that don't exist on the graphql schema(see image below) the lambda runs fine so none error log its created, how can I keep track of these kind of errors using cloudwatch?
The good news is that you can turn on AppSync request logging to get request-level logs in CloudWatch. Configure logging using Cloudformation/CDK LogConfig or directly in the AppSync console.
The bad news is that the logged events do not seem to call out failed validations. Both well-formed and malformed requests are logged similarly, it seems. Log level INFO versus ERROR does not seem to help.
There may be a way to look for log patterns (malformed requests won't call the resolver) to distinguish good from bad requests, but this does not seem straightforward. Perhaps you will have better luck than I did!
Example of log entries for one request:
Begin Request
GraphQL Query: query Labels { labelStats(topic: "cars") { name objects __typename } } , Operation: Labels,...
{"duration":201275210,"logType":"ExecutionSummary","requestId":"4f49d425-18ab-4be7-a40e-0229c17453dc","startTime":"2021-12-17T16:36:30.925216Z","e...
{"logType":"RequestSummary","requestId":"4f49d425-18ab-4be7-a40e-0229c17453dc","graphQLAPIId":"xxxxxxxxxx","statusCode":200,"lat...
Request Headers: {content-length=[148], cloudfront-viewer-country=[US], x-forwarded-proto=[https], x-amzn-r...
Response Headers: {Content-Type=application/json; charset=UTF-8}
End Request
Related
I have node/express + serverless backend api which I deploy to Lambda function.
When I call an api, request goes to API gateway to lambda, lambda connects to S3, reads a large bin file, parses it and generates an output in JSON object.
The response JSON object size is around 8.55 MB (I verified using postman, running node/express code locally). Size can vary as per bin file size.
When I make an api request, it fails with the following msg in cloudwatch,
LAMBDA_RUNTIME Failed to post handler success response. Http response code: 413
I can't/don't want to change this pipeline : HTTP API Gateway + Lambda + S3.
What should I do to resolve the issue ?
the AWS lambda functions have hard limits for the sizes of the request and of the response payloads. These limits cannot be increased.
The limits are:
6MB for Synchronous requests
256KB for Asynchronous requests
You can find additional information in the official documentation here:
https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html
You might have different solutions:
use EC2, ECS/Fargate
use the lambda to parse and transform the bin file into the desired JSON. Then save this JSON directly in an S3 public bucket. In the lambda response, you might return the client the public URL/URI/FileName of the created JSON.
For the last solution, if you don't want to make the JSON file visible to whole the world, you might consider using AWS Amplify in your client or/and AWS Cognito in order to give only an authorised user access to the file that he has just created.
As noted in other questions, API Gateway/Lambda has limits on on response sizes. From the discussion I read that latency is a concern additionally.
With these two requirements Lambda are mostly out of the question, as they need some time to start up (which can be lowered with provisioned concurrency) and do only have normal network connections (whereas EC2,EKS can have enhanced networking).
With this requirements it would be better (from AWS Point Of View) to move away from Lambda.
Looking further we could also question the application itself:
Large JSON objects need to be generated on demand. Why can't these be pre-generated asynchronously and then downloaded from S3 directly? Which would give you the best latency and speed and can be coupled with CloudFront
Why need the JSON be so large? Large JSONs also need to be parsed on the client side requiring more CPU. Maybe it can be split and/or compressed?
I'm designing the backend for a website that uses API Gateway and Lambda to handle API requests, many of which target a MySQL DB on RDS. Some processes need to happen asynchronously but I'm debating which is best practice or cleaner.
In the given scenario, every time a user creates a new row in a certain table, let's say an email also needs to be sent asynchronously. There are many other scenarios similar to this but this will set precedent.
Option 1: In the lambda that handles the API request, first write to the MySQL instance to add the new row. When the response from MySQL comes back successful, write to something like SQS which will later be read from another lambda that sends an email. When the response from SQS is successful that the record was added to the queue, send a 201 response saying the REST API call was successful.
Option 2: In the lambda that handles the API request, write to the MySQL instance to add the new row. When the response from the MySQL comes back successful, send a 201 response saying the REST API call was successful. Then set up a DMS (data migration service) task that runs indefinitely to send database modification binlogs to a kinesis stream which will trigger a lambda that will handle all DB changes, read the change as a new row in a certain table, and send an email.
Option 1:
less infrastructure
more direct tracking of logic from an API call
1 extra http call (to sqs) delaying response times for an api for a web page
Option 2:
more infrastructure (dms task, replication instance)
scaling out shards may mean loss of ordering when processes binlog events if ordering is a requirement (it is)
side question: Are you able to choose hash key for kinesis for dms tasks from mysql?
a single codebase for reacting to all modifications in the DB may actually make following logic in code simpler
Is this the tradeoff or am I missing something? What is best practice in this scenario?
Option 1 in my view seems most logical, but I would replace SQS and second lambda with SNS. So, modified option 1 could be:
Option 1: In the lambda that handles the API request, first write to the MySQL instance to add the new row. When the response from MySQL comes back successful, publish confirmation message to SNS that sends an email. When the response from SNS is successful send a 201 response saying the REST API call was successful.
This should be faster, cheaper and easier to implement then using SQS and second lambda for sending email.
I want to change the default failure message in Alexa, Sorry, I'm having trouble accessing your {} skill right now.
You cannot change that prompt but you can code to avoid that as much as possible. The error happens when Alexa is not able to get a valid response from your skill endpoint. There can be multiple reasons to that as mentioned here
1. Your endpoint is giving an invalid response
This can be due to the errors/exceptions happening in your endpoint code. You can make sure that error/exceptions don't occur and if they occur, thre is code to catch them and provide a valid response back to Alexa, with an error message of your choice.
2. Your endpoint availability
Make sure that your endpoints are available all the time if you have configured them as an endpoint. This is pretty much guaranteed if you are using Lambda endpoints. But if you are your own hosted web service endpoint, then you must put in all the measures to keep it available for Alexa to communicate with it.
3. Your endpoint response time
Make sure that your endpoint gives back the response within the time period that Alexa expects it to get(guess its 10 seconds). Also make sure if you are using Lambda functions, you have configured them with reasonable execution time to avoid timeout errors.
If you cover the exception/error/availability scenarios well then you can avoid the default error message as much as possible.
I'm trying to do ACL by asserting if the item in DynamoDB whose field UserId is really the one logged in which is event.requestContext.identity.cognitoIdentityId.
But, I'm afraid that it can be spoofed just like HTTP headers etc.
My question is, is that safe?
No, this cannot be spoofed in the same way HTTP request headers can. If the request comes in through API Gateway, as a Lambda proxy integration, then there's nothing the browser can do that would allow these values to be overwritten, because this portion of the Lambda event structure is created by API Gateway and not copied from the request. Anything injected into the HTTP request would appear elsewhere in the event structure -- not here. (The HTTP request is in event.input -- which is a sibling object of event.requestContext -- not a parent.)
But then again... yes, this could be spoofed in certain other misconfiguration scenarios -- if, for example, your Lambda function allows itself to be invoked other than by your API Gateway deployment -- then of course the invoker could craft an entire event structure that had nothing to do with any HTTP request and invoke your Lambda function with it. This is perhaps too obvious to mention, since it's implicit from the way you can test a Lambda function from the console, but I mention it for thoroughness. Send a forged test event to your Lambda function using the Lambda console's test function, and naturally the Lambda function processes what you sent it.
So, unsurprisingly, with careless and overly broad permissions, yes, anything is possible... but used as intended behind API Gateway, as a Lambda Proxy Integration, I'd say no.
I have been researching this question for many hours.
I found this post where the author extracts the userId from the Token via:
const userId = await services.getUserIdFromToken(event.headers.Authorization);
This appears to be a safer way to handle setting the userId but all the other examples I have seen use event.requestContext.identity.cognitoIdentityId.
There is a resource /{myvar} defined in API Gateway, with GET method. Integration request points to Lambda function, with Lambda Proxy integration enabled.
When I invoke test execution from API Resource Editor of this resource and method, it works for queries like
/abc
/abc?def=ghi
but it fails to execute query like
/abc?def
with following response body visible in test console:
{
"cause": "Unable to invoke. Please try again later.",
"logref": "f6c905bd-cc71-11e8-a731-37e05a411010",
"message": ""
}
and also Response Headers and Logs boxes below are empty.
When I publish such resource to stage, accessing it through HTTPS in browser results with {"message": "Internal server error"} See edit below
How to deal with that? How can I capture whole resource path with or without query, with no Gateway crash? It fails the same way also for greedy resource /{myvar+}.
EDIT
After redeployment problem no longer occurs on stage. It is still occurs within Method Execution window in API Gateway Resources Editor.
you can capture the query string params through the "event" object sent to your Function Handler, it is called queryStringParameters. You can just log this and go through it on CloudWatch to see what exactly is failing.
P.S. Sorry for posting as an answer, don't have rep for comment ^^