I have an API (through API Gateway) where each resource's method is routed to a single lambda function via lambda proxy integration, where each request is processed internally.
How can I automate a scheduled invocation of one resource's method with a CloudWatch event? This preferably will not be done by directly invoking the lambda function, and would allow a specified (constant) input.
Since CloudWatch Events does not support invoking an HTTP endpoint directly, you should be able to accomplish this by using SNS with a HTTP subscription. The workflow would be the following:
Scheduled CloudWatch Event -> SNS endpoint with constant JSON payload -> HTTP subscription, where the HTTP subscription is your API Gateway endpoint.
Related
I'm trying to build a schedule in AWS EventBridge that is going to invoke an API Gateway endpoint on a specific rate. However, I'm not able to specify the API Gateway endpoint when creating the schedule. I couldn't find examples in the documentation about the JSON body that I need to provide to specify the API Gateway ARN, endpoint and headers.
Follow AWS EventBridge documentation: https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-api-gateway-target.html
I am integrating AWS API Gateway with AWS SQS.
What would happen if API Gateway receives a payload > 256KB? Can I divert it to S3 somehow?
In this use case, you probably want to use a lambda proxy integration with API gateway.
So, your API Gateway endpoint sends the payload to a lambda function.
The lambda function works out the size of the payload. If it is <256KB you may want to place his onto the queue in SQS via the lambda.
If it is >256kb then you could write this down to s3 via the lambda, and then place a message into your queue with the s3 details. Your worker can then use the message in the queue to grab the data in s3 and process it from there.
Note there is a 10MB payload limit for API Gateway.
AWS provide the following guide on Lambda Proxy Integration with API Gateway.
https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-proxy-integrations.html
I have a requirement where I need to send some inbound http query parameters from api A by publishing it to SNS and other api B is subscribed to sns where it should get the query params.
Is there any way I can acheive it?
Note: I have a fixed message structure when I publish to SNS which I cannot change, since I have other endpoints who are already subscribing to SNS, depend on it.
As Michael mentioned, we can use lambda. but I do not want to use another service provided by aws. I want to acheive the same using SNS only. I heard about SNS message attributes, but not sure whether we can acheive the same using it.
Thanks.
SNS doesn't support customizing the HTTP transaction or otherwise modifying the message being delivered... but SNS does support subscriptions that target a Lambda function.
This means you can write a Lambda function that parses the SNS event payload... and then, instead of SNS contacting the target system over HTTPS, your Lambda function makes the HTTPS request to the target endpoint, customized as required. If the HTTPS request fails, or the endpoint returns an error, the Lambda function should throw an exception, so that Lambda can know to retry.
https://docs.aws.amazon.com/lambda/latest/dg/invoking-lambda-function.html#supported-event-source-sns
I want to trigger an AWS lambda function upon the creation/modification of an API in AWS API gateway. Any idea what are the right events to catch?
As noted, API Gateway does not generate events directly. Additionally, it does not appear that you can configure an event rule based on CloudTrail logs for API Gateway either. I'll make sure to forward your request to the CloudWatch Events team as a feature request.
Since there is no event rule of AWS APIGateway in AWS CloudWatch and it does not appear log in AWS CloudTrail, so we can not achieve it now.
I have been experimenting with AWS API Gateway and AWS Lambda to try out a serverless architecture. Have been going through blogs and AWS documentation. Have tried out sample GET/POST. But, I have the following requirement w.r.t tracking user events from my custom application
Events are posted from my application to API end point
I wanted the API to respond back with a custom response (Say {'fine'})
(acknowledging that the request has been received)
After the response is sent, hand over the event payload to a AWS Lambda function
As per the documentation, I understand,
a) I can post events to API end point
b) On GET/POST trigger an AWS Lambda Function
- Respond back from AWS Lambda function to API request
I wanted to change the above and modify it to
a) Post events to API end point
a.0) Respond back acknowledging that request is received [Say {'fine'} ]
b) Trigger AWS Lambda function to process the event payload
Please share across suggestions on how to achieve the same.
Another asynchronous model many customers have used:
Set up an API configured to send requests to Amazon Kinesis. This API could acknowledge the request.
Set up AWS Lambda to consume your Kinesis stream.
This setup has some advantages for high workload APIs as fetches from the Kinesis stream can be batched and don't require a 1-to-1 scaling of both your API Gateway limits and Lambda limits.
Update
To answer your questions about scalability:
Kinesis
Kinesis scales by adding what it calls "shards" to the stream. Each shard handles a portion of your traffic, based on a partition key. Each shard scales up to 1000 rps or 1MBps (see limits). Even with the lower default 25 shards, this would support up to 25,000 rps or 25MBps with an evenly distributed partition key.
API Gateway
API Gateway has a default account level limit of 500 rps, but this can easily be extended by requesting a limit increase. We have customers in production that are using the service at limits above your current suggested scale.
If you want a fast response from the API and not have to wait for the processing of data, you could:
post an event to an API Gateway endpoint
trigger an AWS Lambda Function A
call asynchronously a Lambda Function B using the AWS SDK in the Lambda Function A
Call context.succeed() or context.done() or the callback function in the Lambda Function A so it respond back to API Gateway
the Lambda Function B can process the data while API Gateway already received a response
You should first run some tests to see what type of real world response times you are getting from having your lambda function complete all the logic. If the times are above what you feel are acceptable for your use case, here is another asynchronous solution utilizing an SNS Topic to trigger a secondary Lambda function.
Client Request to API Gateway -> Calls Lambda function A
Lambda A verifies payload and then publishes to SNS Topic X
Lambda A returns {fine} success message -> API Gateway -> client
SNS Topic X triggers Lambda function B
Lambda function B implements given logic