I have two lambdas. Lambda A is triggered once an SQS is populated with messages. I want Lambda B to execute once Lambda A is done executing. How can I do this? Lambda A will have multiple invocations running at the same time, does that make a difference?
Steps functions are created for this purpose. You can transfer data from one lambda to another, so you will have the context of what is being worked on by the previous lambda will be sent to the next lambda.
To do this, you'd need to trigger the Step Function directly from a lambda. As far as I know, you can't trigger Step Functions directly from SQS.
So, either
If Lambda A is triggered by something else than SQS, trigger your Step Function instead. It will then run Lambda A and Lambda B depending on how you set it up.
If Lambda is triggered by SQS, you make a third lambda (Lambda 0), triggered by SQS, whose sole purpose is to in turn trigger your Step Function (which will run Lambda A then Lambda B) or you directly trigger Lambda B from Lambda A (in which case, Step Functions are pointless and you should rather go with SQS / SNS).
A note on this; one thing Step Functions cannot do out of the box is, for example execute Lambda B only once all Lambda A invocations are done. It will act on a per execution basis.
Related
I have an aws lambda function. When it receives only one trigger, it always succeds. But when it receives more than one trigger, it sometimes throws error. The first trigger always succeds.
Can I configure one aws lambda function receives only one trigger?
can one aws lambda function handle multiple triggers at once?
Yes, Lambda functions can handle multiple triggers at once.
when it receives more than one trigger, it sometimes throws error
This is most probably related to your implementation. Are you doing something different based on the inputs? Is the code behaving differently based on time?
Can I configure one aws lambda function receives only one trigger?
You can limit the concurrency of the Lambda function. If you set it to 1, you can only have one Lambda function running at any given time.
See: Set Concurrency Limits on Individual AWS Lambda Functions
Here's what I know, or think I know.
In AWS Lambda, the first time you call a function is commonly called a "cold start" -- this is akin to starting up your program for the first time.
If you make a second function invocation relatively quickly after your first, this cold start won't happen again. This is colloquially known as a "warm start"
If a function is idle for long enough, the execution environment goes away, and the next request will need to cold start again.
It's also possible to have a single AWS Lambda function with multiple triggers. Here's an example of a single function that's handling both API Gateway requests and SQS messages.
My question: Will AWS Lambda reuse (warm start) an execution environment when different event triggers come in? Or will each event trigger have it's own cold start? Or is this behavior that's not guaranteed by Lambda?
Yes, different triggers will use the same containers since the execution environment is the same for different triggers, the only difference is the event that is passed to your Lambda.
You can verify this by executing your Lambda with two types of triggers (i.e. API Gateway and simply the Test function on the Lambda Console) and looking at the CloudWatch logs. Each Lambda container creates its own Log Stream inside of your Lambda's Log Group. You should see both event logs going to the same Log Stream which means the 2nd event is successfully using the warm container created by the first event.
More fundamentally, I'm wondering if there is a reasonable or preferred way to use a lambda function to schedule the execution of another lambda function at a later time.
I have a Scheduled Lambda function (via CloudWatch event rule) which is triggered every minute.
This lambda picks up a request from SQS queue, process the parameters and triggers AWS step functions workflow.
Now, ONLY 1 Lambda function instance is running every minute. How can I trigger multiple (e.g. 10) concurrent Lambda functions like this?
One way I can think of is to create 10 Cloudwatch event rule which runs every 1 minute, but I am not sure if that is the right way of doing it. Also, if I use this way, 10 lambda would be called even if I don't have entries in my SQS queue.
You can use the lambda step function.
Event trigger first function. Then it will call multiple functions parallel.
Some useful links:
https://www.youtube.com/watch?v=c797gM0f_Pc
https://medium.com/soluto-nashville/simplifying-workflows-with-aws-step-functions-57d5fad41e59
since your lambda function fetching data from SQS so you can create event source mapping between lambda and SQS so whenever message published to SQS, your lambda function will invoke concurrently depending on number of messages in queue so you do not need to invoke lamnda from cloudwatch event
I have an aws lambda function which is running daily at same time as cron job and is generating cloudwatch logs. I have another lambda function that takes those cloudwatch logs and move it to S3. So I want that when my first lambda function finishes execution, the logs lambda function starts and pushes the logs to S3 bucket. Kindly suggest how I can achieve this.
You can invoke a Lambda function from another Lambda function through the AWS SDK. So your first function should call the second function when it is finished. Make sure to select the InvocationType "Event" when invoking the second function and do not add any callbacks to avoid having the functions run in parallel and paying twice.