I want to be able to set a time to invoke an AWS Lambda function, then have that function be invoked then and only then. For example, I want my Lambda function to run at 9:00pm on December 19th, 2017. I don't want it to repeat, I don't want it to invoke now, just at 9:00pm on the 19th.
I understand that CloudWatch provides Scheduled Events, and I was thinking that when a time to schedule this reminder for is inputted, a CloudWatch Scheduled Events is created to fire in that amount of time from now (so like if you schedule it at 8:22pm to run at 9pm, it’ll be 38 mins), then it invokes the Lambda function at 9pm which then deletes the CloudWatch Scheduled Event. My issue with this is that when a CloudWatch Scheduled Event is created, it executes right then, then at the specified interval.
Any other ideas would be appreciated, as I can't think of another solution. Thanks in advance!
You can schedule lambda event using following syntax:
cron(Minutes Hours Day-of-month Month Day-of-week Year)
Note: All fields are required and time zone is UTC only
Please refer this AWS Documentation for Details.
Thanks
You can use DynamoDB TTL feature to implement this easily, simply do the following:
1- Put item with TTL, the exact time you want to execute or invoke a lambda function.
2- Configure DynamoDB Streams to trigger a lambda function on item's remove event.
Once the item/record is about to expire, your lambda will be invoked. you don't have to delete or cleanup anything as the item in dynamodb is already gone.
NOTE: However the approach is easy to implement and scales very well, but there's one precaution to mention; using DynamoDB TTL as a scheduling mechanism cannot guarantee exact time precision as there might be a delay. The scheduled tasks are executed couple of minutes behind.
You can schedule a step function which can wait until a specific point in time before invoking the lambda with an arbitrary payload.
https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-wait-state.html
Something like this
const stepFunctions = new AWS.StepFunctions()
const payload = {
stateMachineArn: process.env.SCHEDULED_LAMBDA_SF_ARN,
name: `${base64.encode(email)}-${base64.encode(timestamp)}`, // Dedupe key
input: JSON.stringify({
timestamp,
lambdaName: 'myLambdaName',
lambdaPayload: {
email,
initiatedBy
},
}),
}
await stepFunctions.startExecution(payload).promise()
I understand its quite late to answer this question. But anyone who wants to use CRON expression to trigger an event(or call an API) only once can use following example:
This event will be triggered only once on January 1, 2025 - 12:00:00 GMT
00 12 01 01 ? 2025
For those who do not have much knowledge of cron syntax:
Minutes Hours DayOfMonth Month DayOfWeek Year
I am using this with AWS Cloudwatch Events and the result looks like this:
Note: I did not have to specify Day of week, since I have given it a fixed date and thats obvious.
Invoking a lambda function via Events is asynchronous invocation option. By using CloudWatchEvent to trigger Lambda function we can use cron job but we are still facing issues as Lambda function triggered multiple times for the same cron schedule.PFB Link:
https://cloudonaut.io/your-lambda-function-might-execute-twice-deal-with-it/
But this needs Dynamo DB to be implemented in your account and then make your Lambda function Idempotent.
Related
I need to figure out the CRON expression to trigger a Lambda that runs 1 hour before a specified date and time.
Example:
I get a specific date and time from a SNS notification, say 20th May 2022, 4 pm UTC
Now, I need to create a CRON expression for 20th May 2022, 3 pm UTC (i.e., 1 hour before the specific date and time), so that the Lambda is triggered an hour before the time.
I have gone through AWS docs and understand that I need to create a CRON expression to do this. I need help in figuring out the expression that would perform this and run only once.
There are two variants to define schedule expressions. One is rate() that juts tells it how often to run, the other is cron(), which is a cron-like syntax. The full syntax for the cron call looks like this:
cron(Minutes Hours Day-of-month Month Day-of-week Year)
So in your example you can use:
cron(0 15 20 5 ? 2022)
Note that all fields accept * as the wildcard, except the Day-of-week one, which uses ?.
To actually generate this from an SNS event, you can create another Lambda function that listens to SNS events, reads the desired timestamp, and creates a new EventBridge rule with the timestamp converted to the cron format as the ScheduleExpression. Depending on the language that you use in the Lambda, use the appropriate AWS SDK to create the EventBridge rule and target. For example, if you use Python, you can use the boto3 put_rule method and then call put_target to attach the new rule to the other Lambda function that runs the scheduled code.
Also, I believe that scheduled events are only available on the default EventBridge, not on custom ones. I know this used to be a limitation before, they might have changed it in the meantime.
I have a step function with 3 lambdas, the last lambda is basically writing an entry in the dynamo DB with a timestamp, status = "unpaid" (this is updated to "paid" for some automatically based on another workflow), email and closes the execution. Now I want to schedule a reminder on any entry in the DynamoDB which is unpaid & over 7 days, a second reminder if any entry is unpaid over 14 days, a third last reminder on 19th day - sent via email. So the question is:
Is there any way to do this scheduling per Step function execution (that can then monitor that particular entry in ddb for 7, 14, 19 days and send reminders accordingly until the status is "unpaid").
If yes, would it be too much overhead since there could be millions of transactions.
The second way which I was thinking was to build another scheduler lambda sequence: the first lambda basically parsing through the whole ddb searching for entries valid for reminder (either 7, 14, 19). The second lambda getting the list from the first lambda and prepares the reminder based on whether its first, second or third (in loop) & the third Lambda one sending the reminder through SES.
Is there a better or easier way to do this?
I know we can trigger step functions or lambdas through cloud events or we also have crons that we can use but they were not suiting the use case much.
Any help here is appreciated?
DynamoDB does not have functionality for a delayed notification based on logic, you would need to design this flow yourself. Luckily AWS has all the tools you need to perform this.
I believe the best option would probably be to create a CloudWatch Events/EventBridge when the item is written to DynamoDB (either via your application or as a trigger via a Lambda using DynamoDB Streams).
This event would be scheduled for 7 days time, in the 7 days any checks could be performed to validate if it has been paid or not. If it has not been paid you schedule the next event and send out the notification. If it had been paid you would simply exit the Lambda function. This would then continue for the next 2 time periods.
You could then further enhance this by using DynamoDB streams so that in the event of the DynamoDB table being updated a Lambda is triggered to detect whether status has changed from unpaid. If this occurs simply remove the event trigger to prevent it even having to process.
I have a scenario when i will have set of tasks to be executed at specific timing.
for example
task1: 28-06-2020 1:00 AM
task2: 30-06-2020 2:00 AM
task3: 01-07-2020 12:00 PM
.
.
.
n
i want to trigger my lambda(where me logic is defined), at these specified timing.
Probably i would be storing my timings to execute in a database,
can some tell me a way to execute lambda at a specified time.
I know we have TTL mechanism in dynamo which can trigger lambda but it delays the execution by 48 hours.I want my lambda to execute at the precise timings
You can use CloudWatch Events cron expressions for specific dates to execute only once. You would have to create rules for each date in question. This is based on the assumption that there is no regular pattern to repeatability of the dates.
The rules would trigger your lambda at these specific dates.
For example, for your dates in the question, you could use:
28-06-2020
30-06-2020
Given that you will have potentially 1000+ events at various times of day, you will need to implement your own solution. I would recommend:
Store events in a database (eg date, time, repetition pattern)
Use AWS CloudWatch Events to trigger an AWS Lambda function every minute
Code the Lambda function to:
Query the database for unprocessed events that are due (or past-due)
Invoke the appropriate Lambda function
Delete the event from the database, or mark it as processed (for repeating events, store a 'last processed' time)
Functions will potentially be invoked a few seconds late due to these processing steps, but that should be fine unless you need high-precision timing.
Steps Functions as an ad-hoc scheduler could be a good option for the use-case.
Query the database and Schedule execution for the specific date/time in Step Function state machine
In Step Function execution, map the lambda that needs to be triggered
Once the lambda is triggered at the desired time, the required business functionalities can be implemented.
References:
https://medium.com/serverless-transformation/serverless-event-scheduling-using-aws-step-functions-b4f24997c8e2
https://meetrix.io/blog/aws/06-using-step-functions-to-schedule-your-lambda.html
https://blog.smirnov.la/step-functions-as-an-ad-hoc-scheduling-mechanism-ed1787e44bb1
I have to implement functionality that requires delayed sending of a message to a user once on a specific date, which can be anytime - from tomorrow till in a few months from now.
All our code is so far implemented as lambda functions.
I'm considering three options on how to implement this:
Create an entry in DynamoDB with hash key being date and range key being unique ID. Schedule lambda to run once a day and pick up all entries/tasks scheduled for this day, send a message for each of them.
Using SDK Create cloudwatch event rule with cron expression indicating single execution and make it invoke lambda function (target) with ID of user/message. The lambda would be invoked on a specific schedule with a specific user/message to be delivered.
Create a step function instance and configure it to sleep & invoke step with logic to send a message when the right moment comes.
Do you have perhaps any recommendation on what would be best practice to implement this kind of business requirement? Perhaps an entirely different approach?
It largely depends on scale. If you'll only have a few scheduled at any point in time then I'd use the CloudWatch events approach. It's very low overhead and doesn't involve running code and doing nothing.
If you expect a LOT of schedules then the DynamoDB approach is very possibly the best approach. Run the lambda on a fixed schedule, see what records have not yet been run, and are past/equal to current time. In this model you'll want to delete the records that you've already processed (or mark them in some way) so that you don't process them again. Don't rely on the schedule running at certain intervals and checking for records between the last time and the current time unless you are recording when the last time was (i.e. don't assume you ran a minute ago because you scheduled it to run every minute).
Step functions could work if the time isn't too far out. You can include a delay in the step that causes it to just sit and wait. The delays in step functions are just that, delays, not scheduled times, so you'd have to figure out that delay yourself, and hope it fires close enough to the time you expect it. This one isn't a bad option for mid to low volume.
Edit:
Step functions include a wait_until option on wait states now. This is a really good option for what you are describing.
As of November 2022, the cleanest approach would be to use EventBridge Scheduler's one-time schedule.
A one-time schedule will invoke a target only once at the date and time that you specify using a valid date, and a timestamp. EventBridge Scheduler supports scheduling in Universal Coordinated Time (UTC), or in the time zone that you specify when you create your schedule. You configure a one-time schedule using an at expression.
Here is an example using the AWS CLI:
aws scheduler create-schedule --schedule-expression "at(2022-11-30T13:00:00)" --name schedule-name \
--target '{"RoleArn": "role-arn", "Arn": "QUEUE_ARN", "Input": "TEST_PAYLOAD" }' \
--schedule-expression-timezone "America/Los_Angeles"
--flexible-time-window '{ "Mode": "OFF"}'
Reference: Schedule types on EventBridge Scheduler - EventBridge Scheduler
User Guide
Instead of using DynamoDB I would suggest to use s3. Store the message and time to trigger as key value pairs.
S3 to store the date and time as key value store.
Use s3 lambda trigger to create the cloudwatch rules that would target specific lambda's etc
You can even schedule a cron to a lambda that will read the files from s3 and update the required cron for the message to be sent.
Hope so this is in line with your requirements
I'm experimenting with dynamo db and lambda and am having trouble with the following flow:
Lambda A is triggered by a put to S3 event. It takes the object, an audio file, calculates its duration and writes a record in dynamoDB for each 30 second segment.
Lambda B is triggered by dynamoDB, downloads the file from S3 and operates on the 30 second record defined in the dynamo row.
My trouble is that when I run this flow, function A writes all of the rows required to dynamo, by function B
Does not seem to be triggered for each row in dynamo
Times out after 5 minutes.
Configuration
Function B is set with the highest memory and 5 minute expiration
The trigger is set with a batch size of 1 and starting position latest
Things I've confirmed
When function B is triggered, the download from S3 happens fast. This does not seem to be the blocker
When I trigger function B with a test event it executes perfectly.
When I look at the cloudwatch metrics, function B has a nearly 100% error rate in invocation. I can't tell if this means he function was invoked and had an error or could not be invoked at all.
Has anyone had similar issues? Any idea what to check next?
Thanks
I had the same problem, the solution was to create a VERSION from the Lambda and NOT to use the $LATEST Version, but a 'fixed' one.
It is not possible to use the latest ever-changing version to build a trigger upon.
Place to do that:
Lambda / Functions / YourLambdaName / Qualifiers Dropdown on the page / Switch versions/aliases / Version Tab -> check that you have a version
If not -> Actions / Publish new version
Check for DynamoDB "Stream" is it is enabled on the table.
Checkout this
5 min timeout is default for lambda, you can find this mentioned in forums.