I have a scheduled daemon lambda that is called once a day to analyze data and create needed alert messages for the users of the app for the further sending.
The problem I try to solve is to avoid "sparks" in email sending since if I try to send all the created emails at one moment I will get natural reject from SES. So I plan to store created emails somewhere (SQS, DynamoDB, etc) and then schedule a run of another lambda that will take a portion of emails, call SES.sendEmail(), then if there are still emails to send, schedule another call of itself in a few seconds.
I planned to create CloudWatch event inside lambda and use Cron to schedule lambda call but realized that Cron has 1 minute precision, which is not enough for my purposes. I'm not going to wait inside any lambda (since its just pay for nothing approach), so Lambda.Invoke() is also inappropriate. Are there any other alternatives to recursively schedule call of lambda from itself with conditional delay and seconds precision?
You could have a lambda that publishes the emails to sqs, which allows you to set a delivery delay, and have another lambda which is triggered by messages in the queue.
Related
I am developing an email client that uses SMTP server from my customers. I stock the credentials in an AWS RDS database. Customers are able to create mail campaigns, and email have to be sent asynchronously after the campagin creation.
I want to configure a SQS queue to make messages available with a delay of 2 minutes between each message. The purpose is to send the message to a Lambda function able to send the message through SMTP (using the credentials in the database). Thanks to this delay between messages, I can optimize the delivery of the emails. The order of the delivery is not really important.
Unfortunately, I do not find a way to do that. The timers available in SQS are for the entire Queue, or have to be specific for each message. In my case, if i put 100 messages in the SQS queue, I need the first one to be sent immediately, the second 2 minutes after and so on.
Does someone has solutions with this issue ? If SQS is not the right service to manage this need, is there another one available on the AWS platform ?
We can probably achieve this combination of lambda event source mapping config for sqs and lambda reserved concurrency setting.
Event source Mapping Batch size: 1
This will allow only 1 message to be sent to Lambda at once.
Event source Mapping Batch window: 2 mins
This will allow lambda to be invoked only once every 2 minutes.
Lambda Reserved Concurrency as 1
This will allow only one lambda thread to be called at once.
Thanks #luk2302 for your answer, it is the good one accordingly to the AWS support team. Using the EventBridge service to trigger the lambda at define timing works perfectly.
I have a question related to AWS on how to trigger a Lambda function based on time.
I have a booking system where users can book a ride. I want to send an automated notification for them e.g. 10 mins from the time rider should arrive in case the rider is late.
How can I trigger the lambda function to run (as an example 02:30 and the current time is 12:00)
Presumably you could have hundreds of users who need to receive notifications at any particular time. Therefore, merely triggering the Lambda function at a particular time probably isn't the way to accomplish this.
Instead, you could:
Use a database to maintain the times that users should be notified
Trigger an AWS Lambda function every minute
The Lambda function would:
Consult the database to find any notification that is due to be sent now (or previously) and has not already been sent
Send the notification
Mark the notification as sent (or delete it from the database)
Note that the function should look for any notification that is due, even if it was for a previous time. This will handle situations where notifications were not successfully sent previously, or where the Lambda function doesn't run for some reason.
The Lambda function can be triggered on a schedule by using Amazon CloudWatch Events.
I’m working on figuring out the best way to have Lambda run one time tasks at a given time.
The system I’m envisioning will basically have events that will need to be sent out, either as soon as the event is received/created, at a specific time, or as a recurring action. And I’d like to use AWS as much as possible for this, due to the scalable nature.
My original idea was to have a AWS SQS queue for events to send. Then I’d have a DynamoDB table for future events. I’d also have two AWS Lambda functions, one setup to run on a cron job every few minutes to take the events that are scheduled in the next 15 minutes or so from the DynamoDB table and put them into that AWS SQS queue with a Message Timer setup to delay the message from being visible for that given time. The second Lambda function would be setup and have a trigger to be run from that AWS SQS queue. This function would be responsible for actually sending the event out.
From there I could either add the event to the SQS queue (with or without a message timer) if it’s gonna need to be sent out within the next 15 minutes. Or add it to the DynamoDB table if it’s gonna need to be sent out in the future (beyond 15 minutes).
The biggest problem I just figured out is that AWS SQS FIFO queues doesn’t support Message Timers on individual messages. I need a FIFO queue because I need to prevent these events from being sent out multiple times, or triggering my second Lambda function twice.
I've also looked into the AWS Lambda cron jobs, and although you can schedule invocations every say 5 minutes, I don't think this is what I'm looking for because I'm looking more for scheduling a 1 time invocation in the future, and having that be scalable. So I don't think this is what I'm looking for.
Any ideas on how I can achieve this, since it doesn’t look like Amazon SQS Message Timers will work for what I'm trying to do?
Have you considered Step Function? You could create a wait state before invoking the lambda.
I have a requirement to send email and SMS based on some conditions to users, i want to publish a message to AWS (Any service) with time and message at the time of user creation, is there any way to call a lambda function based on my scheduled time along with message?
Sounds like what you are saying is that you want to store a message and a 'time to send' someplace and then when that time comes, send out that message via SMS and/or SES, correct?
Lots of ways to accomplish it, but one way would be to store your messages into the database of your choice (perhaps dynamodb), and have a lambda function that gets called periodically (every minute or whatever frequency you determine) to find messages that are ready to send.
In this scenario you could use cloudwatch events to call the lambda function at the interval you decide (but no more frequent than once per minute).
Possible enhancement (especially if you have a huge number events) would be to have the lambda function not actually process the sms/ses sends - but just find those messages that are ready to send - and post those messages to an SNS topic and have a different lambda function that takes care of the actual processing (sending) of those messages.
You can use CloudWatch scheduled events for this. It allows you to specify cron expression. The event itself can trigger your lambda that then checks any preconditions you might have and then sends notification via SNS or some other way.
Im looking for a serverless solution for an old system and its working like a charm, there is only one think I have no idea what is the best solution, here is the point
USER ---> API Gateway ---> Lambda ---> DynamoDB
User want to trigger a lambda in a specified time, example.
Im a user and I want to post a message in a dashboard (the function to do this is in a lambda) with some parameters saved in DynamoDB, and should be done tomorrow at 5.
User make a API request throw API Gateway, lambda is executed and put some info in DynamoDB, how can trigger another lambda with this info tomorrow ?
In the old system we have a cron with the time and when it should be triggered it just read the DB to get the parameters.
What can I use ? SQS ? CloudWatch Events ? with S3 ? DynamoDB stream ?
More info, could be like 10-20 executions per day.
When the user invokes the lambda via the API Gateway and when you put the data in dynamo db you can at the same time have a message inserted in SQS with the exact time stamp when you want this user action to have a lambda invocation.
Now have a scheduled lambda that executes every minute or every 5 minutes or whatever suits you. The work of this lambda will be to poll all the messages of SQS and see if any schedule has reached for any of the message. If yes, invoke another lambda and pass this payload for invocation. If not then sleep till next polling time
You could also do the same architecture in cloud. The on-prem cron can be replaced by a Cloudwatch cron schedule. So your second Lambda can be triggered based on the cron schedule and you scan your DB (dynamodb in this case) and do the processing.