I have written some cronjobs in my django app and I want to schedule these jobs using AWS Lambda service. Can someone please recommend a good approach to get this done?
I will answer this based on the question's topic rather than the body, since I am not sure what the OP means with "I want to schedule these jobs using AWS Lambda".
If all you want is trigger your Lambda function based in a cronjob, you can use CloudWatch Events to achieve this. You can specify regular cron expressions or some built-in expressions that AWS makes available, like rate(1 min) will run your function every minute. You can see how to trigger a Lambda function via CloudWatch Events on the docs. See cron/rate to see all the available options.
CloudWatch Events is only one of the many options to trigger you Lambda function. Your function can react to a whole bunch of AWS Events, including S3, SQS, SNS, API Gateway, etc. You can see the full list of events here. Just pick one that fits your needs and you are good to go.
EDIT AFTER OP'S UPDATE:
Yes, what you're looking for is CloudWatch Events. Once you have the Lambda to poll your database in place, you can just create a rule in CloudWatchEvents and have your Lambda be triggered by it. Please see the following images for guidance.
Go to CloudWatch, click on Events and choose Schedule as the Event Source
(make sure to setup your own Cron expression or select the pre-defined rate values)
On the right-hand side, choose your Lambda function accordingly.
Click on "Configure Details" when you are done, give it a name, leave the "Enabled" box checked and finally click on Create.
Go back to your Lambda function and you should see it's now triggered by CloudWatch Events (column on the left-hand side)
Your lambda is now configured properly and will execute once a day.
Related
I am looking for the best way to detect errors in cloudwatch logs which are logged by lambda functions, the log output is structured.
I was considering using a metric filter to trigger a lambda but I think eventbridge is now the preferred way to do this sort of thing but from the documentation I cannot work out what is the right way to approach it.
I would like to trigger the same eventbridge rule for any error in any log group if this is possible as all the logs have the same format.
Is it possible to do this purely from cloudwatch log entries so I do not need to add additional code to my functions to call event bridge using the AWS api's?
Instead I would like to trigger the rule whenever a matching json object gets inserted into cloudwatch logs.
I was not even able to find the event structure for cloudwatch log updates.
Amazon EventBridge is a serverless event bus for building event-driven applications. It is best suited for application to application integration with event filtering. Your use case seems to be of pure monitoring ( or notification)
For your use case (monitoring) using the metric filter will be the simple and elegant option.
For implementation (nodejs) refer :CloudWatch log multiple custom metric filters to trigger lambda function
I am new to AWS lambda.
Will like to seek advice from the experts here.
I understand that Lambda is activated based on a trigger.
If I wanted to send a timed http request (for example, send a http request 4 hours later),
is there any recommendations to do it.
Yes, you can configure scheduled AWS Lambda Triggers using Cloudwatch.
Tutorial: Schedule AWS Lambda Functions Using CloudWatch Events
To create a rule using the console
Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/
In the navigation pane, choose Events, Create rule.
For Event Source, do the following:
a. Choose Schedule.
b. Choose Fixed rate of and specify the schedule interval (for example, 5 minutes).
For Targets, choose Add target, Lambda function.
For Function, select the Lambda function that you created.
Choose Configure details.
For Rule definition, type a name and description for the rule.
Choose Create rule.
I need to start a Lambda Function when an object has been created on an S3 Bucket. I found 2 solutions to do this.
Using AWS::S3::Bucket NotificationConfiguration.
Using a CloudWatch AWS::Events::Rule.
They both seem to do exactly the same thing, which is to track specific changes and launch a Lambda Function when it happens. I could not find any information on which one should be used. I'm using Cloud Formation Template to provision the Lambda, the S3 Bucket and the trigger.
Which one should I use to call a Lambda on Object level changes and why?
Use the 1st one because of
A push model is much better than a pull model. Push means you send data when you get it instead of polling onto something for some set of interval. This is an era for push notifications all over us. You don't go to facebook to check every 5 minutes if someone has liked your picture or not OR someone has replied to your comment, etc.
In terms of cost and efforts also, S3 event notification wins the race.
Cloudwatch was the best option if you didn't have S3 notification but since you have it, that's the best. Plus if you have a feature in the service itself then why will you go for an alternative solution like Cloudwatch rules.
I have AWS lambda function written in c#, and I want to invoke that lambda function for the specific interval. This interval value is not fixed, and the user can customize this interval from the app. Any ideas on how to achieve this?
One method is to programmatically update the schedule of a CloudWatch Scheduled Rule. This works well if you have a relatively low number of schedules, but there are limits to how many schedules you can create. The default limit is 50 rules, though this can be increased to meet your needs by requesting an increase from AWS.
This is an example of creating a rule programmatically in C#, you'll need to permission your Lambda Role to update the rules, also in this doc:
https://docs.aws.amazon.com/sdk-for-net/v3/developer-guide/cloudwatch-examples-sending-events.html#create-a-scheduled-rule
try Using AWS Lambda with Amazon SQS
SQS has each URL.
user (or some API) can send request to that url.
The outline of processing is as follows
user (or some API) -> SQS -> Lambda
My use case is as follows: I need to be able to schedule SQS messages in such a way that scheduled messages can be added to a queue on a specific date/time, and also on a recurring basis as needed.
At the implementation level, what I'm basically be looking to do is have some function I can call where I pass in the SQS queue, message, and schedule I want it to run on, without having to build the actual scheduler logic.
I haven't seen anything in AWS itself that seems to allow for that, I also didn't get the impression Lambda functions would do exactly what I need unless I'm missing something.
Is there any other third party cloud service for scheduled processes I should look into, or am I better off in the end just running a scheduling machine at AWS and have some REST API that can add cron jobs/windows scheduled tasks to it that will handle the scheduling of SQS messages?
I could see two slightly different ways of accomplishing this, both based on Cloudwatch scheduled events. The first would be to have Cloudwatch fire off a Lambda. The Lambda would either have the needed parameters or would get them from somewhere else - for example, a DynamoDB table. Otherwise, the rule target allows you to specify a SQS queue - skipping the Lambda. But I'm not sure if that would have the configuration ability you'd want.
Either way, checkout Cloudwatch -> Events -> Create Rule in the AWS console to see your choices.