I would like to schedule triggering my Lambda function.
AWS EventBridge allows me to do this (e.g. by creating cron based rule) but I don't understand its pricing model.
It states "All state change events published by AWS services are free" but I'm not sure if an event fired by EventBridge (generated by EventBridge rule) relates to this free one. Probably not.
If it's not free, then how 30 events per month will be billed if pricing is based on million of events.
Googling didn't help. Do you have ideas how 30 events will be billed?
Update:
While experimenting I found that we should not pay for 30 events per months but it's not clear why. Probably less than million of events are free.
When you enable AWS EventBridge Rule to trigger based on a fixed rate or a Cron expression, the EventBus selection gets disabled. This means, the events from this Rule will be delivered to the default bus which is where all the events from AWS services end up. Also, if you look at one of the events delivered using the fixed rate or the cron expression, the event source is "aws.events" (which is one of the AWS services. In this case, it is the EventBridge). All events generated by AWS services fall under AWS Service events and they are free.
Related
Is it possible to determine a time for sending alerts on the cloudwatch aws?
For example:
I would like to configure the operation of alerts from Monday to Friday at 8 am to 6 pm
A CloudWatch Alarm will trigger whenever the rule is met. It might take a minute or so to react (depending upon the metric and the aggregate function used).
However, it is not possible to schedule when alerts operate.
If you wish, you could create an AWS Lambda function (triggered on a schedule) that will call disable_alarm_actions() and enable_alarm_actions(). The documentation says:
disable_alarm_actions(): Disables the actions for the specified alarms. When an alarm's actions are disabled, the alarm actions do not execute when the alarm state changes.
Was studying about Amazon web services and fundamentals when came across these 2 concepts:
Amazon CloudWatch
Amazon CloudWatch Events
Even while going through the official documents on AWS, I couldn't find a difference between the two even when Amazon mentions that they are different. Excerpt is:
CloudWatch provides you with data and actionable insights to monitor
your applications, respond to system-wide performance changes,
optimize resource utilization, and get a unified view of operational
health. CloudWatch collects monitoring and operational data in the
form of logs, metrics, and events, providing you with a unified view
of AWS resources, applications, and services that run on AWS and
on-premises servers. You can use CloudWatch to detect anomalous behavior in your environments, set alarms, visualize logs and metrics side by side, take automated actions, troubleshoot issues, and discover insights to keep your applications
running smoothly.
Documentation of AWS CloudWatch
Amazon CloudWatch Events delivers a near real-time stream of system
events that describe changes in Amazon Web Services (AWS) resources.
Using simple rules that you can quickly set up, you can match events
and route them to one or more target functions or streams. CloudWatch
Events becomes aware of operational changes as they occur. CloudWatch
Events responds to these operational changes and takes corrective
action as necessary, by sending messages to respond to the
environment, activating functions, making changes, and capturing
state information.
Documentation of AWS CloudWatch Events
CloudWatch
CloudWatch is a monitoring service for your AWS resources. You can log your log files. By default the resources created within AWS logs in CloudWatch(CW). You can monitor the performance of resources too for example you can monitor how is the CPU utilisation of your EC2 instances. You can set Alarms for your resources
threshold and get an SNS alert on that. For example you can create an Alarm for your DynamoDB if Write capacity is exceeding. You can set an alarm for your billing too. So basically CW is used as a Monitoring solution.
CloudWatch Events
CW Events is also the part of CloudWatch. CloudWatch Events is helpful when you want to schedule something. Say you to want run your lambda every other day, you can create a Rule for that or you want to trigger your lambda by Event Pattern. There are bunch of services supported by CloudWatch Events, you can use anyone of them as your target not just Lambda. Event Buses is used to send your events to multiple accounts also. For example if you have a CICD account and every month you bake new AMI there, to notify all accounts you can use Event Buses, after getting the event from Event Buses other accounts can trigger some important tasks.
I see this question has been asked few times but has not been answered yet. Making another attempt.
What is the basic difference between an S3 event and Cloudwatch events ?
Is one is preferred over the other ?
Appreciate an answer.
Thanks !
S3 Event Notifications are for events that are specific to S3 buckets. S3 Events Notifications can publish events for
New object created
Object removal
Restore object
Reduced Redundancy Storage (RRS) object lost events
Replication events
And it can send notifications to:
SNS topics
SQS queues
Lambda functions
CloudWatch Events, and the associated (preferred, actually) service, Amazon EventBridge, are much broader, and apply to the entire AWS platform. CloudWatch and EventBridge use the same underlying CloudWatch Events API, but EventBridge has more features.
You can use CloudWatch Events/EventBridge to react to any event published by AWS CloudTrail as well as from a very long list of integrated AWS services. These events can also be published on a schedule using a cron-like schedule expression syntax. It can send notifications to more targets as well, including Amazon EC2, Kinesis data streams, ECS tasks, Systems Manager, and much more.
Generally, it's preferable to use EventBridge for anything other than S3. Since EventBridge shares the same underlying API as CloudWatch Events, any change you make to either one will show up in the other. You should use S3 Events for any of the events listed above (see the docs for up an to date list of events).
So our project was using Hangfire to dynamically schedule tasks but keeping in mind auto scaling of server instances we decided to do away with it. I was looking for cloud native serverless solution and decided to use CloudWatch Events with Lambda. I discovered later on that there is an upper limit on the number of Rules that can be created (100 per account) and that wouldn't scale automatically. So now I'm stuck and any suggestions would be great!
As per CloudWatch Events documentation you can request a limit increase.
100 per region per account. You can request a limit increase. For
instructions, see AWS Service Limits.
Before requesting a limit increase, examine your rules. You may have
multiple rules each matching to very specific events. Consider
broadening their scope by using fewer identifiers in your Event
Patterns in CloudWatch Events. In addition, a rule can invoke several
targets each time it matches an event. Consider adding more targets to
your rules.
If you're trying to create a serverless task scheduler one possible way could be:
CloudWatch Event that triggers a lambda function every minute.
Lambda function reads a DynamoDB table and decide which actions need to be executed at that time.
Lambda function could dispatch the execution to other functions or services.
So I decided to do as Diego suggested, use CloudWatch Events to trigger a Lambda every minute which would query DynamoDB to check for the tasks that need to be executed.
I had some concerns regarding the data that would be fetched from dynamoDb (duplicate items in case of longer than 1 minute of execution), so decided to set the concurrency to 1 for that Lambda.
I also had some concerns regarding executing those tasks directly from that Lambda itself (timeouts and tasks at the end of a long list) so what I'm doing is pushing the tasks to SQS each separately and another Lambda is triggered by the SQS to execute those tasks parallely. So far results look good, I'll keep updating this thread if anything comes up.
I have a workflow where I need to pass in some information that would be stored for a period of time, and then sends off a trigger after a scheduled period of time with the same information.
I considered using a TTL on a dynamo db table, but I was wondering if I could use cloudwatch events for this since it seems ideal as it has cron expressions for cloudwatch rules.
I know I can setup a cloudwatch rule to trigger say every 15 minutes, but how do I setup cloudwatch such that only my custom information gets picked up by this rule and I can pass some information into this event so that when the trigger gets sent to the target, my custom information is sent to the target as well?
DynamoDB TTL is a bad fit as you don’t get a ton of granularity.
You can use the CloudWatch events PutEvent api to put a custom event. That should get you where you want.
https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/AddEventsPutEvents.html