Is it possible to configure AWS CloudWatch with a filter/monitor that "listens" or watches for a particular type of log message (ideally with granular or regex-like control where I can tell CloudWatch to look for a particular pattern in the log message) so that it forwards the log message off to a particular SNS endpoint?
Meaning:
My app publishes log messages to CloudWatch
CloudWatch is configured with this filter/monitor to listen for log messages matching a particular regex/pattern
Any messages matching this pattern get forwarded on to an SNS endpoint of my choosing
The best I could find was this article which shows how to have CloudWatch send email through SNS, but not sure if the alarm they use can be configured to watch for message patterns, and not sure if SNS can be configured to do non-SES/email related downstream work.
CloudWatch logs can have subscriptions. The targets can currently be setup for Kinesis streams or Lambda functions, but you could define the subscription filter to send matching messages to a lambda function that puts them onto the SNS topic, if that is required.
For example:
aws logs put-subscription-filter --log-group-name /aws/ecs/mycontainer --destination-arn arn:aws:lambda:us-east-1:123456:function:my-log-watch-sns-feeder --filter-name container-errors --filter-pattern "ERROR"
This would setup a subscription filter that sends log messages from an ECS container called mycontainer that contain the string ERROR to the lambda function named my-log-watch-sns-feeder.
For more information:
https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Subscriptions.html
https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html
Related
I have a number of AWS Alarms setup in CloudWatch. When one of these alarms is tripped, I currently send a notification to ServiceNow. I would like to send some custom parameters in this notification, something like:
{
"source":"machineName",
"dataUrl": "someUrl"
}
My challenge is, I do not see a way to add custom parameters via the AWS Console for an alarm. Is there a way for me to send custom parameters to an SNS topic that will get sent to the target from an AWS CloudWatch Alarm?
I have to monitor CloudWatch logs for particular lambda. When the specific error message such as HTTP-50X is logged to CloudWatch, then we need to send an email notification to alert that the something is wrong.
We need help to create alert email in AWS to monitor manually the logs to see if there is any such error logged.
This is exactly what metric filter is for. Create a metric filter that monitors your log group for specific pattern. Then create a CloudWatch alarm based on a new metric and configure it with an SNS action. Then simply subscribe your email to that SNS topic.
All of this can be easily done via CloudFormation(Metric filter, Alarm, SNS). You can also use CDK.
You can process/monitor logs in real-time using lambda functions. Checkout Using AWS Lambda with Amazon CloudWatch Logs and Real-time Processing of Log Data with Subscriptions for details.
I am trying to create CloudWatch alarms in N. California region because that is where my EC2s are.
I want to send a notification to an SNS topic that supports both email and SMS, which n. california does not, so I created an SNS topic in Oregon instead which supports both.
The topic does not appear in the "Send Notification To:" dropdown, reading online it seems like i'm supposed to be able to click 'enter list' and put in an ARN from another region, however when I do this I get an error saying:
Error:
There was an error saving the alarm. Please try again.
Cross region SNS will not be available to be configured for CloudWatch alarm notifications.
Alternatively, the following steps will help you achieve the same though not a fancy way of doing.
Configure a SNS in the same region and configure CloudWatch to send alarm notifications to this SNS in the same region
Subscribe a Lambda as a consumer for this SNS to listen and process the events sent from CloudWatch
The Lambda will reside in the same region which will receive the event from local SNS and publish (literally forward) the message (using SNS publish method) to the SNS in a different region.
The SNS in other region can have SMS and/or email subscriptions.
I've setup some aws cloudwatch agents to send logs to a cloudwatch log group and what I want to do now is get those into an elasticsearch cluster (on ec2). Ideally, what I want to do is write a lambda function that parses it and then sends it to elasticsearch, but also have the lambda function be triggered anytime a message is written to the log group. Is it possible to trigger lambda from a cloudwatch log group?
Subscribing AWS Lambda functions to CloudWatch Logs is documented here. Also, streaming CloudWatch Logs to the AWS ElasticSearch service is documented here.
Here you can find the documentation about adding a subscription to CloudWatch logs for Lambda.
For streaming to AWS ElasticSearch see this (as mentioned before).
I know that the AWS sends emails about any EC2 instances with Scheduled Events, but I would also like to have these notifications sent through HipChat and text message as well. Is there a default way of selecting a setting that can do this in AWS? Or would I have to create a more custom solution to achieve this?
With Scheduled Events in EC2 being manual tool for monitoring health of instances as of right now, there isn't many options. From the design point of view on AWS platform, this could be one possible solution:
Create AWS SES mailbox and enable it to receive emails,
Create a forwarding rule in your original e-mail inbox where you receive those e-mail notifications from AWS about Scheduled Events to forward e-mails of this type (based on filter you define) to the previously created AWS SES mailbox,
In your AWS SES mailbox create "Receipt Rule" with SNS Action, that will basically publish entire content of the e-mail to the SNS Topic using AWS SNS. (The SNS topic you choose must be in the same AWS region as the Amazon SES endpoint you use to receive email)
Once that e-mail content lands in AWS SNS Topic as an event, you can use AWS Lambda subscribed to your SNS topic as a handler of the event, parse what you need and forward customized notification to any of your third-party (outside of AWS) alerting tools (such as PagerDuty, Slack channel, HipChat, or whatever you use for SMS notifications).
This design does have one week point: relying onto your external mailbox forwarding system (if you are receiving Scheduled Events e-mails in inbox not based on AWS SES).
Ideally you could try to see in your AWS Account settings if it is possible to use AWS SES mailbox for Scheduled Events notifications, but I haven't tried that.
You can receive text messages as notification by setting alarm for that ec2 instance e.g if CPU usages goes above 80% then create an alarm and eventually you can set it to get emails or text messages. Thanks