What I'm trying to implement is that on the AWS billing alarm I want to send an email and invoke a lambda function written in python from where I want to update something in the database. So how can I get these both, I know that against alarm you can send an email to the user but how to do both when an alarm is triggered?
{
"Type": "AWS::CloudWatch::Alarm",
"Properties": {
"AlarmName": "AWS Polly Alarm One",
"AlarmDescription": "AWS Polly Alarm One",
"ActionsEnabled": true,
"OKActions": [],
"AlarmActions": [
"arn:aws:sns:us-east-1:XXXXXXXXXXXX:Turn_Off_AWS_Polly_Send_Email_One"
],
"InsufficientDataActions": [],
"MetricName": "EstimatedCharges",
"Namespace": "AWS/Billing",
"Statistic": "Maximum",
"Dimensions": [
{
"Name": "ServiceName",
"Value": "AmazonPolly"
},
{
"Name": "Currency",
"Value": "USD"
}
],
"Period": 60,
"EvaluationPeriods": 1,
"DatapointsToAlarm": 1,
"Threshold": 0,
"ComparisonOperator": "GreaterThanOrEqualToThreshold",
"TreatMissingData": "missing"
}
}
When configuring an Amazon CloudWatch alarm, you can specify an Amazon Simple Notification Service (SNS) topic. Messages relating to the alarm will be sent to this topic.
Amazon SNS is a 'publish-subscribe' model, meaning that recipients need to subscribe to the topic to receive a notification. Subscription types include:
Email
AWS Lambda functions
All subscribers to a topic will receive messages sent to the topic.
When a subscribed AWS Lambda function receives a message from an SNS topic, it is provided with details of the alarm that triggered the message. You can use this information to store relevant data in the database.
Related
The following flow is executing Lambda.
monitor log files in EC2 with cloudwatch logs
Detects monitored strings with a metrics filter
Execute Lambda with alarm
I would like to know how to get the following information within Lambda.
Path of the log file being monitored
Instance name
Instance id
Alarm name
I am writing in python and trying to get it using boto3.
You can easily achieve this in 2 ways:-
Create a cloudwatch event bridge rule with event type as cloudwatch
alarm state change.
Whenever your alarm will be in an alarm state it will send an event, configure the target of this event type as lambda function or sns topic, whatever suits your need.
Sample event from this rule
{
"version": "0",
"id": "c4c1c1c9-6542-e61b-6ef0-8c4d36933a92",
"detail-type": "CloudWatch Alarm State Change",
"source": "aws.cloudwatch",
"account": "123456789012",
"time": "2019-10-02T17:04:40Z",
"region": "us-east-1",
"resources": ["arn:aws:cloudwatch:us-east-1:123456789012:alarm:ServerCpuTooHigh"],
"detail": {
"alarmName": "ServerCpuTooHigh",
"configuration": {
"description": "Goes into alarm when server CPU utilization is too high!",
"metrics": [{
"id": "30b6c6b2-a864-43a2-4877-c09a1afc3b87",
"metricStat": {
"metric": {
"dimensions": {
"InstanceId": "i-12345678901234567"
},
"name": "CPUUtilization",
"namespace": "AWS/EC2"
},
"period": 300,
"stat": "Average"
},
"returnData": true
}]
},
"previousState": {
"reason": "Threshold Crossed: 1 out of the last 1 datapoints [0.0666851903306472 (01/10/19 13:46:00)] was not greater than the threshold (50.0) (minimum 1 datapoint for ALARM -> OK transition).",
"reasonData": "{\"version\":\"1.0\",\"queryDate\":\"2019-10-01T13:56:40.985+0000\",\"startDate\":\"2019-10-01T13:46:00.000+0000\",\"statistic\":\"Average\",\"period\":300,\"recentDatapoints\":[0.0666851903306472],\"threshold\":50.0}",
"timestamp": "2019-10-01T13:56:40.987+0000",
"value": "OK"
},
"state": {
"reason": "Threshold Crossed: 1 out of the last 1 datapoints [99.50160229693434 (02/10/19 16:59:00)] was greater than the threshold (50.0) (minimum 1 datapoint for OK -> ALARM transition).",
"reasonData": "{\"version\":\"1.0\",\"queryDate\":\"2019-10-02T17:04:40.985+0000\",\"startDate\":\"2019-10-02T16:59:00.000+0000\",\"statistic\":\"Average\",\"period\":300,\"recentDatapoints\":[99.50160229693434],\"threshold\":50.0}",
"timestamp": "2019-10-02T17:04:40.989+0000",
"value": "ALARM"
}
}
}
Inside your cloud watch alarm there is an alarm action there your can add SNS topic to it and then you can easily get your event information, if you want to process it further,you can add lambda to SNS topic.
I'm trying to build an Amazon EventBridge rule that runs on a schedule(weekly), to put an event in the SQS.
There are multiple options to choose from, as to what message is to be sent as an event.
I understand that it's essentially a JSON object, which can be set to a custom JSON, or the default(or some seletive fields from this) Something like:
{
"version": "0",
"id": "6a7e8feb-b491-4cf7-a9f1-bf3703467718",
"detail-type": "EC2 Instance State-change Notification",
"source": "aws.ec2",
"account": "111122223333",
"time": "2017-12-22T18:43:48Z",
"region": "us-west-1",
"resources": [
"arn:aws:ec2:us-west-1:123456789012:instance/i-1234567890abcdef0"
],
"detail": {
"instance-id": " i-1234567890abcdef0",
"state": "terminated"
}
}
AWS EventBridge: https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-create-rule-schedule.html
EB Events: https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-events.html
My question is: How can I send a JSON Object that has a different parameters every time?
Say I want to publish this object, with a date-range different, with
activeFrom: Today-7 days
activeTill: Today's date.
{
"dummyId": "xyz",
"activeFrom": "2021-07-09T18:43:48Z",
"activeTill": "2021-07-15T18:43:48Z"
}
You can let the EventBridge trigger a lambda function on schedule. In that lambda, you can bake your JSON and send the event to SQS.
I'm currently using SNS to fan out messages to a few SQS queues.
Trying to parse the SQS message and was wondering if there is an out of the box message model to parse the queue message body in to because it's a SNS notification.
Is there such out-of-the-box model available for this kind of scenario?
E.g.: For S3+SNS events -> com.amazonaws.services.s3.event.S3EventNotification
I checked the SQS/ SNS Java SDKs and couldn't find anything similar.
Does this mean the model has to be built in the application code to parse such message bodies?
E.g.:
In the below example queue message, "Body" section is a SNS notification where as I'm interested in "Body -> "Message" section. Can the "Body" be parsed in to an out-of-the-box AWS model in this case?
{
"Body": {
"Type": "Notification",
"MessageId": "272a7e6b-ea5e-46c3-991f-3563d7cd3f09",
"Token": null,
"TopicArn": "arn:aws:sns:us-east-1:000000000000:contact-subscription-topic",
"Message": {
"type": "contactSubInstantiationMessage"
},
"SubscribeURL": null,
"Timestamp": "2020-10-20T03:54:14.022Z",
"SignatureVersion": "1",
"Signature": "EXAMPLEpH+..",
"SigningCertURL": "https://sns.us-east-1.amazonaws.com/SimpleNotificationService-0000000000000000000000.pem"
},
"Attributes": {
"ApproximateFirstReceiveTimestamp": "1603166057169",
"SenderId": "AIDAIT2UOQQY3AUEKVGXU",
"ApproximateReceiveCount": "1",
"SentTimestamp": "1603166054040"
},
"ReceiptHandle": "ibetdkwxaxkqzjxhrkqtgtbrneyylminkvatzwcounxxnubhdktzzkdqrgzxqsebrdfuuxpwnhbuyhvrcbrwxbfgvgdekcygsgauxtcmouzzhlyqvaazkpqmvmmpixbhnpfpldlgjzcnkmaupbikegthoqvdmxyjcvetpisdzxpxrtsrtxvpbmyln",
"MD5OfBody": "9657ff8451167353e3d11c492d99d15f",
"MessageId": "879b6742-8006-bef5-d233-f7b8c8bb33d7"
}
Appreciate your thoughts on this.
Thanks team!
The following post describes a way to eliminate the above mentioned SNS metadata by setting "RawMessageDelivery" to "true" for the subscription;
Amazon SNS -> SQS message body
https://docs.aws.amazon.com/sns/latest/dg/sns-large-payload-raw-message-delivery.html
Marking the issue as resolved.
Thanks.
I have integrated pagerduty with AWS cloudwatch and I am trying to publish a message manually to a SNS Topic that is subscribed by pagerduty and email. But I am not able to get incidents in pagerduty. However, cloudwatch alarms are triggering incidents in pagerduty using this same topic.
I referred some document for pagerduty message payload. But unable to make it work. My SNS message JSON is as follows,
{
"default":"test message",
"email":"test email message",
"https":{
"service_key":"XXXX",
"event_type":"trigger",
"description":"Example alert on host1.example.com"
}
}
Its not triggering an incident in pagerduty. I am not sure what I am missing in the request body. I am receiving email messages properly from this same message body. Could someone point out the mistake?
Thanks in advance.
To do so, you must choose the option Custom Event Transformer for the PagerDuty Integration. In the integration, you can write your own JavaScript code as follows:
var normalized_event = {
event_type: PD.Trigger,
description: "SNS Event",
details: PD.inputRequest
};
PD.emitGenericEvents([normalized_event]);
To parse the received payload from SNS, you can use:
var rawBody = PD.inputRequest.rawBody;
var obj = JSON.parse(unescape(rawBody));
And treat obj to treat your event according to your SNS message.
I believe PagerDuty's native AWS CloudWatch integration is opinionated. So a Custom SNS message won't trigger an incident.
But PagerDuty has an inbound integration type that allows you to create a script using JS (ES5) to parse any custom message sent to the this integration - which can then trigger an incident based on the logic of your script.
Docs on the Custom Event Transformer: https://v2.developer.pagerduty.com/docs/creating-an-integration-inline
I'm too late to answer this but still adding as #filipebarretto has suggested we need to use Custom Event Transformer for this type of integration.
Setup: ~ AWS Cloudwatch (RDS Metric) -> AWS SNS -> PagerDuty (CET)
I have successfully integrated AWS SNS to PagerDuty via Custom Event Transformer
var body = JSON.parse(PD.inputRequest.rawBody)
var message = body.NewStateReason
var normalized_event = {
event_type: PD.Trigger,
description: body.AlarmName,
details: message
};
PD.emitGenericEvents([normalized_event]);
The above code will send incident as AlarmName and details as NewStateReason.
I tested with below sample events as SNS message, it works fine.
{
"version": "0",
"id": "bba1bcef-5268-9967-8628-9a6d09e042e9",
"detail-type": "CloudWatch Alarm State Change",
"source": "aws.cloudwatch",
"account": "[Account ID]",
"time": "2020-11-17T06:25:42Z",
"region": "[region Id]",
"resources": [
"arn:aws:cloudwatch:[region Id]:[Account ID]:alarm:CPUUtilize"
],
"detail": {
"alarmName": "CPUUtilize",
"state": {
"value": "ALARM",
"reason": "Threshold Crossed: 1 out of the last 1 datapoints [4.314689265544354 (17/11/20 06:20:00)] was less than the threshold (70.0) (minimum 1 datapoint for OK -> ALARM transition).",
"reasonData": {
"version": "1.0",
"queryDate": "2020-11-17T06:25:42.491+0000",
"startDate": "2020-11-17T06:20:00.000+0000",
"statistic": "Average",
"period": 300,
"recentDatapoints": [
4.314689
],
"threshold": 70
},
"timestamp": "2020-11-17T06:25:42.493+0000"
},
"previousState": {
"value": "OK",
"reason": "Threshold Crossed: 1 out of the last 1 datapoints [4.484088172640544 (17/11/20 05:44:00)] was not greater than or equal to the threshold (70.0) (minimum 1 datapoint for ALARM -> OK transition).",
"reasonData": {
"version": "1.0",
"queryDate": "2020-11-17T05:49:53.688+0000",
"startDate": "2020-11-17T05:44:00.000+0000",
"statistic": "Average",
"period": 300,
"recentDatapoints": [
4.484088
],
"threshold": 70
},
"timestamp": "2020-11-17T05:49:53.691+0000"
},
"configuration": {
"description": "Alarm Notification in my local timezone",
"metrics": [
{
"id": "16baea70-421b-0a6e-f6f1-bc913d2bf647",
"metricStat": {
"metric": {
"namespace": "AWS/EC2",
"name": "CPUUtilization",
"dimensions": {
"InstanceId": "i-0e448XXXXXXXXXXXX"
}
},
"period": 300,
"stat": "Average"
},
"returnData": true
}
]
}
}
}
Took from https://aws.amazon.com/blogs/mt/customize-amazon-cloudwatch-alarm-notifications-to-your-local-time-zone-part-1/
I am even later to the game here, but ...
How are you 'manually' sending the events? Did you check that the Policy on the SNS topic allows publishing of notifications from whichever service you are using to publish the events?
I had a similar issue with publishing notifications/events from AWS Backup. I had to add something like this to the Access Policy:
{
"Sid": "My-statement-id",
"Effect": "Allow",
"Principal": {
"Service": "backup.amazonaws.com"
},
"Action": "SNS:Publish",
"Resource": "arn:aws:sns:region:account-id:myTopic"
}
I have created cloudwatch alarms for cloudtrail events. I am getting the email notification whenever there is a state change. But It is tough for me to search for the instance which is deleted among hundreds of instances. It will be easier if I get the instance name in the notification Email. Have anyone tried this?
The best method is:
Create an Amazon SNS topic to receive the notification
Subscribe to the topic to receive notifications (eg via Email)
Create a rule in Amazon CloudWatch Events to trigger when an instance is terminated:
The result will be a message like this sent via email (or however you subscribed to the topic):
{
"version": "0",
"id": "0c921724-d932-9cc2-b620-4053a0ad3f73",
"detail-type": "EC2 Instance State-change Notification",
"source": "aws.ec2",
"account": "123456789012",
"time": "2018-01-09T07:04:42Z",
"region": "ap-southeast-2",
"resources": [
"arn:aws:ec2:ap-southeast-2:123456789012:instance/i-0a32beef35b8da342"
],
"detail": {
"instance-id": "i-0a32beef35b8da342",
"state": "terminated"
}
}