I set up a CPU alarm for an EC2 instance that triggers an SNS Topic that has an endpoint that is a Lambda function. The Lambda function will then send ma an email and slack message telling me that an instance is in the alarm start and tell me exactly what instance it came from. I have the email and slack working and now I just need to get the instance ID from the event that my Lambda received from the alarm.
I get the following event in the Lambda function. I want to just pull out the instance ID from it, which in this example would be "i-07db9e2f61d100". It is located in "Dimensions".
How about also pulling out the "AlarmName" (which would be "cpu-mon" in this example)?
Here is all the data in the event I receive:
{'Records': [{'EventSource': 'aws:sns', 'EventVersion': '1.0', 'EventSubscriptionArn': 'arn:aws:sns:us-east-2:Alarm-test:db99f3fe-1c4b', 'Sns': {'Type': 'Notification', 'MessageId': '9921c85a-6f59-50c0', 'TopicArn': 'arn:aws:sns:us-east-2:4990:Alarm-test', 'Subject': 'ALARM: "cpu-mon" in US East (Ohio)', 'Message': '{"AlarmName":"cpu-mon","AlarmDescription":"Alarm when CPU exceeds 70 percent","AWSAccountId":"000000000","NewStateValue":"ALARM","NewStateReason":"Threshold Crossed: 2 out of the last 2 datapoints [99.8333333333333 (26/08/19 19:19:00), 99.1803278688525 (26/08/19 19:18:00)] were greater than the threshold (70.0) (minimum 2 datapoints for OK -> ALARM transition).","StateChangeTime":"2019-08-26T19:20:52.350+0000","Region":"US East (Ohio)","OldStateValue":"OK","Trigger":{"MetricName":"CPUUtilization","Namespace":"AWS/EC2","StatisticType":"Statistic","Statistic":"AVERAGE","Unit":"Percent","Dimensions":[{"value":"i-07db9e2f61d100","name":"InstanceId"}],"Period":60,"EvaluationPeriods":2,"ComparisonOperator":"GreaterThanThreshold","Threshold":70.0,"TreatMissingData":"","EvaluateLowSampleCountPercentile":""}}', 'Timestamp': '2019-08-26T19:20:52.403Z', 'SignatureVersion': '1', 'Signature': 'UeWhS==', 'SigningCertUrl': 'https://sns.us-east-2.amazonaws.com/SimpleNotificationService-63f9.pem', 'UnsubscribeUrl': 'https://sns.us-east-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-2:49:Alarm-test:dfe-1c4b-4db9', 'MessageAttributes': {}}}]}
Here is my Lambda function (python) -
# Sends Slack and text message
import json
import subprocess
import boto3
session = boto3.Session(
region_name="us-east-1"
)
sns_client = session.client('sns')
def lambda_handler(event, context):
print("THIS IS THE EVENT - " + str(event))
data = json.dumps({'text': str(event)})
# Send text alerts
alertNumbers = ["1-xxx-xxx-xxxx"]
# Send text message
for i in range(len(alertNumbers)):
sns_client.publish(
PhoneNumber=alertNumbers[i],
Message=msg,
MessageAttributes={
'AWS.SNS.SMS.SenderID': {
'DataType': 'String',
'StringValue': 'SENDERID'
},
'AWS.SNS.SMS.SMSType': {
'DataType': 'String',
'StringValue': 'Promotional'
}
}
)
# Send Slack message
subprocess.call([
'curl',
'-X', 'POST',
'-H', 'Content-type: application/json',
'--data', data,
'https://hooks.slack.com/services/000000'
Thanks for any help!
You simply need to access the data of the event and put it where you want it.
Inside your lambda_handler add this as the first line:
message = json.loads(event['Records'][0]['Sns']['Message'])
Now the SNS message is available as message. To get the AlarmName is as simple as message['AlarmName'] and the instance id is at message['Trigger']['Dimensions'][0]['value']
Related
I'm trying to create a SNS topic in AWS and subscribe a lambda function to it that will send notifications to Slack apps/users.
I did read this article -
https://aws.amazon.com/premiumsupport/knowledge-center/sns-lambda-webhooks-chime-slack-teams/
that describes how to do it using this lambda code:
#!/usr/bin/python3.6
import urllib3
import json
http = urllib3.PoolManager()
def lambda_handler(event, context):
url = "https://hooks.slack.com/services/xxxxxxx"
msg = {
"channel": "#CHANNEL_NAME",
"username": "WEBHOOK_USERNAME",
"text": event['Records'][0]['Sns']['Message'],
"icon_emoji": ""
}
encoded_msg = json.dumps(msg).encode('utf-8')
resp = http.request('POST',url, body=encoded_msg)
print({
"message": event['Records'][0]['Sns']['Message'],
"status_code": resp.status,
"response": resp.data
})
but the problem is, that in that implementation I have to create a lambda function for every user.
I want to subscribe multiple Slack apps/users to one SNS topic.
Is there a way of doing that without creating a lambda function for each one?
You really DON'T need Lambda. Just SNS and SLACK are enough.
I found a way to integrate AWS SNS with slack WITHOUT AWS Lambda or AWS chatbot. With this approach you can confirm the subscription easily.
Follow the video which show all the step clearly.
https://www.youtube.com/watch?v=CszzQcPAqNM
Steps to follow:
Create slack channel or use existing channel
Create a work flow with selecting Webhook
Create a variable name as "SubscribeURL". The name
is very important
Add the above variable in the message body of the
workflow Publish the workflow and get the url
Add the above Url as subscription of the SNS You will see the subscription URL in the
slack channel
Follow the URl and complete the subscription
Come back to the work flow and change the "SubscribeURL" variable to "Message"
The publish the
message in SNS. you will see the message in the slack channel.
Hi i would say you should go for a for loop and make a list of all the users. Either manually state them in the lambda or get them with api call from slack e.g. this one here: https://api.slack.com/methods/users.list
#!/usr/bin/python3.6
import urllib3
import json
http = urllib3.PoolManager()
def lambda_handler(event, context):
userlist = ["name1", "name2"]
for user in userlist:
url = "https://hooks.slack.com/services/xxxxxxx"
msg = {
"channel": "#" + user, # not sure if the hash has to be here
"username": "WEBHOOK_USERNAME",
"text": event['Records'][0]['Sns']['Message'],
"icon_emoji": ""
}
encoded_msg = json.dumps(msg).encode('utf-8')
resp = http.request('POST',url, body=encoded_msg)
print({
"message": event['Records'][0]['Sns']['Message'],
"status_code": resp.status,
"response": resp.data
})
Another solution you can do is set up email for the slack users, see link:
https://slack.com/help/articles/206819278-Send-emails-to-Slack
When you can just add the emails as subscribers to the sns topic. You can fileter the msg that the receiver gets with Subscription filter policy.
I have a device publishing through a gateway on the events topic (/devices/<dev_id>/events/motion) to PubSub. It's landing in PubSub correctly but subFolder is just an empty string.
On the gateway I'm publishing using the code below. f"mb.{device_id}" is the device_id (not the gateway ID and attribute could be anything - motion, temperature, etc
def report(self, device_id, attribute, value):
topic = f"/devices/mb.{device_id}/events/{attribute}"
timestamp = datetime.utcnow().timestamp()
client.publish(topic, json.dumps({"v": value, "ts": timestamp}))
And this is the cloud function listening on the PubSub queue.
def iot_to_bigtable(event, context):
payload = json.loads(base64.b64decode(event["data"]).decode("utf-8"))
timestamp = payload.get("ts")
value = payload.get("v")
if not timestamp or value is None:
raise BadDataException()
attributes = event.get("attributes", {})
device_id = attributes.get("deviceId")
registry_id = attributes.get("deviceRegistryId")
attribute = attributes.get("subFolder")
if not device_id or not registry_id or not attribute:
raise BadDataException()
A sample of the event in Pub/Sub:
{
#type: 'type.googleapis.com/google.pubsub.v1.PubsubMessage',
attributes: {
deviceId: 'mb.26727bab-0f37-4453-82a4-75d93cb3f374',
deviceNumId: '2859313639674234',
deviceRegistryId: 'mb-staging',
deviceRegistryLocation: 'europe-west1',
gatewayId: 'mb.42e29cd5-08ad-40cf-9c1e-a1974144d39a',
projectId: 'mb-staging',
subFolder: ''
},
data: 'eyJ2IjogImxvdyIsICJ0cyI6IDE1OTA3NjgzNjcuMTMyNDQ4fQ=='
}
Why is subFolder empty? Based on the docs I was expecting it to be the attribute (i.e. motion or temperature)
This issue has nothing to do with Cloud IoT Core. It is instead caused by how Pub/Sub handles failed messages. It was retrying messages from ~12 hours ago that had failed (and didn't have an attribute).
You fix this by purging the Subscription in Pub/Sub.
I have an SQS that triggers a Lambda function. The Lambda function is just receiving the messsage and putting it in a DynamoDB.
It works fine, but the problem is that i noted that the message is deleted from the SQS without the need to add delete() statement in my code.
But in the code it's clearly mentionned that the message should be manually deleted by the consumer otherwise it wil be putted again in the SQS.
What's going on here ?
I want to deal with situation where there will be a problem with the process and in that case the message should reappear again in the SQS so another Lambda can try to process it.
Here is my Lambda code :
import json
import time
import boto3
def lambda_handler(event, context):
message_id = event['Records'][0]['messageId']
message_receipt_handle = event['Records'][0]['receiptHandle']
message_body = event['Records'][0]['body']
print('Message received :')
print(message_body)
print('Processing message ...')
dynamo_db = boto3.client('dynamodb')
response_db = dynamo_db.put_item(
TableName='sqs-test-sbx',
Item={
'id': {
'S': message_id,
},
'Message': {
'S': message_body,
}
}
)
print('dynamodb response :')
print(response_db)
# Simulate a proceesing ...
time.sleep(10)
print('Message processed')
return {
'statusCode': 200,
'message_id': message_id,
'message_body': message_body,
'event': json.dumps(event)
}
That is normal behavior, when you trigger the lambda directly from SQS
https://docs.aws.amazon.com/en_gb/lambda/latest/dg/with-sqs.html
When your function successfully processes a batch, Lambda deletes its
messages from the queue.
You need to delete the message, when you fetch the messages by your own from SQS for instancde from a EC2 instance.
I'm using python Boto3 code, when an instance is terminated from Auto Scaling group it notifies SNS which publishes the message to SQS. Lambda is also triggered when SNS is notified, which executes a boto script to grab the message from SQS.
I am using reference code from Sending and Receiving Messages in Amazon SQS.
Here is the code snippet:
if messages.get('Messages'):
m = messages.get('Messages')[0]
body = m['Body']
print('Received and deleted message: %s' % body)
The result is:
START RequestId: 1234-xxxxxxxx Version: $LATEST
{
"Type" : "Notification",
"MessageId" : "d1234xxxxxx",
"TopicArn" : "arn:aws:sns:us-east-1:xxxxxxxxxx:AutoScale-Topic",
"Subject" : "Auto Scaling: termination for group \"ASG\"",
"Message" : "{\"Progress\":50,\"AccountId\":\"xxxxxxxxx\",\"Description\":\"Terminating EC2 instance: i-123456\",\"RequestId\":\"db-xxxxx\",\"EndTime\":\"2017-07-13T22:17:19.678Z\",\"AutoScalingGroupARN\":\"arn:aws:autoscaling:us-east-1:360695249386:autoScalingGroup:fef71649-b184xxxxxx:autoScalingGroupName/ASG\",\"ActivityId\":\"db123xx\",\"EC2InstanceId\":\"i-123456\",\"StatusCode\"\"}",
"Timestamp" : "2017-07-",
"SignatureVersion" : "1",
"Signature" : "",
"SigningCertURL" : "https://sns.us-east-1.amazonaws.com/..",
"UnsubscribeURL" : "https://sns.us-east-1.amazonaws.com/
}
I only need EC2InstanceId of the terminated instance not the whole message. How can I extract the ID?
If your goal is to execute an AWS Lambda function (having the EC2 Instance ID as a parameter), there is no need to also publish the message to an Amazon SQS queue. In fact, this would be unreliable because you cannot guarantee that the message being retrieved from the SQS queue matches the invocation of your Lambda function.
Fortunately, when Auto Scaling sends an event to SNS and SNS then triggers a Lambda function, SNS passes the necessary information directly to the Lambda function.
Start your Lambda function with this code (or similar):
def lambda_handler(event, context):
# Dump the event to the log, for debugging purposes
print("Received event: " + json.dumps(event, indent=2))
# Extract the EC2 instance ID from the Auto Scaling event notification
message = event['Records'][0]['Sns']['Message']
autoscalingInfo = json.loads(message)
ec2InstanceId = autoscalingInfo['EC2InstanceId']
Your code then has the EC2 Instance ID, without having to use Amazon SQS.
The instance id is in the message. It's raw JSON, so you can parse it with the json package and get the information.
import json
if messages.get('Messages'):
m = messages.get('Messages')[0]
body = m['Body']
notification_message = json.loads(body["Message"])
print('instance id is: %s' % notification_message["EC2InstanceId"])
I try to create an alarm if my a json log message is of error log level. My filter works fine but when I create my alarm, it always fails on insufficient data. Seemingly, because there are no errors.
Any ideas?
The way around this was to define two metrics with the same name but with inverse filters. The filter that matches error level log messages must return a metric value of 1, while the second filter should match all messages, or at least one message within the time period and return a metric value of 0. The presence of the 0 value avoids the insufficient data error.
When the alarm is created from the metric, both filters are combined. If a sum statistic is applied and a alarm rule of >0 applied, the alarm will trigger only when error messages arrive and not run into insufficient data.
Here is an example using the boto3 client:
import boto3
client = boto3.client('logs')
logGroupName = 'myLogGroup'
# create this SNS topic with your email subscription...
env['aws_sns_arn_error_email'] = 'arn:aws:sns:eu-west-1:1234567:log_error'
env['sys_type'] = 'production'
metricsNamespace = 'LogMetrics'
metricName = 'ErrorCount' + "_%(sys_type)s" % env
print colors.cyan('Put metric $(metricName)s' % env)
cloudwatch_client = boto3.client('cloudwatch')
response = cloudwatch_client.put_metric_data(
Namespace=metricsNamespace,
MetricData=[
{
'MetricName': metricName,
'Unit': 'Count',
'Value': 1
},
]
)
logs_client = boto3.client('logs')
print colors.cyan('Put metric filter $.levelname-ERROR')
logs_client.put_metric_filter(
logGroupName=env.log_group_name_ea,
filterName='levelname-ERROR',
filterPattern='{ $.levelname = "ERROR" }',
metricTransformations=[
{
'metricNamespace': metricsNamespace,
'metricValue': '0',
'metricName': metricName,
}]
)
print colors.cyan('Put metric filter catchAll')
logs_client.put_metric_filter(
logGroupName=env.log_group_name_ea,
filterName="catchAll",
filterPattern='',
metricTransformations=[
{
'metricNamespace': metricsNamespace,
'metricValue': '1',
'metricName': metricName,
}]
)
print colors.cyan('Put metric alarm, email on error')
response = cloudwatch_client.put_metric_alarm(
AlarmName='email on error',
AlarmDescription='email on error',
ActionsEnabled=True,
AlarmActions=[
env.aws_sns_arn_error_email,
],
MetricName=metricName,
Namespace=metricsNamespace,
Statistic='Sum',
Period=300,
Unit='Count',
EvaluationPeriods=1,
Threshold=0,
ComparisonOperator='GreaterThanThreshold'
)