Adding Custom Metric to CloudFormation template - amazon-web-services

I need to add an alarm the triggers a rollback in CloudFormation. I've settled on adding a custom metric that can be assigned manually. I'm looking at the documentation here. This looks like it would work but I need it to be added to the template. Is this possible? The way I'm thinking initially the value would be 0 failures like below
aws cloudwatch put-metric-data --metric-name Failures --namespace MyNamespace --value 0
when I want to trigger a rollback I would set it to 1
aws cloudwatch put-metric-data --metric-name Failures --namespace MyNamespace --value 1
But I need this to be in the template so that I can have access to the correct/dynamic value of the namespace when defining the alarm.

You can use cloudformation simple template to add metrics
resource "AWS::Logs::MetricFilter" specifies a metric filter that
describes how CloudWatch Logs extracts information from logs and
transforms it into Amazon CloudWatch metrics. If you have multiple
metric filters that are associated with a log group, all the filters
are applied to the log streams in that group.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-logs-metricfilter.html
sample
404MetricFilter:
Type: AWS::Logs::MetricFilter
Properties:
LogGroupName:
Ref: "myLogGroup"
FilterPattern: "[ip, identity, user_id, timestamp, request, status_code = 404, size]"
MetricTransformations:
-
MetricValue: "1"
MetricNamespace: "WebServer/404s"
MetricName: "404Count"

unless you want to do this via a cloudformation custom resource
you can use this as well Running bash commands in AWS CloudFormation templates
With the above you can run your command like
Resources:
CommandRunner:
Type: AWSUtility::CloudFormation::CommandRunner
Properties:
Command: 'aws cloudwatch put-metric-data --metric-name Failures --namespace MyNamespace --value 1'

Related

AWS Cloudwatch Alarms - One alarm for multiple dimension values possible?

In my AWS organization we have multiple (5+) accounts and a Cloudtrail trail for management events with Cloudwatch log group for all the accounts and regions in the org.
For the log group there is a series of generic metric filters setup, each with a Cloudwatch alarm. These are all tied to one or more SNS topics and associated subscriptions and all out of the box.
For example:
aws logs put-metric-filter --log-group-name CLOUDWATCH-LOG-GROUP --filter-name FILTER-NAME --metric-transformations metricName=METRIC-NAME,metricNamespace=NAMESPACE,metricValue=1 --filter-pattern '{ ($.eventName = ConsoleLogin) && ($.errorMessage = "Failed authentication") }'
Creates a metric filter for any failed logins on console for any of the accounts that sets the metric value to 1 when a match for the filter pattern is found in the log.
Paired with
aws cloudwatch put-metric-alarm --alarm-name METRIC-NAME-alarm --metric-name METRIC-NAME --statistic Sum --period 300 --threshold 1 --comparison-operator GreaterThanOrEqualToThreshold --evaluation-periods 1 --namespace NAMESPACE --alarm-actions SNS-TOPIC-ARN
This creates a simple alarm that fires an event to the SNS topic.
The major drawback here however is if my org account ID is #1234 and my worker account ID is #4321 and the failed login happens at the worker account, the Alarm that is sent refers only to the org account ID and I need to login and look to figure out where this happened.
Now, I can add a dimension to the metric-transformation to include something to help me differentiate (for example recipientAccountId). That would look something like
aws logs put-metric-filter --log-group-name CLOUDWATCH-LOG-GROUP --filter-name FILTER-NAME --metric-transformations metricName=METRIC-NAME,metricNamespace=NAMESPACE,metricValue=1,dimensions={"Account"=($.recipientAccountId)} --filter-pattern '{ ($.eventName = ConsoleLogin) && ($.errorMessage = "Failed authentication") }'
But, in order for the Cloudwatch alarm to trigger, it appears that the dimension must also be included there, it isn't possible to have one Alarm for all the possible Value that exist. Which then seems to require one Alarm per recipientAccountId (ACCOUNT-ID in the below).
Ex:
aws cloudwatch put-metric-alarm --alarm-name METRIC-NAME-alarm --metric-name METRIC-NAME --statistic Sum --period 300 --threshold 1 --comparison-operator GreaterThanOrEqualToThreshold --evaluation-periods 1 --dimensions Name=Account,Value=ACCOUNT-ID --namespace NAMESPACE --alarm-actions SNS-TOPIC-ARN
Is this correct?
All of this is really just to include some metadata with the alarm to let me know which Account triggered, is there a better solution?

How to attach a Metric to an existing Lambda log group?

I'm trying to create a custom metric using CloudFormation. I've followed the example from https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-logs-metricfilter.html. My existing lambda logs to a CloudWatch log group that shows up in CloudWatch as /aws/lambda/my-function-name. Here's my CloudFormation YAML for the metric:
ErrorsLogMetric:
Type: AWS::Logs::MetricFilter
Properties:
LogGroupName: !Sub "/aws/lambda/${AWS::StackName}"
FilterPattern: "[ERROR]"
MetricTransformations:
- MetricValue: "1"
MetricNamespace: "LogMetrics"
MetricName: "MyCustomMetric"
${AWS::StackName} resolves to "my-function-name" when it runs. The CloudFormation script runs successfully and says the metric was created, but when I go to CloudWatch the log group for my lambda, it shows zero filters. What do I need to do differently to cause this custom metric to show up as a filter for my lambda log group when it is created via CloudFormation?
If I hard code the property as LogGroupName: "/aws/lambda/my-function-name" then it works. But I don't want to hard code it since the value of ${AWS::StackName} is dynamic in different use cases.
It seemed to be for me the solution was to add
DependsOn: LambdaLogGroup
where LambdaLogGroup was the resource defined earlier in my template. It threw me off because the log group had already existed for some time, and I was attempting to update the stack and add the metric. Something about that still required the dependency even though the log group already existed and was not updated by CloudFormation when I added the Metric.

How to setup retention for CloudWatch log in CloudFormation YAML template?

In the console it's like super easy. But I just couldn't find how to do in the YAML template for CloudFormation?
The AWS::Logs::LogGroup - AWS CloudFormation documentation shows:
myLogGroup:
Type: "AWS::Logs::LogGroup"
Properties:
RetentionInDays: 7
RetentionInDays
The number of days log events are kept in CloudWatch Logs. When a log event expires, CloudWatch Logs automatically deletes it.

AWS CLI: Attach an existing cloudwatch alarm to an existing auto scaling policy

I'm having trouble attaching previously created alarms to an auto scaling group policy. I have created the policy like so:
aws autoscaling put-scaling-policy --policy-name my-scaleout-policy --auto-scaling-group-name my-group --scaling-adjustment 2 --adjustment-type ChangeInCapacity
However, I can't seem to find the correct CLI command to attach an already created alarm my-cpu-alarm to the policy.
Update the existing alarm & add the policy ARN in --alarm-actions parameter.
You can use the put-metric-alarm command for this.
Note that you will have to give all the original parameters to update the alarm as they are overwritten.
You can get the alarm parameters using describe-alarm.

Alarm action definition in ec2_metric_alarm ansible module

I am trying to set up an cloud watch alarm witch ansible ec2_metric_alarm module and I do not know how to set it to send an email on alarm
The code is
- name: add alarm
ec2_metric_alarm:
state: present
region: eu-west-1
name: "LoadAverage"
metric: "LoadAverage"
statistic: Average
comparison: ">"
threshold: 3.0
evaluation_periods: 3
period: 60
unit: "None"
description: "Load Average"
dimensions: {'Role':{{itme[0]}}, Node:{{item[1]}} }
alarm_actions: ["action1","action2"]
What is the syntax or what do I do to express that I want it to send emails on in alarm_actions?
The documentation is crappy for this one:
http://docs.ansible.com/ec2_metric_alarm_module.html
Here is what I would try based on boto:
http://docs.pythonboto.org/en/latest/ref/cloudwatch.html#module-boto.ec2.cloudwatch.alarm
alarm_actions (list of strs) – A list of the ARNs of the actions to take in ALARM state
The current supported ARNS are SNS topics or autoscalling policies.
In your case:
You need to create an SNS topic and subscribe your email address to that topic (also confirm the subscription) and after that put the SNS topic ARN as a string in the alarm_actions param that you pass to the ansible ec2_metric_alarm_module.
Hope this helps.
I agree with #Mircea's answer regarding the documentation and its quality. I personally found the solution to the same/similar problem by creating the alarm through the UI with the desired alert action and then using the AWS CLI to extract the alarm string for use with ansible
aws cloudwatch describe-alarms
In the result you can then find the action string:
ALARMACTIONS arn:aws:sns:us-east-1:**Cust Account ID Here**:NotifyMe
ALARMACTIONS arn:aws:swf:us-east-1:**Cust Account ID Here**:action/actions/AWS_EC2.InstanceId.Stop/1.0
In my case I had two actions, one to email me and the other to Stop the EC2 instance
These values can then be used in your ansible task:
alarm_actions: ["arn:aws:swf:{{ aws_region }}:{{ aws_cust_account_id }}:action/actions/AWS_EC2.InstanceId.Stop/1.0", "arn:aws:sns:{{ aws_region }}:{{ aws_cust_account_id }}:NotifyMe"]