I had already created 7 other metrics based on some log files I send to CloudWatch with no problems.
Some time ago we had a problem with MongoDB connection, and I identified that through logs, so I'd like to create a Metric, so that I can create an Alarm based on it. I did create the Metric, but (of course) there are no data being fed into that Metic, because no more "MongoError" messages exists.
But does that also mean that I can't even access the Metric to create the Alarm? Because this is what is happening right now. The Metric cannot be seen anywhere, only in the "Filters" section of the Logs, which won't allow me to create Alarms or create graphics or anything.
I have already posted this on AWS forums but that usually doesn't help.
An Amazon CloudWatch custom metric is only created when data is stored against the custom metric. Therefore, you'll need to push a data value to make appear and then you will be able to create an alarm.
You can push some data to CloudWatch with the AWS Command-Line Interface (CLI), eg:
aws cloudwatch put-metric-data --namespace MongoDB --metric-name errors --value 0
Related
I am trying to send custom metric data of my application to CloudWatch using Cloudwatch Agent.
I am able to successfully send the data to CloudWatch using aws cloudwatch put-metric-data command with AWS CLI, but instead i would like to use the cloudwatch agent to receive my application metrics (connection count , queue count etc) and send that data to cloudwatch via Cloudwatch agent. Please help.
At first, you need to check what metrics CWAgent supports and collects.
Reference: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/metrics-collected-by-CloudWatch-agent.html
From my point of view, CWAgent mostly collects OS metrics that it can read directly from your machine instead of metrics such as connection-count from your application.
The way you did using the put-metric-data is a good direction, you can customize it with your own code.
But recently, AWS did mention about OpenTelemetry which supports application metrics and integrates with CloudWatch Agent. I believe this way is the one you are looking for.
Reference:
https://aws-otel.github.io/docs/introduction
https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-open-telemetry.html
I am trying to send an SNS notification when a backup in the backup vault fails twice consecutively. Is there a CloudWatch alarm or any other way to do this in CloudFormation?
You can use CloudWatch metrics for this purpose and then setup alarms based on the thresholds that you need.
You can find the list of Metrics that are emitted to CloudWatch in this document: https://docs.aws.amazon.com/aws-backup/latest/devguide/cloudwatch.html
For instance you can setup an alarm on NumberOfBackupJobsFailed metric.
I want to create a separate custom metric for each ECS task that I will run when AutoScaling automatically will be creating more tasks. Is there a way to easily implement this using Cloudformation or maybe something similar? Maybe automatically creating a separate log group with a separate metric filter for each task. Or only a metric filter but assigning it to a log stream of the created task.
If the custom metric is published by your task, then you can just start publishing the metrics directly. https://aws.amazon.com/premiumsupport/knowledge-center/cloudwatch-custom-metrics/
You would want to use the dimension of the task identifier. You would get that from the metadata. https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-metadata-endpoint.html
So you would get something like. Note I haven't tried the dimension type of taskArn which might cause a concern, but AWS already publishes cloudwatch events with this field.
aws cloudwatch put-metric-data --metric-name petro_custom_1 --dimensions taskArn=$TaskARN --namespace "Custom" --value $Value
I would like to create a cloudwatch alarm for the sns metric NumberOfMessagesPublished on a new sns topic. If I attempt to do this before I trigger the topic (and there is no data for it) then the option to create an alarm does not exist in the web console.
How can I do this with the web console? Is using a cli tool the only option?
I noticed that too -- you cannot select a metric until the data has been sent to CloudWatch.
I suspect that CloudWatch actually doesn't have any visibility into metrics until they are sent from the originating service into CloudWatch. Thus, it can't populate the console because it doesn't know what data will be sent. That would be a nice, loosely-coupled design.
My experiments show that it is possible create an alarm via the AWS Command-Line Interface (CLI) on metrics that do not exist -- even within the EC2 namespace. That's the way to go!
I suggest you to use Cloudformation. It is the best way to create resources stack in AWS and overall you can easily experiment with it and delete everything when you are done. Here there is a sample application that uses Cloudformation and Cloudwatch https://github.com/awslabs/cloudwatch-dashboards-cloudformation-sample
The manual solution I went with was to create the sns topic and then use the Publish to topic option in the sns menu to send out a test message to my subscribers. This creates a set of notifications without the need to set off the cloudwatch rule. After this the option appears in the cloudwatch alarm creation menu.
The other CLI options would be better for automation.
Yes, it is possible. The AWS Official Documentation answers your question. Just to quote the relevant lines:
You can create an alarm for a custom metric before you've created that custom metric. For the alarm to be valid, you must include all of the dimensions for the custom metric in addition to the metric namespace and metric name in the alarm definition.
Environment – Two Different ec2 instances running tomcat separately.
Requirement – If there is any Error in logs – we should get an alert.
Implementation –
We implemented AWS customer logging for this which is successfully sending alerts on the Error Pattern Matching.
It automatically created a log groups – “/opt/tomcat/logs/catalina.out”.
Under this log group – there are two log streams – two instances separately showing.
Problem –
Now I want separate alarm for separate instances
Problem is when I create an alarm – it does not let me choose the instance. It takes both instance by default, which means one alarm – monitoring both instances simultaneously. And sending alert without mentioning instance name. so it is difficult to find which instance has actually sent alert.
And the second problem is - we created few log metrics for testing – like on keyword – info – which we want to delete and not able to do so.
It appears that you are using the CloudWatch Logs functionality that permits automated sending of log files from an EC2 instance (or elsewhere) to the CloudWatch service. CloudWatch Logs can then be configured to look for strings in the log files, which will trigger the recording of metrics.
To create separate alarms for separate instances, each EC2 instance should be configured to use a different CloudWatch Log stream. The CloudWatch Logs agent takes a Destination Log Group name.
See: Quick Start: Install and Configure the CloudWatch Logs Agent on an Existing EC2 Instance
As for the metrics that you wish to delete, it is not possible to delete metrics from Amazon CloudWatch. However, metrics will automatically disappear after 14 days.