I have put some custom metrics in AWS cloudwatch using CLI.
aws cloudwatch put-metric-data --metric-name queuelength --namespace testservice --value 10000 --unit Count --timestamp 2022-11-06T12:05:00.000Z --region us-east-1
aws cloudwatch put-metric-data --metric-name queuelength --namespace testservice --value 20000 --unit Count --timestamp 2022-11-05T12:05:00.000Z --region us-east-1
I am getting this namespace and metric name on AWS console after pushing the metrics but when I select it for plotting graph it doesn't show any line.
This is the query that was formed from query builder.
SELECT SUM(queuelength) FROM SCHEMA(testservice)
Your timestamps are 3 days in the past. It can take some time for backfilled data to show up.
Try publishing a datapoint more recent to the current time, within the last 3 hours. If that works and you see your data, then the backfilled data should show up within an hour or so.
Related
I Have created simple AWS cloudwatch alarm with console. It is triggered by metric "Cognito-By UserPoolAndUserPoolClient-SignInSuccesses". It works as supposed and sends an email via SNS Topic.
I tried to create same alarm with CLI, but it doesn't work. Alarm doesn't get triggered. I compared in console what is difference between those two alarms. When creating alarm with console in metric selection I see metric with information "UserPool" and "UserPoolClient". When using CLI command there isn't parameters to set those two. How could I configure new alarm with CLI so that it is identical to one created in console?
CLI command:
aws cloudwatch put-metric-alarm --alarm-name NewSignupAlarm --alarm-description "New Signup have been done." --alarm-actions arn:aws:sns:eu-west-3:681278469643:NewTestTopic --metric-name SignUpSuccesses --namespace AWS/Cognito --statistic Sum --period 300 --unit Count --evaluation-periods 1 --threshold 0 --comparison-operator GreaterThanThreshold --profile user3
These values are dimensions on the metric. You can specify them with the --dimensions parameter, like this:
--dimensions Name=UserPool,Value=eu-west-3_... Name=UserPoolClient,Value=117...
Make sure the values are exact, I used ... just to illustrate the usage.
See here for more info on all different parameters that you may need to set if you want the alarm be the same as the one created in the console:
https://docs.aws.amazon.com/cli/latest/reference/cloudwatch/put-metric-alarm.html
I am sending memory metrics of our EC2 instances to cloudwatch using put-metric-data API call. I am giving a custom namespace for these metrics. But is there a way for these namespaces to be nested so that I will be able to consolidate all my metric data?
For example, if we look into AWS service namespaces EC2, the metrics inside these namespaces will be aggregated ImageID, InstanceType, Per Instance Metrics etc. This can further be filtered using dimensions.
Is there any way to aggregate custom metrics like this?
Adding another dimenstion for the same namespace should work
Example
aws cloudwatch put-metric-data --metric-name memory-usage --dimensions sampleDimension=value --namespace "The-Namespace" --unit Percent
aws cloudwatch put-metric-data --metric-name memory-usage --dimensions sampleDimension2=value --namespace "The-Namespace" --unit Percent
I see no reason that this shouldn't work. Anyways, let us know
i have been working on project, and i want to export metrics data from cloudwatch like CPU Utilization and Network Out data, is there any way to get that data? and convert it to csv?
Export CloudWatch Metrics
You can't do it directly but following is the step by step instructions:
Pre Reqs
AWS CLI (Command Line Interface) from here
jq is a lightweight and flexible command-line JSON processor from here
How to export
Use the following CLI:
aws cloudwatch get-metric-statistics
--namespace AWS/EC2 --metric-name CPUUtilization
--dimensions Name=InstanceId,Value=i-xxxxxxxxxxxxxxxxx
--statistics Average
--start-time 2020-03-01T00:00:00
--end-time 2020-03-31T23:59:00
--period 3600
--region us-east-1
Valid options for --metric-name depend on the --name-space parameter. For AWS/EC2, the full list can be seen by running the following CLI command:
aws cloudwatch list-metrics --namespace "AWS/EC2"
Valid options for --statistics are:
SampleCount
Average
Sum
Minimum
Maximum
--start-time and --end-time specify the range.
--period The granularity, in seconds, of the returned data points.
--region The region where the CloudWatch metric is being meatured (us-east-1, us-west-2 etc.)
Data output will look something like the following:
{
"Label": "CPUUtilization",
"Datapoints": []
}
To convert it to CSV we will use jq. For this you have two options:
Option 1
Pipe all the aws cli output to jq:
aws cloudwatch get-metric-statistics
--namespace AWS/EC2 --metric-name CPUUtilization
--dimensions Name=InstanceId,Value=i-xxxxxxxxxxxxxxxxx
--statistics Average
--start-time 2020-03-01T00:00:00
--end-time 2020-03-31T23:59:00
--period 3600
--region us-east-1
| jq -r '.Datapoints[] | [.Timestamp, .Minimum, .Unit] | #csv'
Option 2
Export the data to JSON:
aws cloudwatch get-metric-statistics
--namespace AWS/EC2 --metric-name CPUUtilization
--dimensions Name=InstanceId,Value=i-xxxxxxxxxxxxxxxxx
--statistics Average
--start-time 2020-03-01T00:00:00
--end-time 2020-03-31T23:59:00
--period 3600
--region us-east-1 >> data.json
Use jq to read the json to csv:
jq -r '.Datapoints[] | [.Timestamp, .Minimum, .Unit] | #csv' data.json
Output
Here is how the output will look like:
"2020-03-24T11:00:00Z",0.327868852454245,"Percent"
"2020-03-11T21:00:00Z",0.327868852454245,"Percent"
"2020-03-15T04:00:00Z",0.322580645156596,"Percent"
"2020-03-27T18:00:00Z",0.327868852478101,"Percent"
There is no in-built capability to export Amazon CloudWatch metrics to CSV.
There are API calls available to extract metrics, but you would need to write a program to call the API, receive the metrics and store it in a suitable format.
There are projects available that can assist with this, such as:
mogproject/cloudwatch-dump: Just dump all the CloudWatch metrics.
AWS – CloudWatch – DataExtractor – API | vMan
romgapuz/awsmetric2csv: AWS Metric to CSV is a Python command-line utility to extract CloudWatch metric data from an AWS resources (e.g. EC2, RDS) and save to a CSV file.
If you are looking for these types of tools, make sure they refer to CloudWatch metrics, not CloudWatch Logs (which is something different).
You can also try out official Prometheus cloudwatch_exporter
I am trying to get the cloudwatch metric data for CPU Utilazation of an EC2 instance i-014448f54423cc0, but i am getting the following output without any metrics data
AWS CLI COMMAND
$ aws cloudwatch get-metric-statistics --metric-name CPUUtilization --start-time 2017-03-20T23:18:00 --end-time 2017-03-25T23:18:00 --period 3600 --namespace AWS/EC2 --statistics Average --dimensions Name=i-014448f54423cc0,Value=i-abcdef
OUTPUT
-----------------------------
| GetMetricStatistics |
+--------+------------------+
| Label | CPUUtilization |
+--------+------------------+
As per the following thread on AWS forum: Link
It is said that cloudwatch metrics data can be accessed through API's only if detailed monitoring is enabled, so I tried both scenarios of with & without enabling detailed monitoring but still in both the cases the output is same
You have defined your dimension incorrectly. The dimension you are filtering (literally 'InstanceId') needs to be defined as the Name, and your value (in this case, the instance id's value) as the Value.
Example
Snippet:
Name=InstanceId,Value=i-014448f54423cc0
Full example:
aws cloudwatch get-metric-statistics --metric-name CPUUtilization --start-time 2017-03-20T23:18:00 --end-time 2017-03-25T23:18:00 --period 3600 --namespace AWS/EC2 --statistics Average --dimensions Name=InstanceId,Value=i-014448f54423cc0
Discussion
The linked AWS forum discussion isn't relevant to this situation, because detailed monitoring only enables monitoring at 1-minute granularity. Basic monitoring samples data on five-minute intervals. Since you are requesting CPUUtilization for each hour (period=3600 seconds) on a single instance, you will have data available with no need for detailed monitoring.
Further Reading
AWS Documentation - Enable or Disable Detailed Monitoring for Your Instances
AWS Documentation - Amazon EC2 Metrics and Dimensions
AWS Documentation - AWS CLI get-metric-statistics
I am looking to get a notification schema setup for my EC2 instances that are in the us-west-2 data center. I have read the CloudWatch and SNS docs, and I am aware that SMS notifications are only available when setting up SNS Topics in us-east-1.
I am wondering if anyone has found a way to get CloudWatch alarms for instances NOT in us-east-1 to broadcast on a topic setup in us-east-1?
It appears that the documentation is incorrect - when trying to add a new alarm in us-west-2 that points at a topic in us-east-1 for the SMS capabilities, the following error is observed (tried this today on Sept 10 2014):
A client error (ValidationError) occurred when calling the PutMetricAlarm operation: Invalid region us-east-1 specified. Only us-west-2 is supported.
This is based on running the following command using the aws cli (account number changed to 1234567890):
aws cloudwatch put-metric-alarm --alarm-name "ELB UnHealthy > 0" --alarm-description "ELB UnHealthy > 0" --actions-enabled --ok-actions arn:aws:sns:us-west-2:1234567890:EmailOnly --alarm-actions arn:aws:sns:us-east-1:1234567890:EmailAndSMS --insufficient-data-actions arn:aws:sns:us-west-2:1234567890:EmailOnly --metric-name UnHealthyHostCount --namespace AWS/ELB --statistic Average --dimensions Name=LoadBalancerName,Value=elb-name --period 60 --evaluation-periods 3 --threshold 1 --comparison-operator GreaterThanOrEqualToThreshold
I guess the previous answer was from somebody who didn't actually test this out... It's that or AWS disabled the ability to reference other region sns topics in the last few months and forgot to update their documentation...
You can set up your alarms to send notifications to a different region with the Amazon Cloudwatch CLI. You just have to create the alarm in us-west-2 by calling mon-put-metric-alarm and provide the ARN of your us-east-1 topic as alarm-actions.
export AWS_CLOUDWATCH_URL=http://monitoring.us-west-2.amazonaws.com/
mon-put-metric-alarm --alarm-name my-alarm --alarm-description "some desc"
--metric-name CPUUtilization --namespace AWS/EC2
--statistic Average --period 60 --threshold 90
--comparison-operator GreaterThanThreshold
--dimensions InstanceId=i-abcdef --evaluation-periods 3 --unit Percent
--alarm-actions arn:aws:sns:us-east-1:1234567890:my-topic
You can read the official docs here:
http://docs.aws.amazon.com/AmazonCloudWatch/latest/cli/cli-mon-put-metric-alarm.html
And find CLI setup instructions here:
http://docs.aws.amazon.com/AmazonCloudWatch/latest/cli/SetupCLI.html
If you're trying to create alarms in a region other than your default you must explicitly specify it with the --region option (despite this already being implicit in the ARN).
This isn't exactly what you asked for, but I have instances in the us-west-1 region that I needed SMS alerts for, so I found a way to get this to work. The trick is to create both the metrics and the alerts in the us-east-1 region.
On your instance in the us-west-1 region, run the following command to create a metric in the us-east-1 region and and publish a value to it.
aws cloudwatch put-metric-data
--region us-east-1 --namespace NameSpace --metric-name MetricName
--dimensions InstanceId=i-12345678 --value 10
Next, in the us-east-1 region, create an SNS topic, along with associated SMS subscription(s).
Finally, in the us-east-1 region, the topic you created will be visible. create an alarm based on that metric, specifying the SNS topic you just created.
This alarm will now send SMS notifications via SNS when triggered.
I had thought there might be an additional cost of sending the metric data from us-west-1 to us-east-1, but it turns out that the cost is zero for the number of metrics and frequency I used, which was a single metric sent once per minute.