I am trying to get get-metric-statistics from Cloudwatch, but I found a little difference from what I have in Cloudwatch and what I receive from the following aws cli command:
aws cloudwatch get-metric-statistics --metric-name EnqueueCount
--start-time 2018-08-13T10:15:00Z --end-time 2018-08-13T10:16:01Z --period 60 --namespace AWS/AmazonMQ --statistics SampleCount --dimensions Name=Broker,Value=BrokerNameName=Queue,Value=QueueName
For the same period of time in Cloudwatch I have different EnqueueCount, but from aws cli command I receive the same value all over again.
Does anyone know why is that?
AWS get-metric-statistics considers end-time as exclusive and start-time as inclusive.
In you case, to get the right count set start-time to 2018-08-13T10:15:00Z and end-time to 2018-08-13T10:17:00Z with period of 60s.
For further reading
https://docs.aws.amazon.com/cli/latest/reference/cloudwatch/get-metric-statistics.html
Related
I Have created simple AWS cloudwatch alarm with console. It is triggered by metric "Cognito-By UserPoolAndUserPoolClient-SignInSuccesses". It works as supposed and sends an email via SNS Topic.
I tried to create same alarm with CLI, but it doesn't work. Alarm doesn't get triggered. I compared in console what is difference between those two alarms. When creating alarm with console in metric selection I see metric with information "UserPool" and "UserPoolClient". When using CLI command there isn't parameters to set those two. How could I configure new alarm with CLI so that it is identical to one created in console?
CLI command:
aws cloudwatch put-metric-alarm --alarm-name NewSignupAlarm --alarm-description "New Signup have been done." --alarm-actions arn:aws:sns:eu-west-3:681278469643:NewTestTopic --metric-name SignUpSuccesses --namespace AWS/Cognito --statistic Sum --period 300 --unit Count --evaluation-periods 1 --threshold 0 --comparison-operator GreaterThanThreshold --profile user3
These values are dimensions on the metric. You can specify them with the --dimensions parameter, like this:
--dimensions Name=UserPool,Value=eu-west-3_... Name=UserPoolClient,Value=117...
Make sure the values are exact, I used ... just to illustrate the usage.
See here for more info on all different parameters that you may need to set if you want the alarm be the same as the one created in the console:
https://docs.aws.amazon.com/cli/latest/reference/cloudwatch/put-metric-alarm.html
I have put some custom metrics in AWS cloudwatch using CLI.
aws cloudwatch put-metric-data --metric-name queuelength --namespace testservice --value 10000 --unit Count --timestamp 2022-11-06T12:05:00.000Z --region us-east-1
aws cloudwatch put-metric-data --metric-name queuelength --namespace testservice --value 20000 --unit Count --timestamp 2022-11-05T12:05:00.000Z --region us-east-1
I am getting this namespace and metric name on AWS console after pushing the metrics but when I select it for plotting graph it doesn't show any line.
This is the query that was formed from query builder.
SELECT SUM(queuelength) FROM SCHEMA(testservice)
Your timestamps are 3 days in the past. It can take some time for backfilled data to show up.
Try publishing a datapoint more recent to the current time, within the last 3 hours. If that works and you see your data, then the backfilled data should show up within an hour or so.
i have been working on project, and i want to export metrics data from cloudwatch like CPU Utilization and Network Out data, is there any way to get that data? and convert it to csv?
Export CloudWatch Metrics
You can't do it directly but following is the step by step instructions:
Pre Reqs
AWS CLI (Command Line Interface) from here
jq is a lightweight and flexible command-line JSON processor from here
How to export
Use the following CLI:
aws cloudwatch get-metric-statistics
--namespace AWS/EC2 --metric-name CPUUtilization
--dimensions Name=InstanceId,Value=i-xxxxxxxxxxxxxxxxx
--statistics Average
--start-time 2020-03-01T00:00:00
--end-time 2020-03-31T23:59:00
--period 3600
--region us-east-1
Valid options for --metric-name depend on the --name-space parameter. For AWS/EC2, the full list can be seen by running the following CLI command:
aws cloudwatch list-metrics --namespace "AWS/EC2"
Valid options for --statistics are:
SampleCount
Average
Sum
Minimum
Maximum
--start-time and --end-time specify the range.
--period The granularity, in seconds, of the returned data points.
--region The region where the CloudWatch metric is being meatured (us-east-1, us-west-2 etc.)
Data output will look something like the following:
{
"Label": "CPUUtilization",
"Datapoints": []
}
To convert it to CSV we will use jq. For this you have two options:
Option 1
Pipe all the aws cli output to jq:
aws cloudwatch get-metric-statistics
--namespace AWS/EC2 --metric-name CPUUtilization
--dimensions Name=InstanceId,Value=i-xxxxxxxxxxxxxxxxx
--statistics Average
--start-time 2020-03-01T00:00:00
--end-time 2020-03-31T23:59:00
--period 3600
--region us-east-1
| jq -r '.Datapoints[] | [.Timestamp, .Minimum, .Unit] | #csv'
Option 2
Export the data to JSON:
aws cloudwatch get-metric-statistics
--namespace AWS/EC2 --metric-name CPUUtilization
--dimensions Name=InstanceId,Value=i-xxxxxxxxxxxxxxxxx
--statistics Average
--start-time 2020-03-01T00:00:00
--end-time 2020-03-31T23:59:00
--period 3600
--region us-east-1 >> data.json
Use jq to read the json to csv:
jq -r '.Datapoints[] | [.Timestamp, .Minimum, .Unit] | #csv' data.json
Output
Here is how the output will look like:
"2020-03-24T11:00:00Z",0.327868852454245,"Percent"
"2020-03-11T21:00:00Z",0.327868852454245,"Percent"
"2020-03-15T04:00:00Z",0.322580645156596,"Percent"
"2020-03-27T18:00:00Z",0.327868852478101,"Percent"
There is no in-built capability to export Amazon CloudWatch metrics to CSV.
There are API calls available to extract metrics, but you would need to write a program to call the API, receive the metrics and store it in a suitable format.
There are projects available that can assist with this, such as:
mogproject/cloudwatch-dump: Just dump all the CloudWatch metrics.
AWS – CloudWatch – DataExtractor – API | vMan
romgapuz/awsmetric2csv: AWS Metric to CSV is a Python command-line utility to extract CloudWatch metric data from an AWS resources (e.g. EC2, RDS) and save to a CSV file.
If you are looking for these types of tools, make sure they refer to CloudWatch metrics, not CloudWatch Logs (which is something different).
You can also try out official Prometheus cloudwatch_exporter
I am trying to get the cloudwatch metric data for CPU Utilazation of an EC2 instance i-014448f54423cc0, but i am getting the following output without any metrics data
AWS CLI COMMAND
$ aws cloudwatch get-metric-statistics --metric-name CPUUtilization --start-time 2017-03-20T23:18:00 --end-time 2017-03-25T23:18:00 --period 3600 --namespace AWS/EC2 --statistics Average --dimensions Name=i-014448f54423cc0,Value=i-abcdef
OUTPUT
-----------------------------
| GetMetricStatistics |
+--------+------------------+
| Label | CPUUtilization |
+--------+------------------+
As per the following thread on AWS forum: Link
It is said that cloudwatch metrics data can be accessed through API's only if detailed monitoring is enabled, so I tried both scenarios of with & without enabling detailed monitoring but still in both the cases the output is same
You have defined your dimension incorrectly. The dimension you are filtering (literally 'InstanceId') needs to be defined as the Name, and your value (in this case, the instance id's value) as the Value.
Example
Snippet:
Name=InstanceId,Value=i-014448f54423cc0
Full example:
aws cloudwatch get-metric-statistics --metric-name CPUUtilization --start-time 2017-03-20T23:18:00 --end-time 2017-03-25T23:18:00 --period 3600 --namespace AWS/EC2 --statistics Average --dimensions Name=InstanceId,Value=i-014448f54423cc0
Discussion
The linked AWS forum discussion isn't relevant to this situation, because detailed monitoring only enables monitoring at 1-minute granularity. Basic monitoring samples data on five-minute intervals. Since you are requesting CPUUtilization for each hour (period=3600 seconds) on a single instance, you will have data available with no need for detailed monitoring.
Further Reading
AWS Documentation - Enable or Disable Detailed Monitoring for Your Instances
AWS Documentation - Amazon EC2 Metrics and Dimensions
AWS Documentation - AWS CLI get-metric-statistics
I am publishing a custom metric into CloudWatch with two dimensions:
aws cloudwatch --region ap-southeast-1 put-metric-data --namespace CustomNS --metric-name ApiReqCount --dimensions ApiName=TestAPI,ApplicationName=App1 --timestamp 2017-03-07T05:00:00.000Z --statistic-value Sum=7,Minimum=1,Maximum=7,SampleCount=1
If I query with all of the dimensions, it gives the datapoints:
aws cloudwatch --region ap-southeast-1 get-metric-statistics --namespace CustomNS --metric-name ApiReqCount --dimensions Name=ApiName,Value=TestAPI Name=ApplicationName,Value=App1 --statistics Sum --start-time 2017-03-05T00:00:00Z --end-time 2017-03-08T12:00:00Z --period 300
However when I query, and no dimensions or partial dimensions are specified, I do not get back datapoints:
aws cloudwatch --region ap-southeast-1 get-metric-statistics --namespace CustomNS --metric-name ApiReqCount --dimensions Name=ApiName,Value=TestAPI --statistics Sum --start-time 2017-03-05T00:00:00Z --end-time 2017-03-08T12:00:00Z --period 300
What I really want is that I need all datapoints returned when no dimensions are specified, I'd like to optionally filter these datapoints via Dimensions.
You can't query for partial dimension or mix dimensions even if they are identical.
From the documentation:
CloudWatch treats each unique combination of dimensions as a separate metric, even if the metrics have the same metric name. You can't retrieve statistics using combinations of dimensions that you did not specifically publish.
See examples on: Amazon CloudWatch Concepts - Dimensions