I am decommissioning a service in Amazon Web Services DynamoDB. We have various metrics and alarms for our DyDB tables. I am able to delete the alarms using 'aws cloudwatch delete-alarms --alarm-names ...' but I do not see any AWS CLI command to delete metrics.
I know metrics keep data for 2 weeks. Is it possible that the metrics will automatically disappear in 2 weeks when no more data is being stored in it? I see no documentation stating that, unfortunately.
You're correct: there is no API to delete AWS Cloudwatch Metrics.
Just wait two weeks after your last metric has been pushed. It will disappear automatically.
Since November 1, 2016, CloudWatch has extended the retention of metrics (both custom and AWS NameSpace) from previous 14 days to 15 months.
But the CW console limits the search of metrics to 2 weeks after a metric is last ingested (which means if there is a metric in which no datapoints has been pushed for past 14 days then the CW console will not display it but you can still get it using the cli get-metric-statistics or by tweaking the console url to refer the correct resource name and start and end time older than 14 days).
And there is no api to delete the metrics.
https://aws.amazon.com/cloudwatch/faqs/
https://aws.amazon.com/cloudwatch/faqs/
Amazon CloudWatch FAQs
[…]
Q: What is the retention period of all metrics?
CloudWatch retains metric data as follows:
Data points with a period of less than 60 seconds are available for 3 hours. These data points are high-resolution custom metrics.
Data points with a period of 60 seconds (1 minute) are available for 15 days
Data points with a period of 300 seconds (5 minute) are available for 63 days
Data points with a period of 3600 seconds (1 hour) are available for 455 days (15 months)
Data points that are initially published with a shorter period are aggregated together for long-term storage.
[…]
Q: Can I delete any metrics?
CloudWatch does not support metric deletion. Metrics expire based on the retention schedules described above.
In support of the above, this is still an issue unfortunately. To read more on the AWS forum, please see:
https://forums.aws.amazon.com/message.jspa?messageID=281904
Related
I am using getMetricData and the following expression to get the CPU Utilization of all the instances I had:
SEARCH('{AWS/EC2,InstanceId} MetricName="CPUUtilization"', 'Average', 3600)
My instances were terminated around 15 days ago. This was working. But now, I'm not getting data anymore.
Are metrics supposed to being retained for 15 months? I don't see them in the AWS cloudwatch console either.
This is true if the data is still being added to CloudWatch. However the documentation states that if no data is pushed you will lose the data after 2 weeks.
CloudWatch console limits the search of metrics to 2 weeks after a metric is last ingested to ensure that the most up to date instances are shown in your namespace.
It looks like custom metrics will be kept for 15 months, if I understand it correctly, since they get aggregated to higher resolution, according to https://aws.amazon.com/cloudwatch/faqs. Does it mean we have to pay for at least 15 months once we create a custom metric?
I have CloudWatch agent installed to collect various metrics using user_data. It creates new metrics for every new instances. After running many tests, I have more than 6,000 custom metrics, but most of them are unused. Since there is no way to delete custom metrics, do I get charged for those unused metrics until they expire (15 months)? I hope I'm wrong on this :]
Please clarify how we get charged for unused custom metrics.
You will not get charged for those. You will get charged for the metric for the duration you publish data onto them. It's not very clear on the CloudWatch pricing page but they hint it in the [original pricing reduction blogpost]{https://aws.amazon.com/blogs/aws/aws-price-reduction-cloudwatch-custom-metrics/).
You will get charged though on the retrieval of those (API costs).
we have the same issue to resolve. waiting for clearification from aws
I am using Immutable deployment.
I have included script as mentioned at http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mon-scripts.html to monitor ec2 memory at autoscale level.
Now while every deployment for temporary autoscale cloudwatch metric is created.
How do I delete them when temporary autoscale is deleted?
Or how do I delete cloudwatch metrics created while deployment.
This is because my metric list will increase on every deployment.
It is not possible to delete metrics from Amazon CloudWatch. Metrics will eventually rotate out.
Yes, this will increase the list of metrics, but typically AWS users would ask for a specific filter of metrics, so it doesn't matter how many different metrics are actually being stored by CloudWatch.
From the CloudWatch FAQs:
Data points with a period of less than 60 seconds are available for 3 hours. These data points are high-resolution custom metrics.
Data points with a period of 60 seconds (1 minute) are available for 15 days
Data points with a period of 300 seconds (5 minute) are available for 63 days
Data points with a period of 3600 seconds (1 hour) are available for 455 days (15 months)
Data points that are initially published with a shorter period are aggregated together for long-term storage. For example, if you collect data using a period of 1 minute, the data remains available for 15 days with 1-minute resolution. After 15 days this data is still available, but is aggregated and is retrievable only with a resolution of 5 minutes. After 63 days, the data is further aggregated and is available with a resolution of 1 hour. If you need availability of metrics longer than these periods, you can use the GetMetricStatistics API to retrieve the datapoints for offline or different storage.
How many requests per minute I can send for each custom metric in CloudWatch?
p.s. I know for standard metrics is 1 per minute.
Here's a quote from section Publishing Single Data Points within the Amazon CloudWatch Developer Guide:
Although you can publish data points with time stamps as granular as one-thousandth of a second, Amazon CloudWatch aggregates the data to a minimum granularity of one minute
Their API reference says the date start date should be less than 14 days from the current date. I would like to know whether the data older than this is deleted and not available
Metrics used to be kept for 2 weeks, but as #sfgeorge points out, AWS has increased storage times.
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch.html
When you use the mon-put-data command,
you must use a date range within the
past two weeks. There is currently no
function to delete data points. Amazon
CloudWatch automatically deletes data
points with a timestamp more than two
weeks old.
As of November 1st, 2016, the retention window for AWS metrics in CloudWatch has expanded from 14 days to 15 months.
Note that the data granularity will be reduced when you widen your range beyond the past 15 days:
One minute data points are available for 15 days.
Five minute data points are available for 63 days.
One hour data points are available for 455 days (15 months).
As found in https://aws.amazon.com/ec2/faqs/ :
Q: Will I lose the metrics data if I disable monitoring for an Amazon EC2 instance?
You can retrieve metrics data for any Amazon EC2 instance up to 2 weeks from the time you started to monitor it. After 2 weeks, metrics data for an Amazon EC2 instance will not be available if monitoring was disabled for that Amazon EC2 instance. If you want to archive metrics beyond 2 weeks you can do so by calling mon-get-stats command from the command line and storing the results in Amazon S3 or Amazon SimpleDB