In cloudwatch -> metrics, there is only one option for X-Ray metric: ApproximateTraceCount. I assume this could be used to monitor the TOTAL number of X-Ray traces RECORDED. But how do I monitor the TOTAL number of x-ray traces retreived ? (so when I make a query to retrieve traces for example from last 6hours).
Because from billing perspective:
The first 100,000 traces recorded each month are free.
The first 1,000,000 traces retrieved or scanned each month are free.
So I guest i can track the first one with ApproximateTraceCount SUM.
What about the second one, am I able to track it somehow ?
Related
I would like to get all the AWS EC2 instances list which are idle and underutilized. I need to filter them based on CPU utilization used less than 2% and network I/O less than 5Mb for last 30 days.
Can you please provide me with the commands or any scripts to get the list or guide me to get that achieved.
I need this to get that list and terminate those instances for cost management.
You can create a CloudWatch alarm that is triggered when the average
CPU utilization percentage has been lower than 2 percent for 24 hours,
signaling that it is idle and no longer in use. You can adjust the
threshold, duration, and period to suit your needs, plus you can add
an SNS notification so that you will receive an email when the alarm
is triggered.
Kindly refer to this documentation to create an alarm to terminate an idle instance using the Amazon CloudWatch console.
I'd like to know if possible to discover which resource is behind this cost in my Cost Explorer, grouping by usage type I can see it is Data Processing bytes, but I don't know which resource would be consuming this amount of data.
Have some any idea how to discover it on CloudWatch?
This is almost certainly because something is writing more data to CloudWatch than previous months.
As stated this AWS Support page about unexpected CloudWatch logs bill increases:
Sudden increases in CloudWatch Logs bills are often caused by an
increase in ingested or storage data in a particular log group. Check
data usage using CloudWatch Logs Metrics and review your Amazon Web
Services (AWS) bill to identify the log group responsible for bill
increases.
Your screenshot identifies the large usage type as APS2-DataProcessing-Bytes. I believe that the APS2 part is telling you it's about the ap-southeast-2 region, so start by looking in that region when following the instructions below.
Here's a brief summary of the steps you need to take to find out which log groups are ingesting the most data:
How to check how much data you're ingesting
The IncomingBytes metric shows you how much data is being ingested in your CloudWatch log groups in near-real time. This metric can help you to determine:
Which log group is the highest contributor towards your bill
Whether there's been a spike in the incoming data to your log groups or a gradual increase due to new applications
How much data was pushed in a particular period
To query a small set of log groups:
Open the Amazon CloudWatch console.
In the navigation pane, choose Metrics.
For each of your log groups, select the IncomingBytes metric, and then choose the Graphed metrics tab.
For Statistic, choose Sum.
For Period, choose 30 Days.
Choose the Graph options tab and choose Number.
At the top right of the graph, choose custom, and then choose Absolute. Select a start and end date that corresponds with the last 30 days.
For more details, and for instructions on how to query hundreds of log groups, read the full AWS support article linked above.
Apart from the steps which Gabe mentioned what helped me identify the resource which was creating large number of logs was by:
heading over to Cloudwatch
selecting the region which showed in Cost explorer
Selecting Log Groups
From settings under Log Groups, Enabling column Stored bytes to be visible
This showed me which service was causing a lot of logs to be written to Cloudwatch.
Context:
I'm attempting to use Google Cloud's monitoring SDK to publish metrics on error status codes, latency and other server-side metrics.
Due to the rate of requests per second on my machines, this will exceed Google's metric limit of 1 datapoint for 10 seconds.
I am using the instance_id as one of the labels, so they will be unique per machine, but I will still exceed the 1 datapoint / 10 seconds.
Question:
As mentioned in a similar question, here, an option would be to log, buffer and forward the messages. This seems strange to have each customer implement this for common high-rate metric use-cases.
Is there an alternative way of recording high-rate metrics with the sdk such as latency, num requests, num errors?
Resources:
https://cloud.google.com/monitoring/quotas
https://cloud.google.com/monitoring/custom-metrics/creating-metrics#monitoring_create_metric-nodejs
One or more points were written more frequently than the maximum sampling period configured for the metric
Recently I have configured the ALARM in Cloudwatch for tracking VPN Tunnel connection. It is well known that 0 indicates tunnel is DOWN and 1 indicates tunnel is UP. When Connection is down, I have seen some data points on the graph shown as 0.66, 0.75.
So what does that mean, is the connection is DOWN or UP?
The correct statistic for each metric depends on your use case, and the underling metric.
From CloudWatch Concepts - Statistics
Statistics are metric data aggregations over specified periods of
time. CloudWatch provides statistics based on the metric data points
provided by your custom data or provided by other AWS services to
CloudWatch. Aggregations are made using the namespace, metric name,
dimensions, and the data point unit of measure, within the time period
you specify. The following table describes the available statistics.
Given the VPN metric above, try using the Maximum or Minimum statistics for the alarm. You are using the Average statistic, which, as you noted, will not produce meaningful data for your use case.
Minimum
The lowest value observed during the specified period. You can use this value to determine low volumes of activity for your application.
Maximum
The highest value observed during the specified period. You can use this value to determine high volumes of activity for your application.
That happens if your graph shows averages (that is why both of your values are between 1 and 0). In the ClouWatch console select your metric and then click on the Graphed metrics tab. There you will see Statistics column which is most likely set to Average now.
I want to use aws api to get logs for read/write operations with consumed read/write capacity for last two days or previous day.
How can I do that ?
Cloudwatch tracks the read and write cap units in the below metrics
ConsumedReadCapacityUnits
The number of read capacity units consumed over the specified time period, so you can track how much of your provisioned throughput is used. You can retrieve the total consumed read capacity for a table and all of its global secondary indexes, or for a particular global secondary index.
ConsumedWriteCapacityUnits
The number of write capacity units consumed over the specified time period, so you can track how much of your provisioned throughput is used. You can retrieve the total consumed write capacity for a table and all of its global secondary indexes, or for a particular global secondary index.
To view metrics (console)
Metrics are grouped first by the service namespace, and then by the various dimension combinations within each namespace.
Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.
In the navigation pane, choose Metrics.
Select the DynamoDB namespace.
To view metrics (CLI)
At a command prompt, use the following command:
aws cloudwatch list-metrics --namespace "AWS/DynamoDB"
You can also use the CLI to get the metrics for a given time period.
http://docs.aws.amazon.com/cli/latest/reference/cloudwatch/get-metric-statistics.html
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/metrics-dimensions.html