I'd like to use an OpenCensus metric in a Cloud Monitoring (Stackdriver) Alert Policy.
When I try to click the Add button, I get This query must contain a resource type. error. The odd thing, is that I can view this metric in MQL and can chart it.
According to the MQL charts that us this metric, the Resource: field is blank, and the charts work fine. The MQL charts show a resource type (on metric hover) of knative_broker, dataflow_job, aws_rds_database, k8s_control_plane_component, aws_lambda_function, and 36 more.
What Resource type should be used to alert on Open Census metrics in Cloud Monitoring alerts?
In Cloud Monitoring, application-specific metrics are typically called “custom metrics”. You can create custom metrics with opencensus. For more detailed information please follow Custom Metrics with OpenCensus. We define a Stackdriver exporter as the goal to create an alert policy on the aggregation of two metrics. Refer Alert policy metric thresholds with stackdriver and opencensus for more information.
Related
I have created some custom metric filters for a cloudtrail log group, 11 in total.
Each metric filter is filtering for multiple related events (one is for iam changes, another is for user logon activity, etc).
I want to log each time one of these metric filters captures an event and show it on an AWS Grafana dashboard.
I have the CDK to deploy the metric filters, they show up in cloudwatch and I can see them graphing events in the AWS Console.
I can even add the datasource and correct permissions to access it from AWS Grafana.
It's only when I go to render the results onto the dashboard panel that I start to see a problem. All of he metrics have the same data.
I have tried to add all metrics and they all show the same data. I have included some screenshots to demonstrate the issue.
In Google Cloud Logging (nee Stackdriver), can I create a log-based metric for logs aggregated at folder/organization level? I want to have a log-based metric for a certain audit event across many projects.
This isn't currently supported. You can export logs to a different project, but can't have metrics encapsulating more than one project.
If you think that functionality should be available, you can create a Feature Request at Public Issue Tracker.
I went through the docs but I couldn't find a way of sending an alert based on my Cloud function logs, is this possible?
What I want to do is trigger an alert when I get an specific log node.
Like, every time I have this:
Triggers an alert in my GCP, is that possible?
Create an log-based metric from the advanced filter of the Logs and create an alerting from the log-based metric. To create alert from the log-based metric you need to click on three dot right side of the metric name there you will see the create alert option.
As an alternative to log-based metrics, now log-based alerts are also supported. This lets you receive notification whenever a specific message appears in logs.
I'm just starting out using Apache Beam on Google Cloud Dataflow. I have a project set up with a billing account. The only things I plan on using this project for are:
1. dataflow - for all data processing
2. pubsub - for exporting stackdriver logs to be consumed by Datadog
Right now, as I write this, I am not currently running any dataflow jobs.
Looking at the past month, I see ~$15 in dataflow costs and ~$18 in Stackdriver Monitor API costs. It looks as though Stackdriver Monitor API is close to a fixed $1.46/day.
I'm curious how to mitigate this. I do not believe I want or need Stackdriver Monitoring. Is it mandatory? Further, while I feel I have nothing running, I see this over the past hour:
So I suppose the questions are these:
1. what are these calls?
2. is it possible to disable Stackdriver Monitoring for dataflow or otherwise mitigate the cost?
Per Yuri's suggestion, I found the culprit, and this is how (thanks to Google Support for walking me through this):
In GCP Cloud Console, navigate to 'APIs & Services' -> Library
Search for 'Strackdriver Monitoring Api' and click
Click 'Manage' on the next screen
Click 'Metrics' from the left-hand side menu
In the 'Select Graphs' dropdown, select "Traffic by Credential" and click 'OK'
This showed me a graph making it clear just about all of my requests were coming from a credential named datadog-metrics-collection, a service account I'd set up previously to collect GCP metrics and emit to Datadog.
Considering the answer posted and question, If we think we do not need Stackdriver monitoring, we can disable stackdriver monitoring API using bellow steps:
From the Cloud Console,go to APIs & Services.
Select Stackdriver Monitoring API.
Click Disable API.
In addition you can view Stackdriver usage by billing account and also can estimate cost using Stackdriver pricing calculator [a] [b].
View Stackdriver usage by billing account:
From anywhere in the Cloud Console, click Navigation menu and select Billing.
If you have more than one billing account, select Go to linked billing account to
view the current project's billing account. To locate a different billing account,
select Manage billing accounts and choose the account for which you'd like to get
usage reports.
Select Reports.
4.Select Group By > SKU. This menu might be hidden; you can access it by clicking Show
Filters.
From the SKUs drop-down list, make the following selections:
Log Volume (Stackdriver Logging usage)
Spans Ingested (Stackdriver Trace usage)
Metric Volume and Monitoring API Requests (Stackdriver Monitoring usage)
Your usage data, filtered by the SKUs you selected, will appear.
You can also select just one or some of these SKUs if you don't want to group your usage data.
Note: If your usage of any of these SKUs is 0, they don't appear in the Group By > SKU pull-down menu. For example, who use only the Cloud console might never generate API requests, so Monitoring API Requests doesn't appear in the list.
Use the Stackdriver pricing calculator [b]:
Add your current or projected Monitoring usage data to the Metrics section and click Add to estimate.
Add your current or projected Logging usage data to the Logs section and click Add to estimate.
Add your current Trace usage data to the Trace spans section and click Add to estimate.
Once you have input your usage data, click Estimate.
Estimates of your future Stackdriver bills appear. You can also Email Estimate or Save Estimate.
[a] https://cloud.google.com/stackdriver/estimating-bills#billing-acct-usage
[b] https://cloud.google.com/products/calculator/#tab=google-stackdriver
I can find a graph of "Group size" in the page of the instance group.
However, when I try to find this metric in Stackdriver, it doesn't exist.
I tried looking in the metricDescriptors API, but it doesn't seem to be there either.
Where can I find this metric?
I'm particularly interested in sending alerts when this metrics goes to 0.
There is not a Stackdriver Monitoring metric for this data yet. You can fetch the size using the instanceGroups.get API call. You could create a system that polls this data and posts it back to Stackdriver Monitoring as a custom metric and then you will be able to access it from Stackdriver.