I am trying to fetch the cluster level metrics of vmware, and I'm doing so by iterating over the hosts and summing the overall usage metrics. However, there seems to be a mismatch between the obtained result and the output as visible on vsphere client.
Related
I'm trying to set up Googlecloud monitor Dashboard for my GCE's. I'm expiriancing some difficulties though when tring to filter.
I have serveral GCE, and some are not running and are as backup, but are still displayed in the Cloud Monitor.
I would like to monitor 3 metrics (for now) : CPU, Mem, Disk usage.
CPU wasnt a problem as i could just filter by GCE instance name:
But now if i try to do the same for Memory and Disk usage, I dont have the option to filter as I did using CPU. I tried serveral different approaches like filter by "metadata labels:name", "label", "zone" etc. - all result in a "no data avalible for selected timeframe" (without the filter data is displayed). I feel like I'm missing something trivial:
What am I doing wrong? How can I filter by Instancename? Do i need to Activate some logger on Google cloud? Thank you verymuch in advance!
Use the Cloud Monitoring agent to gather system and application metrics (disk, CPU, network and process) from VM instances and send them to Monitoring.
Install the Monitoring Agent
Use the Cloud Logging agent to gather logging metrics from VM instances and send them to Cloud Monitoring.
Install the Logging Agent
I used the provided YCSB driver to benchmark the Spanner in Google Cloud Service.
For Spanner, I opted for Regional Configuration (instead of multi-region configuration) at Singapore (regional-asia-southeast1). So there are 3 replicas, one in each of three zones.
The benchmark workload is simple: first load 1M 1KB-size records (a total of 1GB storage) and then do the single-record update.
However, when setting the node number = 3, YCSB driver measures 17ms for update latency. When the node number is 12, it is reported for 49ms.
In addition, we follow this guide to benchmark on transactions updating two records. And we get a similar pattern (29ms with 3 nodes and 82ms with 12 nodes).
Since Google Spanner is a managed service, is anyone aware of this cause?
I have a small number of HTTP servers on GCP VMs. I have a mixture of different server languages and Linux based OS's.
Questions
A. It it possible to use the Stackdriver monitoring service to set alerts at specific percentiles for HTTP response latencies?
B. Can I do this without editing the code of each server process?
C. Will installing the agent into the VM report HTTP latencies?
For example, if the 95th percentile goes over 100ms for a certain time period I want to know.
I know I can do this for CPU utilisation and other hypervisor provided stats using:
https://console.cloud.google.com/monitoring/alerting
Thanks.
Request latencies are extracted by cloud load balancers. As long as you are using cloud load balancer you don't need to install monitoring agent to create alerts based 95th Percentile Metrics.
Monitoring agent captures latencies for some preconfigured systems such as riak, cassandra and some others. Here's a full list of systems and metrics monitoring agent supports by default: https://cloud.google.com/monitoring/api/metrics_agent
But if you want anything custom, i.e. you want to measure request latencies from VM you would need to capture response times yourself and configure logging agent to create a custom metric which you can use to create alerts. And as long as you are capturing them as distribution metrics you should be able to visualise different percentiles (i.e. 25, 50, 75, 80, 90, 95 and 99th etc.) and create alert based on that.
see: https://cloud.google.com/logging/docs/logs-based-metrics/distribution-metrics
A. It it possible to use the Stackdriver monitoring service to set
alerts at specific percentiles for HTTP response latencies?
If you want to simply consider network traffic, yes it is possible. Also if you are using a load balancer it's also possible to set alerts on that.
What you want to do should be pretty straight forward from the interface, however you can also find more info in the documentation.
If you want to use some advanced metric on top of tomcat/apache2 etc, you should check the list of metrics provided by the stackdriver monitoring agent here.
B. Can I do this without editing the code of each server process?
Yes, no need to update any program, stackdriver monitoring works transparently and will be able to fetch basic metrics from a GCP VMs without the need of the monitoring agent, including network traffic and cpu utilization.
C. Will installing the agent into the VM report HTTP latencies?
No, the agent shouldn't cause any http latencies.
Is there a way to find the IP addresses of all connections to AWS RDS instances.
You can get connected Host IP from information_schema using MySQL Query.
select host from information_schema.processlist WHERE ID=connection_id();
Because in basic monitoring only these metrics are available.
For Monitoring, choose the option for how you want to view your metrics from these:
CloudWatch:
Shows a summary of DB instance metrics available from Amazon
CloudWatch. Each metric includes a graph showing the metric monitored
over a specific time span.
Enhanced monitoring:
Shows a summary of OS metrics available for a DB instance with
Enhanced Monitoring enabled. Each metric includes a graph showing the
metric monitored over a specific time span.
OS Process list:
Shows details for each process running in the selected instance.
So the option that can help you in this regards is Performance Insights.
Performance Insights:
If you are interested in detail matrics like HOST IPs, the number of connections, Slow queries and many more which can eliminate the need of DBA I believe as very good experience with Using Amazon RDS Performance Insights
Top activity can list any of the dimensions indicated at the top of the list. For Aurora PostgreSQL, Performance Insights currently supports listing top SQL, waits, hosts, and users.
analyzing-amazon-rds-database-workload-with-performance-insights
If you are using MariaDB:
select id, user, host, db, command, time, state, info, progress from information_schema.processlist;
Other options specific to AWS.
Turn on VPC Flowlogs , then use loginishgts to query historical
traffic to the instances.
Inspect the security group of the RDS, it may list the client(s) which is allowed to connect to it.
Turn on the general query log https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.MariaDB.html
I am running a VM instance on GCP (actually a ready Deep Learning Package: 8 CPUs, 1 Tesla V100 GPU, ..., access via a Jupyter Notebook).
Is there a way to monitor the overall usage and costs in real-time?
I am thinking about a "Live usage" link inside https://console.cloud.google.com/, which shows which products are currently used, and their price per second/hour.
I think it is not possible to monitor the services usage per second/hour. In case you want to analyze your projects bills, GCP offers several options that you can use for this matter, such as Billng Cicles, Billing Reports, Export Billing Data to a File or BigQuery and Visualize your spend with Data Studio; however, it is important to keep in mind that these alternatives may require certain amount of time to reflect each service usage.
Additionally, you can use the Cloud Billing Catalog API to get the list of all the public services and SKUs metadata in a programmatic, real-time way that can be used as a complement of the cost management tools mentioned above to reconcile list pricing rates.