I have an Aurora Postgres instance that I am analyzing some performance metrics. Something caught my eye and I cannot make sense of the data.
If I go to the Monitoring tab of that Aurora instance, I see that the CPU Utilization (powered by Cloudwatch) is 20-30%. If I change the Monitoring to use "Enhanced Monitoring", the "CPU Total" shows the CPU always on 99+%.
Why do those numbers don't match?
PS: I do have another instance that those 2 metrics have a very similar value
Related
I have enabled Default GCP Monitoring in my Google Kubernetes Cluster. So GKE Dashboard is created which contains System Metrics. Now I need to enable alert for Kubernetes container's CPU and memory Utilization from GKE dashboard. I tried to create own alert, but it didn't match with metrics defined in GKE dashboard.
This is a Guide1 and Guide2 for monitoring the Kubernetes engine. In it, you can know about alerting and how to monitor your system. In case you were already familiar, here is a list of the metrics for the new Kubernetes engine in comparison to the previous metrics. Additionally, the complete list of metrics, which are always useful, can be found here.
In Monitoring dashboard, dashboard displays CPU and Memory utilization in time range:
CPU utilization: The CPU utilization of containers that can be attributed to a resource within the selected time span. The metric used is here check with For CPU Utilization
Memory utilization: The memory utilization of containers that can be attributed to a resource within the selected time span. The metric used is here check with For Memory Utilization
The command "kubectl top node" displays resource (CPU/Memory/Storage) usage at that moment, not time span.
I deployed the Machine Learning classification model in AWS EC2 (UBUNTU)instance successfully. I am able to access the instance "http://ec2-18-191-31-0.us-east-2.compute.amazonaws.com" and predictions are working fine only for few minutes. After that I or my colleagues are not able to access this. Getting an error "cannot connected to the server".
Security group that I crated as attached.
t2.micro instances are not suitable for any long running calculations. They are burstable. This means that their performance can be sustained only for short periods of time, e.g., sudden, short lived spikes in CPU usage. On top of that they have only 1 GB of RAM which limits its usefulness in machine learning.
For calculations, you could consider Compute optimized or Memory optimized instances. Obviously, these instance types are not free, but they are suited for calculations.
You can change instance type if you want and test with other, more power types. What you are describing indicates that your t2.micro exhausts all its RAM and/or CPU burst credits after few minutes and it freezes.
You can use CloudWatch Metrics for EC2 to monitor your instances and observer its CPU utilization and other metrics which can help you determine what exactly is causing the backlog. You can also monitor RAM and disc usage but this requires CloudWatch Agent setup on the instance.
Compute optimizer shows right sizing recommendations for ec2 instances that have been active for a large period of time over the last 2 weeks. Through this it produces graphs that show you your cpu, memory and network usage from 0 to 100% of the instance capacity.
How do i get access to this information without the use of compute optimizer. I want to know so i can right size instances after 3 days instead of waiting for 2 weeks for compute optimizer to get enough data to produce a recommendation. I have tried with cost explorer but can't see any way to filter ec2 instances by cpu or memory utilisation.
EC2 instances by default check a few baseline stats in 5-minute intervals which you could increase to "Enhanced Monitoring" that run checks every minute.
CPU is monitored by default and you can see it in the console if you go to EC2, your instance, and click the monitoring tab that shows up. To track memory you'll need to install the CloudWatch agent on the EC2 instance (second link).
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/viewing_metrics_with_cloudwatch.html
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/monitoring_ec2.html
We want to use AWS predictive scaling to forecast the load and CPU so this will certainly help us move away from manually launching instances based on load. We created new scaling plan by choosing EC2 Autoscaling group and enabling predictive scaling(forecast only for now). But we noticed that the CPU graph on Grafana is different from AWS Average CPU utilization. Grafana is getting alerts from elasticsearch which gets logs directly from services running in ec2. I am not sure why they don't show the same percentage of CPU Utilization and am wondering why AWS CPU Utilization is lower than the CPU shows on Grafana? If so can autoscaling scales the instances correctly?
AWS Autoscaling group Average CPU utilization
Grafana Averge CPU graph
AWS has its own method of computing CPU Util which is based on "EC2 compute units" so it is possible that value will differ when compared to another way of calculating the same metrics.
I have a script that I run 24/7 that uses 90-100% CPU constantly. I am running this script in multiple virtual machines from Google Cloud Platform. I run one script per VM.
I am trying to reduce cost by using AWS EC2. I looked at the price per hour of t3-micro (2 vCPU) instances and it says the cost is around $0.01/h, which is cheaper than the GCP's equivalent instance with 2 vCPU.
Now, I tried to run the script in one t3-micro instance, just to have a real estimate of how much each t3-instance running my script will cost. I was expecting the monthly cost per instance to be ~$7.20 (720h/month * $0.01/h). The thing is that I have been running the script for 2-3 days, and the cost reports already show a cost of more than $4.
I am trying to understand why the cost is so far from my estimate (and from AWS monthly calculator's estimate). All these extra cost seem to be from "EC2 Other" and "CPU Credit", but I don't understand these costs.
I suspect these come from my 24-7 full CPU usage, but could someone explain what are these costs and if there is a way to reduce them?
The EC2 instance allows a certain baseline CPU usage: 10% for a t3.micro. When the instance is operating below that threshold it accumulates vCPU credits: which are applied to usage above the threshold. A t3.micro can accumulate up to 12 credits an hour (with one credit being equal to 100% CPU ulitilisation for 1 minute). If you are regularly using more CPU credits than the instance allows will be charged at a higher rate: which I understand to be 5c per vCPU hour.
It may be that t3.micro is not your best choice for that type of workload and you may need to select a different instance type or a bigger instance.
The purple in your chart is CPU credits, not instance usage.
Looks like you enabled “T2/T3 Unlimited” when launching your instance and your script is causing it to bursting beyond the provided capacity. When you burst beyond the baseline capacity, you’re charged for that usage at the prevailing rate. You can read more about T2/T3 Unlimited and burstable performance here.
To bring these costs down, disable T2/T3 unlimited by following instructions here.