I am a bit confused in monitoring of EC2 with and without Cloudwatch agent. As far as I know, Cloudwatch agent does not get installed by default on EC2 linux but some basic system metrics like CPU usage can still be monitored and shown in Cloudwatch.
My questions
If I need to monitor memory usage which is not being monitored by default in EC2 now, should I just setup Cloudwatch agent and memory usage can be published to CloudWatch metrics?
What about I don't setup Cloudwatch agent but just enable detailed monitoring? Can memory usage be monitored by just enabling detailed monitoring without cloudwatch agent?
If I need to monitor memory usage which is not being monitored by
default in EC2 now, should I just setup Cloudwatch agent and memory
usage can be published to CloudWatch metrics?
Yes, this is the correct way to monitor OS level metrics on your EC2 instances.
What about I don't setup Cloudwatch agent but just enable detailed
monitoring? Can memory usage be monitored by just enabling detailed
monitoring without cloudwatch agent?
Detailed monitoring just changes the monitoring interval from 5 minutes to 1 minute, it doesn't enable additional metrics. CloudWatch can't reach into the EC2 operating system to see things like memory usage, so you have to install the CloudWatch agent on the server to monitor memory usage.
Related
I want to monitor free disk space on an EC2 instance using CloudWatch but can´t find any good tutorials. That instance is an Ubuntu Linux 18.
Any helps?
Start here: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html - this will install the cloudwatch agent.
The https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html - this shows the how to configure the agent. You can get it to send various metrics to CloudWatch including Disk usage.
One you have the metrics in CloudWatch you can setup alarms on the metric also in CloudWatch
I want to stop my EC2 instances if the memory utilization is more than x% from my Lambda function(python) , is their any possibility to check the memory utilization of an EC2 instance
For EC2 by default it will only have the host level metrics be accessible, this includes CPU, Disk Performance and Network Performance but does not include other metrics such as Memory Utilization.
For this you will need to push a custom metric from the EC2 instance into AWS, this can be performed by installing the CloudWatch Agent.
Once you have the memory metric being pushed into CloudWatch you can create an alarm that will trigger on a specific threshold being exceeded, allowing you to trigger an SNS topic. This can have a Lambda subscribe to the topic to be triggered under the condition.
You need to install the CloudWatch agent on the EC2 instances, if it's not there already. Then the memory usage will be a metric in CloudWatch that your Lambda function can query.
CloudWatch can be used to perform system monitoring(AWS resources) & application monitoring on AWS resources.
Does CloudWatch rely on SNMP mib(of AWS instance) to monitor the metrics?
No. CloudWatch collects all these metrics from the hypervisor on the physical host that is running your instance. It does not communicate with any processes running inside your instance, SNMP or otherwise.
No definitely not. CloudWatch standard EC2metrics are hypervisor-level and not reported by the instance. If you put the instance in a Security Group that blocks ALL outbound network traffic, the metrics still update. That also explains why CloudWatch metrics don’t include any OS-level stuff like memory utilization, free disk space, etc.
You can install a CloudWatch agent on the instance that pushes additional metrics (like memory utilization and free disk space), but that communicates over HTTPS like all the other AWS APIs. The CloudWatch API is well documented here: https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/Welcome.html
Furthermore, SNMP is typically a LAN protocol. So if CloudWatch was using SNMP it would have an IP address in your VPC. CloudWatch does not have a VPC endpoint by default.
No, CloudWatch doesn't support SNMP monitoring. But you can use some external tools, which will read SNMP metrics and it will put metrics to the CloudWatch.
For example, you can use customized collectd with the SNMP plugin.
No, CloudWatch can´t get SNMP directly, but you can use other solutions to "collect" traditional SNMP data into CloudWatch.
Follow reference:
https://aws.amazon.com/blogs/mt/snmp-monitoring-using-amazon-cloudwatch-and-elastic-logstash/
Just wondering if the AWS cloudwatch runs on the same VPC where i have all my applications are running?
Is there any chance that AWS cloudwatch might go down and we may loose the monitoring capability?
Do we need to have a monitoring mechanism to check the Cloudwatch health?
Thanks
AWS Cloudwatch isn't run on your instances. Its infrastructure is fully managed by Amazon and independent from your VPC. You can see it as a SaaS (Software as a Service).
So you don't have to worry about that. For more informations, please see: https://aws.amazon.com/cloudwatch/
Cloudwatch collects data from the host OS, where your VMs are actually running.
If the physical server had a significant issue both cloudwatch and your VM would go down but in that case the VM would get started automatically on another physical server. In such a case, recovery would be usually quite quickly.
You don't need to check Cloudwatch at all because AWS handles that but you could add alerts for things such as CPU usage on your VMs.
Because Cloudwatch doesn't run on your machines it can't know some things such as memory usage, disk space usage or others so if you need more advanced monitoring capabilities you might consider running something like collectd inside your virtual machine.
Just wondering if the AWS cloudwatch runs on the same VPC where i have all my applications are running?
If you chose to install CloudWatch Agent on your EC2 then only it runs in your EC2 and thus in the VPC your EC2 is provisioned.
CloudWatch service that publishes/maintain logs, metrics, alarms etc is managed by AWS and runs outside your VPC.
CloudWatch has a SLA of 99.9%
https://aws.amazon.com/cloudwatch/sla/
Is there any chance that AWS cloudwatch might go down and we may loose the monitoring capability?
CloudWatch like any other service can have outages and it did have some in the past but I have never seen any data getting lost, only temporarily not being available or slow to retrieve during the outage.
Do we need to have a monitoring mechanism to check the Cloudwatch health?
SLA is already 99.9% for CloudWatch Service so chances of catching a blip is very rare on your own monitoring mechanism.
If you are using CloudWatch Agent then consider checking health of agent to make sure it is in running state (you can use AWS System Manager Run command).
Suppose I have an EC2 instance, which I understood is a VM instance. So if I enable CloudWatch for this EC2 instance, should this monitoring capabilitiy offered by CloudWatch added into my EC2 instance, or it is just running in the hypervisor like XEN?
Thanks.
CloudWatch monitoring is always enabled by default for every EC2 instance at 5-min granularity. What you can enable is detailed monitoring which means you get 1-min observation granularity and aggregate metrics. Default monitoring at 5-min level is free, but detailed monitoring costs money.
Out-of-the-box CloudWatch metrics are measured at hypervisor level and you do not need to do anything to turn them on. See more info on what metrics are available here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch.html
Things like memory utilization and disk space can't be measured at hypervisor level so CloudWatch distributes a simple package with scripts that can be installed on the instance (Linux or Windows.) Those scripts report the data as custom metrics which also costs money. See http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/mon-scripts.html
It is monitored at the hypervisor layer. Aamazon generally will not look into the instance at VM layer so they can't monitor some feature such as memroy usage at the VM.