Suppose I have an EC2 instance, which I understood is a VM instance. So if I enable CloudWatch for this EC2 instance, should this monitoring capabilitiy offered by CloudWatch added into my EC2 instance, or it is just running in the hypervisor like XEN?
Thanks.
CloudWatch monitoring is always enabled by default for every EC2 instance at 5-min granularity. What you can enable is detailed monitoring which means you get 1-min observation granularity and aggregate metrics. Default monitoring at 5-min level is free, but detailed monitoring costs money.
Out-of-the-box CloudWatch metrics are measured at hypervisor level and you do not need to do anything to turn them on. See more info on what metrics are available here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch.html
Things like memory utilization and disk space can't be measured at hypervisor level so CloudWatch distributes a simple package with scripts that can be installed on the instance (Linux or Windows.) Those scripts report the data as custom metrics which also costs money. See http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/mon-scripts.html
It is monitored at the hypervisor layer. Aamazon generally will not look into the instance at VM layer so they can't monitor some feature such as memroy usage at the VM.
Related
I have setup a system for RDS monitoring and used mysqld_exporter and postgres_exporter and exposed their metrics to prometheus server but I am not getting important metrics to monitor like CPU_Utilization, memory available, iops rate, latency etc. I am getting about 1800 metrics but none of them are these I suppose. I think i need node exporter for RDS monitoring for getting cpu utilization but i don't know how to configure node_exporter for RDS instance.
PS: I don't want to use AWS Cloudwatch metrics
How to I can know, for example, RAM size on the AWS instance?
I select View details on the needed instance, but I can't find any needed information about the productivity (CPU speed, Storage size, RAM):
CPU usage for your instance is reported in CloudWatch by default. The RAM and disk volume usage is only known by the operating system running on the instance, so you have to either login to the instance to check that, or you can install the AWS CloudWatch Agent on the instance to have those values reported to CloudWatch.
I am a bit confused in monitoring of EC2 with and without Cloudwatch agent. As far as I know, Cloudwatch agent does not get installed by default on EC2 linux but some basic system metrics like CPU usage can still be monitored and shown in Cloudwatch.
My questions
If I need to monitor memory usage which is not being monitored by default in EC2 now, should I just setup Cloudwatch agent and memory usage can be published to CloudWatch metrics?
What about I don't setup Cloudwatch agent but just enable detailed monitoring? Can memory usage be monitored by just enabling detailed monitoring without cloudwatch agent?
If I need to monitor memory usage which is not being monitored by
default in EC2 now, should I just setup Cloudwatch agent and memory
usage can be published to CloudWatch metrics?
Yes, this is the correct way to monitor OS level metrics on your EC2 instances.
What about I don't setup Cloudwatch agent but just enable detailed
monitoring? Can memory usage be monitored by just enabling detailed
monitoring without cloudwatch agent?
Detailed monitoring just changes the monitoring interval from 5 minutes to 1 minute, it doesn't enable additional metrics. CloudWatch can't reach into the EC2 operating system to see things like memory usage, so you have to install the CloudWatch agent on the server to monitor memory usage.
Just wondering if the AWS cloudwatch runs on the same VPC where i have all my applications are running?
Is there any chance that AWS cloudwatch might go down and we may loose the monitoring capability?
Do we need to have a monitoring mechanism to check the Cloudwatch health?
Thanks
AWS Cloudwatch isn't run on your instances. Its infrastructure is fully managed by Amazon and independent from your VPC. You can see it as a SaaS (Software as a Service).
So you don't have to worry about that. For more informations, please see: https://aws.amazon.com/cloudwatch/
Cloudwatch collects data from the host OS, where your VMs are actually running.
If the physical server had a significant issue both cloudwatch and your VM would go down but in that case the VM would get started automatically on another physical server. In such a case, recovery would be usually quite quickly.
You don't need to check Cloudwatch at all because AWS handles that but you could add alerts for things such as CPU usage on your VMs.
Because Cloudwatch doesn't run on your machines it can't know some things such as memory usage, disk space usage or others so if you need more advanced monitoring capabilities you might consider running something like collectd inside your virtual machine.
Just wondering if the AWS cloudwatch runs on the same VPC where i have all my applications are running?
If you chose to install CloudWatch Agent on your EC2 then only it runs in your EC2 and thus in the VPC your EC2 is provisioned.
CloudWatch service that publishes/maintain logs, metrics, alarms etc is managed by AWS and runs outside your VPC.
CloudWatch has a SLA of 99.9%
https://aws.amazon.com/cloudwatch/sla/
Is there any chance that AWS cloudwatch might go down and we may loose the monitoring capability?
CloudWatch like any other service can have outages and it did have some in the past but I have never seen any data getting lost, only temporarily not being available or slow to retrieve during the outage.
Do we need to have a monitoring mechanism to check the Cloudwatch health?
SLA is already 99.9% for CloudWatch Service so chances of catching a blip is very rare on your own monitoring mechanism.
If you are using CloudWatch Agent then consider checking health of agent to make sure it is in running state (you can use AWS System Manager Run command).
I am in need of a fairly short/simple script to monitor my EC2 instances for Memory and CPU (for now).
After using Get-EC2Instance -Region , it lists all of the instances. from here where can i go?
Cloudwatch is the monitoring tool for AWS instances. While it can support custom metrics, by default it only measures what the hypervisor can see for your instance.
CPU utilization is supported by default, this is often a more accurate way to see your true CPU utilization since the value comes from the hypervisor.
Memory utilization however is not. This depends largely on your OS and is not visible to the hypervisor. However, you can set up a script that will report this metric to Cloudwatch. Some scripts to help you do this are here: http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/mon-scripts-perl.html
There are a few possibilities for monitoring EC2 instances.
Nagios - http://www.nagios.com/solutions/aws-monitoring
StackDriver - http://www.stackdriver.com/
CopperEgg - http://copperegg.com/aws/
But my favorite is Datadog - http://www.datadoghq.com/ - (not just because I work here, but its important to disclose I do work for Datadog.) 5 hosts or less is free and I bet you can be up and running in less than 5 minutes.
Depends what your requirements are for service availability of the monitoring solution itself, as well as how you want to be alerted about host/service notifications.
Nagios, Icinga etc... will allow you to customise an extremely large number of parameters that can be passed to your EC2 hosts, specifying exactly what you want to monitor or check up on. You can run any of the default (or custom) scripts which then feed data back to a central system, then handle those notifications however you want (i.e. send an email, SMS, execute an arbitrary script). Downside of this approach is that you need to self-manage your backend for all of the aggregated monitoring data.
The CloudWatch approach means your instances can push metric data into AWS, then define custom policies around thresholds. For example, 90% CPU usage for more than 5 minutes on an instance or ASG, which might then push a message out to your email via SNS (Simple Notification Service). This method reduces the amount of backend components to manage/maintain, but lacks the extreme customisation abilities of self-hosted monitoring platforms.