Had couple of questions on AWS:
Is there a way by which I can recreate/write AWS CloudWatch metrics to DynamoDB?
If an Amazon EC2 instance is deleted or if I change a VPC, I need to recreate all CloudWatch metrics manually every time. Is there a way by which I can automate CloudWatch metrics creation for every new VPC instance? Through Terraform, I can only create CloudWatch metric alarms, events and logs but not CloudWatch metrics (eg, EC2, RDS metrics etc).
#1 I could achieve it via AWS CLI and via Python script thereby writing it to dynamodb as well. #2 is still open.
Related
In AWS Production Account, we have a Classic VPN, and this category doesn't have cloudwatch metrics enabled. It is not possible to migrate right away to AWS VPN solutions. How to monitor with cloudwatch metrics the tunnels (Status of tunnels UP/Down) ?
How to put metric data ?
You could try creating a script to run the describe-vpn-connections command.
Then check the tunnel status, and push the result of this to a CloudWatch custom metric.
I'm trying to automate the turning on and off process of Redis Cluster in aws. I saw the following link for reference (https://forums.aws.amazon.com/thread.jspa?threadID=149772). Is there a way to do it via cloudwatch ?
I am very new to aws platform.
Check the documentation regarding scale in/out
https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/redis-cluster-resharding-online.html It also has commands to reshard a cluster manually.
Check CloudWatch metrics from the Redis cluster. https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/CacheMetrics.HostLevel.html and https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/CacheMetrics.Redis.html Choose the metrics that will trigger autoscaling
You can trigger an AWS Lambda on some event for a metric https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/RunLambdaSchedule.html
From the Lambda you cal call aws cli to reshard the cluster as described in 1. Example: https://alestic.com/2016/11/aws-lambda-awscli/
If you need to turn off the cluster completely, instead of the resharding commands just use https://docs.aws.amazon.com/cli/latest/reference/elasticache/delete-cache-cluster.html
As far as I understand there are special logs (like for EC2 Instances, RDS) which can be enabled to collect metrics and are not enabled automatically. Is there an OVERVIEW somewhere stating for which resources do special AWS Logs exist and how they can be enabled (via CloudFormation)?
If not I would suggest to create such an Overview since AWS DOCUMENTATION doesn’t have it.
Merci A
Amazon CloudWatch collects metrics from AWS services that you use. (Or, more accurately, the services you use send metrics to Amazon CloudWatch.)
The documentation for each service shows the metrics that are collected.
The only logs that go beyond this are special ones sent from within an Amazon EC2 instance. By default, the Amazon EC2 service can only examine your virtual machine ('instance') from outside of the instance. This provides basic metrics for CPU, disk and network. It is not possible to see inside your instance.
However, if you run additional software inside the instance that communicates with the operating system, then the software can send additional metrics to Amazon CloudWatch. This is typically done with the CloudWatch Agent.
See: Collect Metrics and Logs from Amazon EC2 Instances and On-Premises Servers with the CloudWatch Agent - Amazon CloudWatch
There are additional Metrics Collected by the CloudWatch Agent that can provide details such as free memory and free disk space because the agent communicates with the Operating System running inside the instance. These metrics are then sent to Amazon CloudWatch as custom metrics.
The CloudWatch Agent can also send application logs from an Amazon EC2 instance to Amazon CloudWatch Logs, which retains the individual log entries and can trigger alarms based on their content.
The CloudWatch Agent must be installed into the Amazon EC2 instance, either via a startup script ('User Data' script) or via AWS Systems Manager.
Similarly, Amazon RDS can send database logs to Amazon CloudWatch Logs. However, I couldn't see an option in Amazon CloudFormation to activate these log exports.
How can we monitor the system status check of all EC2 instances simultaneously rather than setting up cloudwatch alarm at each EC2 level individually?
If it's not possible via cloudwatch service, can it be done using boto3?
If you don't want to setup alarms individually, you can automate it, whenever you boot new instance, you can setup cloudwatch rule(when instance state changes from pending to online) to trigger a lambda function, in lambda function you can setup cloudwatch alarm for that instance, for already existing instances also you can setup the alarm with little modification of that script.
Say I set up aws cloudwatch logging on an ec2 instance to centralize logs from various files. If I have auto-scaling and a new machine gets started up due to high traffic, will the new copied machine start sending logs then too? Does logging work with auto-scale?
As long as the CloudWatch Logs agent is installed and configured on the AMI that is used for auto-scaling, the logs for the new instance(s) will be sent to CloudWatch. You can use the Instance ID when configuring the CloudWatch Logs agent to be able to identify which instance originated the event in the logs.
Also, make sure the instances have the necessary IAM role policy to publish the logs to CloudWatch.