How to send OS logs and container logs from AWS EKS to Cloudwatch? - amazon-web-services

My understanding is that the Cloudwatch agent is available both as a Linux binary and as a Kubernetes deamonset.
I am aware that the EKS container logs could be forwarded to Cloudwatch using the cloudwatch agent that runs as EKS daemonset.
I have a query on how to send OS logs from EKS nodes to cloudwatch? Would the cloudwatch agent daemonset service be able to send the OS logs to Cloudwatch? or is the Linux binary required to be run on the EKS nodes to send OS logs?

I believe that either can be used independently.
You can set up Fluentd or Fluent Bit as a DaemonSet to send logs to CloudWatch Logs.
Alternatively, you can install CloudWatch agent on Amazon EKS nodes using Distributor and State Manager. You may also consider including it in the launch template to automate installation because the EKS-optimized AMI does not include the agent by default.

Related

AWS EKS Fargate logging on cloudwatch

I have create an eks cluster on fargate. I want to send container logs to cloudwatch. for that i have use this guideline(eks-fargate-logging. I want to create cloudwatch stream as namespace-pod-name. how can we do this ??
Need to use older cloudwatch plugins. dont use cloudwatch_logs plugin.cloudwatch plugin

How to dockerize the cloudwatch agent and collectd to collect the metrics?

I need to collect the custom metrics from my ECS instances and from the documentation
These are steps I need to follow
Install aws cloudwatch agent
Install collectd daemon
Configure cloudwatch agent to get the metrics from collectd daemon
It seems I can
Dockerize the cloudwatch agent (which seems to be already done, but with lack of documentation)
Dockerize collectd daemon
Why to dockerize or not use awslogs driver for collecting metrics ?
Currently we already have some services running as docker instances managed by amazon ecs and configured to use awslog driver to send logs to amazon cloudwatch logs.
But in order to collect more custom metrics from the services e.g number of requests per particular user from service a the only solution which aws suggested is to use colletd with curl plugin along with cloud watch agent.
As due to some scaling issues, instead of running cloudwatch agent and collectd in a instance. I want to run those as containers.
Question:
Is there any way to run cloudwatch agent in docker container which can read the metrics from collectd daemon which runs in different container but on the same machine ?
you do not need to run cloudwatch agent in your container, do not forget the rule of thumb, one process per container.
All you need to push application logs to stdout or stderr of the container and docker daemon will take care of it.
Important Configuration:
All you need to set log driver to AWS log driver in the task definition.
Amazon CloudWatch Logs logging driver
The awslogs logging driver sends container logs to Amazon CloudWatch
Logs. Log entries can be retrieved through the AWS Management Console
or the AWS SDKs and Command Line Tools.
Specifying a Log Configuration in your Task Definition
Before your containers can send logs to CloudWatch, you must
specify the awslogs log driver for containers in your task definition.
This section describes the log configuration for a container to use
the awslogs log driver. For more information, see Creating a Task
Definition.
"logConfiguration": {
"logDriver": "awslogs",
"options": {
"awslogs-group": "awslogs-mysql",
"awslogs-region": "us-west-2",
"awslogs-stream-prefix": "awslogs-example"
}
}
AmazonECS-using_awslogs

AWS ECS custom CloudWatch metrics

I'm looking for a way to establish custom metrics over StatsD protocol for Amazon Elastic Container Service. I've found a documentation on how to establish Amazon CloudWatch Agent on EC2. It works well. However I'm failing to find a correct configuration for Dockerfile. Quite probably some set of custom IAM permissions will also be required there.
Is it possible to have Docker containers working from AWS ECS with custom metrics using StatsD reporting to AWS CloudWatch?
Rather than building your own container, you can use the one provided by Amazon. This article explains how, including a link to an example daemon service task configuration.

Overview about AWS logs that can be used in CloudWatch

As far as I understand there are special logs (like for EC2 Instances, RDS) which can be enabled to collect metrics and are not enabled automatically. Is there an OVERVIEW somewhere stating for which resources do special AWS Logs exist and how they can be enabled (via CloudFormation)?
If not I would suggest to create such an Overview since AWS DOCUMENTATION doesn’t have it.
Merci A
Amazon CloudWatch collects metrics from AWS services that you use. (Or, more accurately, the services you use send metrics to Amazon CloudWatch.)
The documentation for each service shows the metrics that are collected.
The only logs that go beyond this are special ones sent from within an Amazon EC2 instance. By default, the Amazon EC2 service can only examine your virtual machine ('instance') from outside of the instance. This provides basic metrics for CPU, disk and network. It is not possible to see inside your instance.
However, if you run additional software inside the instance that communicates with the operating system, then the software can send additional metrics to Amazon CloudWatch. This is typically done with the CloudWatch Agent.
See: Collect Metrics and Logs from Amazon EC2 Instances and On-Premises Servers with the CloudWatch Agent - Amazon CloudWatch
There are additional Metrics Collected by the CloudWatch Agent that can provide details such as free memory and free disk space because the agent communicates with the Operating System running inside the instance. These metrics are then sent to Amazon CloudWatch as custom metrics.
The CloudWatch Agent can also send application logs from an Amazon EC2 instance to Amazon CloudWatch Logs, which retains the individual log entries and can trigger alarms based on their content.
The CloudWatch Agent must be installed into the Amazon EC2 instance, either via a startup script ('User Data' script) or via AWS Systems Manager.
Similarly, Amazon RDS can send database logs to Amazon CloudWatch Logs. However, I couldn't see an option in Amazon CloudFormation to activate these log exports.

How cloudwatch logging works with auto scaling?

Say I set up aws cloudwatch logging on an ec2 instance to centralize logs from various files. If I have auto-scaling and a new machine gets started up due to high traffic, will the new copied machine start sending logs then too? Does logging work with auto-scale?
As long as the CloudWatch Logs agent is installed and configured on the AMI that is used for auto-scaling, the logs for the new instance(s) will be sent to CloudWatch. You can use the Instance ID when configuring the CloudWatch Logs agent to be able to identify which instance originated the event in the logs.
Also, make sure the instances have the necessary IAM role policy to publish the logs to CloudWatch.