What could be the backend storage for Amazon CloudWatch Log Groups.
Is it an Amazon S3 backend?
Or is it a physical managed service that AWS manages?
AWS has not exposed what backend the CloudWatch uses.
But if you are interested in doing it yourself.
You can create a StatsD backend for your applications.
https://github.com/camitz/aws-cloudwatch-statsd-backend
Related
I'm working on Centralized Logging of my environnement AWS presented in this guide:https://docs.aws.amazon.com/solutions/latest/centralized-logging/overview.html
I'm using the same architecture as presented in AWS Doc.
I'm searching the possibility to send metrics from CloudWatch of each account to OpenSearch in centralized account. I was thinking of a Lambda function that retrieves each X minutes the data with get_metric_data ? I'm collecting the metrics with CloudWatch Agent
Is there an another method to do it ?
Thank you
Is there anyway to Query AWS Service Status e.g. if Kinesis Stream is Up or not, or S3 Storage Service is available or not ? And on the basis of status, I want to execute certain CURL commands ?
I am looking to get a status for below services and then use it in our custom StatusPage
S3
API Gwy
Kinesis
SQS
Route53 + Cloudfront
I know there is statuspage for AWS Services but not sure how to use it programmatically and publish a CURL command.
The Service Health Dashboard is usually delayed and not very useful for real-time monitoring of AWS health status.
Instead, you could use AWS Health from your account, which provides much more up-to-date events. What's more you can setup automated notifications and response to any issues detected as described in:
Monitoring AWS Health events with Amazon CloudWatch Events
Suppose multiple aws services are running (like EC2,S3) and If someone changes the configuration of these services , I want to know immediately and want to pull the updated configuration. So how can i listen to these services to pull configuration immediately . I went through couple of options like aws SNS and all. But i don't want to implement this using any aws services for this.
I am using spring boot application. Any help is appreciated , Thanks in advance.
The two services you should be interested in are:
AWS CloudTrail:
AWS CloudTrail is an AWS service that helps you enable governance, compliance, and operational and risk auditing of your AWS account. Actions taken by a user, role, or an AWS service are recorded as events in CloudTrail. Events include actions taken in the AWS Management Console, AWS Command Line Interface, and AWS SDKs and APIs.
AWS Config:
AWS Config provides a detailed view of the configuration of AWS resources in your AWS account. This includes how the resources are related to one another and how they were configured in the past so that you can see how the configurations and relationships change over time.
You can also configure Amazon CloudWatch Events rules that trigger when a particular API call is made (eg when a user changes some configuration). This could trigger a notification, a Lambda function, etc.
See:
Creating a CloudWatch Events Rule That Triggers on an AWS API Call Using AWS CloudTrail - Amazon CloudWatch Events
How to monitor AWS account activity with Cloudtrail, Cloudwatch Events and Serverless
Is there a way where we can check how many S3 List requests are happening on a bucket
The general solution on AWS to finding out what is going on at the API level is to use CloudTrail.
See Logging Amazon S3 API Calls by Using AWS CloudTrail.
This can be done by enabling the request metrics on S3 bucket by using CloudWatch service. The metrics are available at 1-minute intervals after some latency to process. These CloudWatch metrics are billed at the same rate as the Amazon CloudWatch Custom Metrics.
You can refer this Document to Configure Request Metrics for an S3 Bucket.
I am having a first look at Amazon Kinesis to compare it to Apache Kafka.
Can Kinesis Producers/Consumers access the Kinesis broker from outside AWS? Or have the (Java) clients to be deployed on a EC2 instance?
Yes, you can create and interact with your streams using the API and the AWS SDKs.