I have an ECS container that is deployed by the Docker image and I have used aws_powertools for logging and creating the metrics in Cloudwatch. However, I can only see my loggings and printed out metrics. But in Cloudwatch my custom metrics do not show up automatically as they do normally in Lambda usage.
So, how can I solve this situation, or are there any possibilities to get metrics from ECS container and put them into CW?
this how I create metrics:
metrics = Metrics(service="servicename", namespace="namespace")
metrics.add_metric(
name="name",
unit=MetricUnit.Seconds,
value=execution_duration,
)
serialized_metric = metrics.serialize_metric_set()
metrics.clear_metrics()
print(json.dumps(serialized_metric, separators=(",", ":")))
Thank you in advance!
Related
I am running a kube cluster in AWS/EKS. All the alarms are managed in AWS CloudWatch. While that could change in the futur, this a requirement I have to deal with today.
I also have alerts in Prometheus. I wish to "export" them to CloudWatch. What would be the best solution for this? I see only two possibilities so far:
I create a lambda in AWS, which query the ALARM{} metrics to Prometheus, then export the result in CW. I then create an additional alarm in CW monitoring the state of the Prometheus alarme.
I create a webhook in alert manager calling an API gateway in AWS, which would turn on/off the alarm in CW.
Any other suggestions ?
I've got a training job running on Sagemaker. I would like to retrieve instance metrics like MemoryUtilization etc by CLI or boto3 client.
Obviously I can see them in the console. However, I cannot see them in the CLI/API. For example, when running:
aws cloudwatch list-metrics --namespace "AWS/SageMaker"
I can see only metrics regarding endpoint invocation but not any training job related metrics.
Any idea?
Thanks!
I have a view on a PostgreSQL RDS instance that lists any ongoing deadlocks. Ideally, there are no deadlocks in the database, causing the view to show nothing, but on rare occasions, there are.
How would I setup an alarm in Cloudwatch to query this view and raise an alarm if any records return?
I found the cool script on Github specifically for this:
A Serverless MySQL RDS Data Collection script to push Custom Metrics to CloudWatch on AWS
Basically, there are 2 main possibilities to publish any custom metrics on CloudWatch:
Via API
You can run it on a schedule on EC2 instance (AWS example) or as a lambda function (great manual with code examples)
With CloudWatch agent
Here is the pretty example for Monitor your Microsoft SQL Server using custom metrics with Amazon CloudWatch and AWS Systems Manager.
After all, you should set up CloudWatch alarms with Metric Math and relevant thresholds.
It is not possible to configure Amazon CloudWatch to look inside an Amazon RDS database.
You will need some code running somewhere that regularly runs a query on the database and sends a custom metric to Amazon CloudWatch.
For example, you could trigger an AWS Lambda function, or use cron on an Amazon EC2 instance to trigger a script.
We have a service running in aws ecs that we want to scale in and out based on 2 metrics.
Scale out when: cpu > 80% or connection_count > 9500
Scale in when: cpu < 50% and connection_count < 5000
We have access to both the cpu and connection count metrics and alarms in cloud watch. However, we can't figure out how to setup a dynamic scaling policy like this based on both of them.
Using the standard aws console interface for creating the auto scaling rules I don't see any options for multiple. Any links to a tutorial or aws docs on this would be appreciated.
Based on the responses posted in the support aws forums, nothing can be done for AND/OR/IF conditions. (https://forums.aws.amazon.com/thread.jspa?threadID=94984)
It does mention however that they already put a feature request to the cloudwatch team.
The following is mentioned as a workaround:
"In the meantime, a possible workaround can be to create a custom metric using a custom script which would run after every five minutes and get the data points from the CloudWatch metrics, then perform the AND or OR operation and then push the output to a custom metric. You can then create a CloudWatch alarm which would monitor this custom metric and then trigger actions accordingly."
I had already created 7 other metrics based on some log files I send to CloudWatch with no problems.
Some time ago we had a problem with MongoDB connection, and I identified that through logs, so I'd like to create a Metric, so that I can create an Alarm based on it. I did create the Metric, but (of course) there are no data being fed into that Metic, because no more "MongoError" messages exists.
But does that also mean that I can't even access the Metric to create the Alarm? Because this is what is happening right now. The Metric cannot be seen anywhere, only in the "Filters" section of the Logs, which won't allow me to create Alarms or create graphics or anything.
I have already posted this on AWS forums but that usually doesn't help.
An Amazon CloudWatch custom metric is only created when data is stored against the custom metric. Therefore, you'll need to push a data value to make appear and then you will be able to create an alarm.
You can push some data to CloudWatch with the AWS Command-Line Interface (CLI), eg:
aws cloudwatch put-metric-data --namespace MongoDB --metric-name errors --value 0