I have created cloudwatch event rule for glue job state change. I am getting notifications correctly for all glue job state change. But i need to send notifications for some particular glue jobs. i tried with adding multiple jobs but not working properly.
glue jobs:
glue_job1
glue_job2
glue_job3
event rule: with this rule i can get notifications for all jobs.
{
"source": [
"aws.glue"
],
"detail-type": [
"Glue Job State Change"
],
"detail": {
"state": [
"FAILED",
"TIMEOUT",
"SUCCEEDED"
]
}
}
event rule2: with this i am not getting proper notifications
{
"source": [
"aws.glue"
],
"detail-type": [
"Glue Job State Change"
],
"detail": {
"state": [
"FAILED",
"TIMEOUT",
"SUCCEEDED"
],
"jobName": [
"glue_job1",
"glue_job2",
"glue_job3"
]
}
}
how can we send notifications for only specific glue jobs ? how can we create event rule for this scenario. Thank you
Related
I have a CloudWatch alarm for state change with the following code:
{
"source": [
"aws.ec2"
],
"detail-type": [
"EC2 Instance State-change Notification"
],
"detail": {
"state": [
"shutting-down",
"stopping"
]
}
}
It is working fine, but it also triggers events for AWS Instance Scheduler. Is there any way to prevent scheduled state changes from triggering this alarm? I googled for it, but no success.
I have found out that Status Interval Update Event is send to Cloudwatch event every minute when the job is progressing,the interval can be changed to 10 seconds as well.How to show the percentage at the client side from the Cloudwatch event sending events after every 10 seconds?
You will want to capture the STATUS_UPDATE event from CloudWatch and feed that into a service like Lambda that could update a database, or whatever data source you are using to display job stats out.
Example of the Event Pattern
{
"source": [
"aws.mediaconvert"
],
"detail-type": [
"MediaConvert Job State Change"
],
"detail": {
"status": [
"STATUS_UPDATE"
]
}
}
Example of what the Status Update event payload looks like:
{
"version": "0",
"id": "ABC",
"detail-type": "MediaConvert Job State Change",
"source": "aws.mediaconvert",
"account": "111122223333 ",
"time": "2021-02-18T17:52:32Z",
"region": "us-west-2",
"resources": [
"arn:aws:mediaconvert:us-west-2:111122223333 :jobs/1613670689802-emcngz"
],
"detail": {
"timestamp": 1613670752653,
"accountId": "111122223333 ",
"queue": "arn:aws:mediaconvert:us-west-2:111122223333 :queues/Default",
"jobId": "1613670689802-emcngz",
"status": "STATUS_UPDATE",
"userMetadata": {},
"framesDecoded": 2024,
"jobProgress": {
"phaseProgress": {
"PROBING": {
"status": "COMPLETE",
"percentComplete": 100
},
"TRANSCODING": {
"status": "PROGRESSING",
"percentComplete": 2
},
"UPLOADING": {
"status": "PENDING",
"percentComplete": 0
}
},
"jobPercentComplete": 7,
"currentPhase": "TRANSCODING",
"retryCount": 0
}
}
}
MediaConvert provides granular percentages per phases the job is in (probing input, transcoding, and uploading outputs) as well as an overall percentage. The one that is displayed on the MediaConvert Console UI is the jobPercentComplete, and would probably be the one you want to capture.
Documentation:
CloudWatch Events supported by MediaConvert:
https://docs.aws.amazon.com/mediaconvert/latest/ug/mediaconvert_cwe_events.html
How to setup CloudWatch Event:
https://docs.aws.amazon.com/mediaconvert/latest/ug/setting-up-cloudwatch-event-rules.html
I want to setup Cloudwatch events rule based on the task state changes of my ECS cluster via terraform. I would like to be notified whenever the task goes to STOPPED or RUNNING state.
Will this rule work and match whenever either of the events happen? I will then create resource "aws_cloudwatch_event_target" accordingly:
resource "aws_cloudwatch_event_rule" "ecs-sns-rule" {
name = "ecs task state change"
event_pattern = <<PATTERN
{
"source": [
"aws.ecs"
],
"detail-type": [
"ECS Task State Change"
],
"detail": {
"lastStatus": [
"STOPPED",
"RUNNING"
],
"clusterArn": "arn:aws:ecs:us-west-2:XXXXXXXXX:Cluster/XXXXXXXX"
}
}
PATTERN
}
From the docs:
Match values are always in arrays.
Therefore, I think you could try the following (assuming everything else is fine):
resource "aws_cloudwatch_event_rule" "ecs-sns-rule" {
name = "ecs task state change"
event_pattern = <<PATTERN
{
"source": [
"aws.ecs"
],
"detail-type": [
"ECS Task State Change"
],
"detail": {
"lastStatus": [
"STOPPED",
"RUNNING"
],
"clusterArn": ["arn:aws:ecs:us-west-2:XXXXXXXXX:Cluster/XXXXXXXX"]
}
}
PATTERN
}
I want to receive an email every time I update my ECS service (and once the update finishes or the desired state was reached)
I thought about CloudWatch Events Rules setting an SNS topic as target (which a confirmed email address). However, it doesn't work.
This is my custom Event pattern:
{
"detail-type": [
"ECS Update"
],
"resources": [
"arn:aws:ecs:us-east-1:aws-account:service/myService"
],
"source": [
"aws.ecs"
],
"detail": {
"clusterArn": [
"arn:aws:ecs:us-east-1:aws-account:cluster/myCluster"
],
"eventName": [
"SERVICE_STEADY_STATE"
],
"eventType": [
"INFO"
]
}
}
I also tried:
TASKSET_STEADY_STATE
CAPACITY_PROVIDER_STEADY_STATE
SERVICE_DESIRED_COUNT_UPDATED
I'm updating the service through the cli
aws ecs update-service --cluster myCluster --service myService --task-definition myTaskDef --force-new-deployment --desired-count 2
The status of the event rule is enabled and the target is the SNS topic. The input is matched event.
I don't have any clue. Am I using the wrong event name?
You can also set email notification on Task instead of service, also there is an issue regarding ECS notification.
I was not able to make it base on ECS status change, I controlled notification at lambda level. you can set this rule and its working for me.
{
"source": [
"aws.ecs"
],
"detail-type": [
"ECS Service Action"
]
}
you can expect a bit delay as I already experienced this and also reported in GitHub Issue.
Here is the JSON event that you will receive for above rule.
{
"version": "0",
"id": "c3c27e7b-abcd-efgh-c84e-highgclkl",
"detail-type": "ECS Service Action",
"source": "aws.ecs",
"account": "1234567890",
"time": "2020-06-27T00:00:00.00Z",
"region": "us-west-2",
"resources": [
"arn:aws:ecs:us-west-2:1234567890:service/test"
],
"detail": {
"eventType": "INFO",
"eventName": "SERVICE_STEADY_STATE",
"clusterArn": "arn:aws:ecs:us-west-2:123456789:cluster/mycluster",
"createdAt": "2020-06-27T00:00:00.00Z"
}
}
ecs_cwe_events
or the other option is so you can try task-based changes.
{
"source": [
"aws.ecs"
],
"detail-type": [
"ECS Task State Change"
],
"detail": {
"lastStatus": [
"STOPPED",
"RUNNING"
],
"clusterArn": [
"arn:aws:ecs:us-west-2:123456789:cluster/my_cluster",
]
}
}
I have an issue to trigger Cloudwatch Rule on CloudWatch Alarm State Change. This is an Event pattern for a Rule. It doesn't send a message to SNS of state change.
{
"detail-type": [
"CloudWatch Alarm State Change"
],
"resources": [
!Sub "arn:aws:cloudwatch:${AWS:Region}:${AWS:AccountId}:alarm:Admin dead"
],
"source": [
"aws.cloudwatch"
],
"detail": {
"state": [
"ALARM"
]
}
}
The Alarm itself works properly and send a message to SNS in parallel. Also if I will remove this part:
"detail": {
"state": [
"ALARM"
]
}
then the Rule works properly for each state change. But I need only on it's changed to "In alarm" (as it's displayed in UI).
Thanks for any advise
A good way to debug this would be to remove the "detail" part, and subscribe to the SNS topic with email or a lambda function or similar to see the actual alarm event content.
Looks like your rule for "detail" is missing "value" parameter, the following rule works:
{
"source": [
"aws.cloudwatch"
],
"detail-type": [
"CloudWatch Alarm State Change"
],
"detail": {
"state": {
"value": [
"ALARM"
]
}
}
}
According to this, an example event looks like:
{
"version": "0",
"id": "2dde0eb1-528b-d2d5-9ca6-6d590caf2329",
"detail-type": "CloudWatch Alarm State Change",
"source": "aws.cloudwatch",
"account": "123456789012",
"time": "2019-10-02T17:20:48Z",
"region": "us-east-1",
"resources": [
"arn:aws:cloudwatch:us-east-1:123456789012:alarm:TotalNetworkTrafficTooHigh"
],
"detail": {
"alarmName": "TotalNetworkTrafficTooHigh",
"configuration": {
"description": "Goes into alarm if total network traffic exceeds 10Kb",
"metrics": [...]
},
"previousState": {
"reason": "Unchecked: Initial alarm creation",
"timestamp": "2019-10-02T17:20:03.642+0000",
"value": "INSUFFICIENT_DATA"
},
"state": {
"reason": "Threshold Crossed: 1 out of the last 1 datapoints [45628.0 (02/10/19 17:10:00)] was greater than the threshold (10000.0) (minimum 1 datapoint for OK -> ALARM transition).",
"reasonData": "{\"version\":\"1.0\",\"queryDate\":\"2019-10-02T17:20:48.551+0000\",\"startDate\":\"2019-10-02T17:10:00.000+0000\",\"period\":300,\"recentDatapoints\":[45628.0],\"threshold\":10000.0}",
"timestamp": "2019-10-02T17:20:48.554+0000",
"value": "ALARM"
}
}
}
below trick worked for me.I wanted to fetch all the alerts are in alarm state by cloud watch rule.
{
"source": [
"aws.cloudwatch"
],
"detail-type": [
"CloudWatch Alarm State Change"
],
"detail": {
"state": {
"value": [
"ALARM"
]
}
}
}