I create a rule in cloudwatch to trigger a lambda function when a glue job state is changed. The rule patterned is defined:
{
"detail-type": [
"Glue Job State Change"
],
"source": [
"aws.glue"
]
}
In Show metrics for the rule view I can see that there is one FailedInvocation but I can't find a way to see why the invocation is failed. I have checked the lambda function log but it is not being called. So how can I view the log of the failed invocation?
Related
I have a setup wherein I need to trigger a lambda function when my glue crawler has run and data is ready in redshift. Is there a way to create such a trigger?
Edit:
I added an Event bridge rule for crawler state change, that works and triggers the lambda function but it triggers when any of my crawlers are running. I want to isolate it to trigger only after a specific crawler is run. I tested with the code below but it doesn't seem to pick my crawler name. Is there any other way to specify the crawler name in the rule or am I making a syntactical error?
{
"source": ["aws.glue"],
"detail-type": ["Glue Crawler State Change"],
"eventName": "crawler_name",
"detail": {
"state": ["Succeeded"]
}
}
Solution: Add an EvenBridge rule with the following Event Pattern
{
"source": ["aws.glue"],
"detail-type": ["Glue Crawler State Change"],
"detail": {
"crawlerName": ["newton_pfi_new_raw_to_source"],
"state": ["Succeeded"]
}
}
I want to Run cloudwatch rule once the previous step function completes.
This needs to be done multiple times(you can say reusable).
Example- once I trigger a step function rule and its execution gets complete, the next cloudwatch rule shoud get triggered and so on.
Can this be done like- once step function completes, a message should be published to SQS and then using the sqs, a cloudwatch event can get triggered?
We can create a cloudwatch rule with Event Source on Step Function with status SUCCEEDED
{
"source": [
"aws.states"
],
"detail-type": [
"Step Functions Execution Status Change"
],
"detail": {
"status": [
"SUCCEEDED"
],
"stateMachineArn": [
"arn:aws:states:us-east-1:555666611111:stateMachine:my-state-machine-qa"
]
}
}
and add next Stepfunction as Target.
How can I trigger a lambda when a log group is created in cloudwatch? What I am thinking the easiest way to do is to create a cloudwatch rule to send cloudtrail event to lambda. Is it reasonable to do? If yes, how can I filter out other events but only trigger lambda when a log group is created?
The only event type supported by CloudWatch Events (CWE) for CW Logs (CWL) is:
AWS API Call via CloudTrail
Therefore, you can catch the events of interests when you enabled CloudTrail (CT) trail. Once enable, API events would be available in CWE. Then, you would have to create CWE rule which captures CreateLogGroup API call. The rule would trigger your lambda function.
An example CWE rule could be:
{
"source": [
"aws.logs"
],
"detail-type": [
"AWS API Call via CloudTrail"
],
"detail": {
"eventSource": [
"logs.amazonaws.com"
],
"eventName": [
"CreateLogGroup"
]
}
}
I want to be able to setup an AWS CloudWatch event rule that will trigger to an SNS topic whenever one of my Step Functions completes (either success or failure). I do not want this to run for all Step Functions, but there will be an indeterminate number of them based on a common name prefix. Ideally, I'd like to be able to do something like this, but it appears that wildcards are not allowed in Event Patterns. Are there any creative ways to work around this?
{
"source": [
"aws.states"
],
"detail-type": [
"Step Functions Execution Status Change"
],
"detail": {
"status": [
"FAILED",
"SUCCEEDED"
],
"stateMachineArn": [
"arn:aws:states:us-west-1:123456789012:stateMachine:Prefix-*"
]
}
}
Wildcards are not supported in Cloudwatch event rule according to AWS official forum.
You will have to add all the arn's in the state machine ARN list. To do it easily you may write a script that does the following:
Get all the state machine names with specific prefix.
Update the Cloudwatch Event Rule to include all the state machine arn's with specific prefix.
My solution is below:
{
"source": ["aws.states"],
"detail-type": ["Step Functions Execution Status Change"],
"detail": {
"status": ["SUCCEEDED", "FAILED"],
"stateMachineArn": [ { "prefix": "arn:aws:states::us-west-1:123456789012:stateMachine:prefix-" } ]
}
}
I am trying to set up a cloudwatch event that triggers a lambda when the permissions on the EBS snapshot are changed.
For now, the cloudwatch event is limited to the following:
createSnapshot
copySnapshot
shareSnapshot
But obviously, by using any of these, my Lambda is not triggered.
This is the event pattern I am currently using:
{
"source": [
"aws.ec2"
],
"detail-type": [
"EBS Snapshot Notification"
]
}
Does anyone have a suggestion on how the venet patten should look like to trigger my Lambda on a a change on the Permissions?