Is there a way to configure AWS, EMR, and SNS such that a message is published to an SNS topic when an EMR job fails? I am using a transient EMR cluster for a nightly ETL, and need to be notified if the process fails for any reason.
I needed the same functionality and what I did is I use the AWS data pipeline to create the cluster and send the SNS notification if the pipeline fails.
http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-object-emractivity.html
http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-object-snsalarm.html
Related
In AWS Glue, I am executing a couple of ETL jobs using workflow, Now I want to inform business via email on the failure of any of the ETL jobs. I need help to get name of failed job and the error caused the job to fail, and pass it to job which would trigger an email using Amazon SES.
It has to be done using only a Glue Workflow to trigger a second job that read the output message from the first job and send the email. Need to perform without using EventBridge for this.
Is it possible to call a glue job/ or python script from within another glue job without passing by glue endpoint and adding a new rule in SG?
You can use EventBridge for that. EventBridge supports Glue events.
Here's the scenario, after AWS Backup completes backup cluster aurora RDS, it will send 1 event message to SNS topic. Finally, I'll use lambda to get cluster information.
AWS Backup job completed -> Event message -> SNS topic -> trigger lambda
The thing is that I cannot find any information related to cluster ID ( tried with EventBridge ). I've tried to use Event Subscription ( RDS), but, it won't send any information while I perform AWS Backup.
Please let me know if you have any other ideas, thanks in advance
How to write a cloudwatch alarm for EMR using CLI command??
My requirement is to terminate cluster which are idle for more than 2hours. I need to do this using aws CLI command.
From Monitor Metrics with CloudWatch - Amazon EMR:
The IsIdle metric tracks whether a cluster is live, but not currently running tasks. You can set an alarm to fire when the cluster has been idle for a given period of time, such as thirty minutes.
Therefore, you can configure an alarm on this metric. However, the alarm itself is not able to terminate an Amazon EMR cluster. You would need an additional component, such as an AWS Lambda function, to actually terminate the cluster.
The components would be:
An Amazon CloudWatch IsIdle metric (automatically provided)
An Alarm on the metric that triggers when the cluster is idle for longer than the desired period
Configure the Alarm to send a message to an Amazon SNS topic
Create an AWS Lambda function and subscribe the function to the SNS topic
Code the Lambda function to terminate the Amazon EMR cluster
There is a more complex version of this auto-shutdown process documented at: Optimize Amazon EMR costs with idle checks and automatic resource termination using advanced Amazon CloudWatch metrics and AWS Lambda | AWS Big Data Blog
I have a general AWS question. I have started using AWS sdk, but looks like if I want to receive events asynchronously from AWS(ex: cloudwatch events), lambda functions is the only way. I want to write a simple application that registers a callback to AWS for events, but i couldn't find a way to do that till now, since i don't want to use lambda, i have been polling from my application. Please, let me know if polling is the only option or if there is a better way to resolve it without polling.
From the documentation:
You can configure the following AWS services as targets for CloudWatch Events:
Amazon EC2 instances
AWS Lambda functions
Streams in Amazon Kinesis Streams
Delivery streams in Amazon Kinesis Firehose
Amazon ECS tasks
SSM Run Command
SSM Automation
Step Functions state machines
Pipelines in AWS CodePipeline
Amazon Inspector assessment templates
Amazon SNS topics
Amazon SQS queues
Built-in targets
The default event bus of another AWS account
That's a lot more than just Lambda, so I'm not sure why you state in your question that Lambda is the only option. The options of Amazon EC2 instances and Amazon SNS topics both provide a method for Amazon to "push" the events to your services, instead of requiring your services to poll.
With cloudwatch events, you can set rules and trigger a number of different targets, including SQS queues which you can poll from your EC2 Instances.
Lambda is certainly a popular endpoint, but based on the docs, there are other targets you can send the events to
Already above answers might also be helpful, but one of the possible options to address your problem could be one of this as well.
You can make use of AWS SNS service to subscribe for the events on AWS resources. And the SNS can publish the events to your application end point. Which is nothing but pub/sub model.
Refer this link http://docs.aws.amazon.com/sns/latest/api/API_Subscribe.html
The end-point could be your http or https based application.
I need to receive notifications whenver my instance in terminated. I know it can be done by cloudtrail and then using sns and sqs to get email for it, if you receive event of termination.
Is there a simpler way to do that ?
Any solution will is appreciated, but I prefer is doing using boto.
While it is not possible to receive a notification directly from Amazon EC2 when an instance is terminated, there are a couple of ways this could be accomplished:
Auto Scaling can send a notification when an instance managed by Auto Scaling is terminated. See: Configure Your Auto Scaling Group to Send Notifications
AWS Config can also be configured to send a Simple Notification Service (SNS) notification when resources change. This would send many notifications, so you would need to inspect and filter the notifications to find the one(s) indicating an instance termination. See the SNS reference in: Set Up AWS Config Using the Console and Example Amazon SNS Notification and Email from AWS Config.
Amazon Simple Notification Service (SNS) can also push a message to Amazon Queueing Service (SQS), which can be easily polled with the boto python SDK.
Receiving notifications via CloudTrail and CloudWatch Logs is somewhat messier, so I'd recommend the AWS Config method.
Now AWS introduced "rules" Under "Events" in AWS CloudWatch. In your case, you can select EC2 as Event Selector and SNS or SQS as Targets.
https://aws.amazon.com/blogs/aws/new-cloudwatch-events-track-and-respond-to-changes-to-your-aws-resources/
According to the AWS doc: Spot Instance Interruptions, it is possible to pool the instance-metadata in order to get an approximation of the termination time. You can build any custom monitoring solution around that.
> curl http://169.254.169.254/latest/meta-data/spot/instance-action
{"action": "stop", "time": "2017-09-18T08:22:00Z"}
If the instance is not scheduled for termination a http:400 will be returned.