I have an application running in EC2 that listen to many ports, some external devices connect to those ports to send data to my application. This is fine, but my client has a requirement that i must monitoring those ports and if one of them stop listening, the instance must be terminated and a new one started.
I was reading about couldwatch, but i didn't found an alarm that i can customize like this (doing requests to ports). Is it possible to do this using cloudwatch ? i'm looking for a direction to create this monitoring, using internal aws services or develop a new solution (maybe a sheel script).
thanks!
I'm not aware of any AWS provided EC2 healthcheck monitoring system for custom checks.
You could write an AWS lambda function which sends requests to the ports on the EC2 instance you require. You can then schedule that lambda to run periodically with whatever frequency you want with Cloudwatch Events. The lambda function could publish this as a metric to cloudwatch which would then make it possible for you to use it in an alarm and thus take action when whatever threshold you deem reasonable to spin up a new replacement instance.
One part of AWS that does have basically what you are looking for built-in though is ECS. Instead of an EC2 instance, you'd have a Docker instance (running on an EC2 instance or Fargate) which can have healthchecks defined.
There are many ways to do what you are asking for.
Simplest solution: I will write a boto3/shell script to monitor the port and call TerminateInstance API or use AWS CLI to terminate the current instance. Needless to say, you need to pass AWS credentials or attach instance profile with sufficient privileges to terminate the instance.
Using Cloudwatch: Have a script to check port status and send 1 or 0 (Dimension: Count) to Cloudwatch. Set a threshold in Cloudwatch if there is consecutive 0s or NoData, then terminate the instance. Or do not send any data to Cloudwatch if the port is not available and NoData in Cloudwatch can trigger TerminateInstance. See: Cloudwatch - AddingTerminateActions
Related
I am using CloudWatch alarm to stop ec2 instance. For my case I am pushing log information from my ec2 instance to CloudWatch via log grope. And I filter those information via filter with specific pattern that detect error messages due to failed authentication. Unlike standard ways to stop ec2 instance using CPU utilization. I am using a custom metric (figure). Then I am configuring the action to stop the ec2 instance (figure).
But my alarm appears with state "Insufficient data" all the time. can any one help me to solve the problem and stop my ec2 instance once it is in alarm (means that the logs match the pattern of the filter), Thanks a lot!
I would like to investigate that whether it is possible or not that if someone tries to hit the application url then instance should be re-enabled and remain active as long as there is active use. If the resources are inactive for 10 to 20 mins they should automatically disable themselves i.e. instance should get disabled.
Here there are Multi host Application deployed on ec2 instance and configured record set dns in Route53.
Pls suggest
Create a ec2 start lambda function that gets called when you hit the url hosted on APIGateway backed by this lambda, once the instance is up redirect to the actual ec2 instance url(So the lambda will have to keep checking the status of the ec2 instance once running redirect to this url).
On starting the ec2 instance trigger another lambda on the event pattern on the state as running basis which will attach a cloudwatch alarm to the instance.
The cloudwatch alarm will check on the cpu usage and if it goes below 10% for 3 consecutive times it will stop the instance.
The lambda should have the role with policy having full access to the EC2 instance(later on change it to the required method privileges).
blog on stop/start ec2 instance
aws knowledge centre
aws Instance Scheduler
Create an AWS Alarm to monitor usage or activity.
Use an SNS topic to trigger a Lambda function based on the Alarm
Turn off the ec2 instance based using Python in Lambda
This should help with the code:
https://medium.com/geekculture/terraform-setup-for-automatically-turning-off-ec2-instances-upon-inactivity-d7f414390800
I have an EC2 with the HTTP server; I want to stop and start it automatically when it was not responding on HTTP port after 2 minutes.
What is the best way to implement on AWS without using the scale group and elastic load balancer(ELB).
As I mentioned before, I don't need to create the new instance, just stop and start.
First instead of stopping and starting instance considering restarting service with help of monit or another monitoring service because restarting instance will take time and not a good idea.
But if you are worried about instance going down scenarios, you can configure auto healing(https://aws.amazon.com/blogs/aws/new-auto-recovery-for-amazon-ec2/).
Another custom way of doing would be, inside instance do a simple hello check using curl and store the response log and schedule it in a cron, Sync the log to cloudwatch and in cloudwatch you can plot metric using logs, and configure alarm if the metric count goes below a threshold for 2 mins, you can write lambda function to restart the instance, and associate the lambda to the alarm.(https://aws.amazon.com/premiumsupport/knowledge-center/start-stop-lambda-cloudwatch/). Since you have mentioned you are having one instance this approach will work, if you have more than one instance you need to handle namespace, But again restarting instance is a not a good idea.
I am using Route53 healthcheck which, when triggered, sends notification to SNS topic that triggers Lambda function that reboots the server.
Just looking the way to start/stop a AWS EC2 instance in case of CPU utilization increase or decrease on another EC2 instacne. I know there is service available Auto Scaling in AWS but I have a scenario where I can't take advantage of this service.
So just looking if it is possible or anyone can help me on this.
Just detailing the concern like suppose I have 2 EC2 instance on AWS account by name EC21 and EC22. By default, EC22 instance is stopped.
Now I need to setup CloudWatch or any other service to check if load/CPU utilization increase on EC21 instance by 70% then need to start EC22 server and similarly if load decrease on EC21 instance by 30% then stop EC22 server.
Please advice!
When your CloudWatch alarm is triggered, it will notify an SNS topic. You can have that SNS topic then invoke a Lambda function, which can then start your EC2 instance.
Create an AWS Lambda function that starts your EC2 instance.
Configure your SNS topic to invoke your Lambda function when it receives messages. You can read about that here: Invoking Lambda functions using Amazon SNS notifications
Finally, ensure your CloudWatch alert sends messages to the SNS topic.
Yes this is possible for certain types of EC2 instances. Check this detailed guide using which you can set up the triggers in your EC2 instances based on AWS Cloud Watch metrics.
http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/UsingAlarmActions.html
I think your problem might fit the scenario which I'm also trying to solve now - I have some functionality which cannot be solved with Lambdas because of their low lifetime, so I need a relatively short-lived EC2 instance to accomplish the task.
The solution is similar to the one described by Matt, but without SNS, using AWS triggers to launch a lambda function to start the instance. Added benefit is that the lambda function can itself verify whether the EC2 start is really needed.
How do I stop and start Amazon EC2 instances at regular intervals using AWS Lambda?
Issue
I want to reduce my Amazon Elastic Cloud Compute (Amazon EC2) usage by
stopping and starting instances at predefined times or utilization
thresholds. Can I configure AWS Lambda and Amazon CloudWatch to help
me do that automatically?
Short Description
You can use a CloudWatch Event to trigger a Lambda function to start
and stop your EC2 instances at scheduled intervals.
Source: AWS Knowledge Center
I have launched an ec2 instance from an ami using lambda function.
I haven't enabled detailed monitoring. Now I want to keep track of the instances each time lambda set triggered to launch an instance. I want to get an email with instance id and status of that, when an instance is launched, stops/terminates and instances which running more than 2 hours. I tried cloudwatch, but is instance specific can't get configured for a newly launched instance. I can uses SNS, but how to keep track of these?
Use AWS Cloudtrail: http://aws.amazon.com/cloudtrail/
It gives more info than you are asking for. In cloudtrail, enable SNS Notifications of API activity and set a filter to notify you only when an instance creation/start/stop/terminate etc., For instances that are running for more than 2 hours, you can explore if cloudtrail provides it or it is very easy to write a script using Boto to fetch that information.
There are some AWS partners providing a similar service. Hope this helps.
I think you will be better off using an auto scale group and then use the auto scale life cycle hook to get notifications whenever any instance is added or removed.
http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/adding-lifecycle-hooks.html
When you use the life cycle hooks, you can have an SNS notification to not only get intimidated about the new instance but also take custom actions.
You can still use the aws lambda function to continue adding and removing new instances, only that you will increase and decrease the size of the auto scaling group.