Just looking the way to start/stop a AWS EC2 instance in case of CPU utilization increase or decrease on another EC2 instacne. I know there is service available Auto Scaling in AWS but I have a scenario where I can't take advantage of this service.
So just looking if it is possible or anyone can help me on this.
Just detailing the concern like suppose I have 2 EC2 instance on AWS account by name EC21 and EC22. By default, EC22 instance is stopped.
Now I need to setup CloudWatch or any other service to check if load/CPU utilization increase on EC21 instance by 70% then need to start EC22 server and similarly if load decrease on EC21 instance by 30% then stop EC22 server.
Please advice!
When your CloudWatch alarm is triggered, it will notify an SNS topic. You can have that SNS topic then invoke a Lambda function, which can then start your EC2 instance.
Create an AWS Lambda function that starts your EC2 instance.
Configure your SNS topic to invoke your Lambda function when it receives messages. You can read about that here: Invoking Lambda functions using Amazon SNS notifications
Finally, ensure your CloudWatch alert sends messages to the SNS topic.
Yes this is possible for certain types of EC2 instances. Check this detailed guide using which you can set up the triggers in your EC2 instances based on AWS Cloud Watch metrics.
http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/UsingAlarmActions.html
I think your problem might fit the scenario which I'm also trying to solve now - I have some functionality which cannot be solved with Lambdas because of their low lifetime, so I need a relatively short-lived EC2 instance to accomplish the task.
The solution is similar to the one described by Matt, but without SNS, using AWS triggers to launch a lambda function to start the instance. Added benefit is that the lambda function can itself verify whether the EC2 start is really needed.
How do I stop and start Amazon EC2 instances at regular intervals using AWS Lambda?
Issue
I want to reduce my Amazon Elastic Cloud Compute (Amazon EC2) usage by
stopping and starting instances at predefined times or utilization
thresholds. Can I configure AWS Lambda and Amazon CloudWatch to help
me do that automatically?
Short Description
You can use a CloudWatch Event to trigger a Lambda function to start
and stop your EC2 instances at scheduled intervals.
Source: AWS Knowledge Center
Related
We have hosted a docker container on AWS ECS with EC2 instances and would like to terminate/showdown these EC2 instances in the night & weekend for Staging/QA to save the cost.
Thanks in advance :)
The AWS Instance Scheduler is a simple AWS-provided solution that enables customers to easily configure custom start and stop schedules for their Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Relational Database Service (Amazon RDS) instances. The solution is easy to deploy and can help reduce operational costs for both development and production environments.
https://aws.amazon.com/solutions/implementations/instance-scheduler/
If you run the instances in an AutoScaling Group (ASG) , you could use scheduled policy to set a desired capacity of the ASG to zero for the off-peak times. A second policy would start it for work time.
Alternative would be setup a CloudWatch Event scheduled rule using cron with target of lambda function. The function would do same as the scaling policy. But because this is lambda function, you could also do some other things there. For example, do some pre-shutdown checks or post-shutdown cleanup.
This will work, because if your tasks run in service, ECS will automatically relaunch your tasks when the instances are back.
You could also manage the number of tasks using scheduling capability of Amazon ECS.
I have a utility in Lambda which does automatic DNS registration via an intelligent automated process. Instances that are created with a Name tag including XXXXXX and a autodns tag set to true will have Route 53 records created and deleted when these instances are created and deleted.
Previously, I had been using an autoscaling event listener on targeted autoscaling groups, but this has the unfortunate side-effect of not catching events when autoscaling groups are initially created, as the ASG needs to be created before the subscription can be, so I'm missing instances. A workaround that I've used is to just schedule the Lambda execution every minute and have it search and apply actions, but this is severely limiting.
Is there a way for me to listen to EC2 to receive instance creation and deletion events for all EC2 instances? I have been digging around in CloudWatch and haven't found anything useful.
Yes, you can use Amazon CloudWatch Events to trigger an AWS Lambda function when an Amazon EC2 instance changes state.
I would like to investigate that whether it is possible or not that if someone tries to hit the application url then instance should be re-enabled and remain active as long as there is active use. If the resources are inactive for 10 to 20 mins they should automatically disable themselves i.e. instance should get disabled.
Here there are Multi host Application deployed on ec2 instance and configured record set dns in Route53.
Pls suggest
Create a ec2 start lambda function that gets called when you hit the url hosted on APIGateway backed by this lambda, once the instance is up redirect to the actual ec2 instance url(So the lambda will have to keep checking the status of the ec2 instance once running redirect to this url).
On starting the ec2 instance trigger another lambda on the event pattern on the state as running basis which will attach a cloudwatch alarm to the instance.
The cloudwatch alarm will check on the cpu usage and if it goes below 10% for 3 consecutive times it will stop the instance.
The lambda should have the role with policy having full access to the EC2 instance(later on change it to the required method privileges).
blog on stop/start ec2 instance
aws knowledge centre
aws Instance Scheduler
Create an AWS Alarm to monitor usage or activity.
Use an SNS topic to trigger a Lambda function based on the Alarm
Turn off the ec2 instance based using Python in Lambda
This should help with the code:
https://medium.com/geekculture/terraform-setup-for-automatically-turning-off-ec2-instances-upon-inactivity-d7f414390800
I have an application running in EC2 that listen to many ports, some external devices connect to those ports to send data to my application. This is fine, but my client has a requirement that i must monitoring those ports and if one of them stop listening, the instance must be terminated and a new one started.
I was reading about couldwatch, but i didn't found an alarm that i can customize like this (doing requests to ports). Is it possible to do this using cloudwatch ? i'm looking for a direction to create this monitoring, using internal aws services or develop a new solution (maybe a sheel script).
thanks!
I'm not aware of any AWS provided EC2 healthcheck monitoring system for custom checks.
You could write an AWS lambda function which sends requests to the ports on the EC2 instance you require. You can then schedule that lambda to run periodically with whatever frequency you want with Cloudwatch Events. The lambda function could publish this as a metric to cloudwatch which would then make it possible for you to use it in an alarm and thus take action when whatever threshold you deem reasonable to spin up a new replacement instance.
One part of AWS that does have basically what you are looking for built-in though is ECS. Instead of an EC2 instance, you'd have a Docker instance (running on an EC2 instance or Fargate) which can have healthchecks defined.
There are many ways to do what you are asking for.
Simplest solution: I will write a boto3/shell script to monitor the port and call TerminateInstance API or use AWS CLI to terminate the current instance. Needless to say, you need to pass AWS credentials or attach instance profile with sufficient privileges to terminate the instance.
Using Cloudwatch: Have a script to check port status and send 1 or 0 (Dimension: Count) to Cloudwatch. Set a threshold in Cloudwatch if there is consecutive 0s or NoData, then terminate the instance. Or do not send any data to Cloudwatch if the port is not available and NoData in Cloudwatch can trigger TerminateInstance. See: Cloudwatch - AddingTerminateActions
I have launched an ec2 instance from an ami using lambda function.
I haven't enabled detailed monitoring. Now I want to keep track of the instances each time lambda set triggered to launch an instance. I want to get an email with instance id and status of that, when an instance is launched, stops/terminates and instances which running more than 2 hours. I tried cloudwatch, but is instance specific can't get configured for a newly launched instance. I can uses SNS, but how to keep track of these?
Use AWS Cloudtrail: http://aws.amazon.com/cloudtrail/
It gives more info than you are asking for. In cloudtrail, enable SNS Notifications of API activity and set a filter to notify you only when an instance creation/start/stop/terminate etc., For instances that are running for more than 2 hours, you can explore if cloudtrail provides it or it is very easy to write a script using Boto to fetch that information.
There are some AWS partners providing a similar service. Hope this helps.
I think you will be better off using an auto scale group and then use the auto scale life cycle hook to get notifications whenever any instance is added or removed.
http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/adding-lifecycle-hooks.html
When you use the life cycle hooks, you can have an SNS notification to not only get intimidated about the new instance but also take custom actions.
You can still use the aws lambda function to continue adding and removing new instances, only that you will increase and decrease the size of the auto scaling group.