I would like to investigate that whether it is possible or not that if someone tries to hit the application url then instance should be re-enabled and remain active as long as there is active use. If the resources are inactive for 10 to 20 mins they should automatically disable themselves i.e. instance should get disabled.
Here there are Multi host Application deployed on ec2 instance and configured record set dns in Route53.
Pls suggest
Create a ec2 start lambda function that gets called when you hit the url hosted on APIGateway backed by this lambda, once the instance is up redirect to the actual ec2 instance url(So the lambda will have to keep checking the status of the ec2 instance once running redirect to this url).
On starting the ec2 instance trigger another lambda on the event pattern on the state as running basis which will attach a cloudwatch alarm to the instance.
The cloudwatch alarm will check on the cpu usage and if it goes below 10% for 3 consecutive times it will stop the instance.
The lambda should have the role with policy having full access to the EC2 instance(later on change it to the required method privileges).
blog on stop/start ec2 instance
aws knowledge centre
aws Instance Scheduler
Create an AWS Alarm to monitor usage or activity.
Use an SNS topic to trigger a Lambda function based on the Alarm
Turn off the ec2 instance based using Python in Lambda
This should help with the code:
https://medium.com/geekculture/terraform-setup-for-automatically-turning-off-ec2-instances-upon-inactivity-d7f414390800
Related
I have a utility in Lambda which does automatic DNS registration via an intelligent automated process. Instances that are created with a Name tag including XXXXXX and a autodns tag set to true will have Route 53 records created and deleted when these instances are created and deleted.
Previously, I had been using an autoscaling event listener on targeted autoscaling groups, but this has the unfortunate side-effect of not catching events when autoscaling groups are initially created, as the ASG needs to be created before the subscription can be, so I'm missing instances. A workaround that I've used is to just schedule the Lambda execution every minute and have it search and apply actions, but this is severely limiting.
Is there a way for me to listen to EC2 to receive instance creation and deletion events for all EC2 instances? I have been digging around in CloudWatch and haven't found anything useful.
Yes, you can use Amazon CloudWatch Events to trigger an AWS Lambda function when an Amazon EC2 instance changes state.
I have an application running in EC2 that listen to many ports, some external devices connect to those ports to send data to my application. This is fine, but my client has a requirement that i must monitoring those ports and if one of them stop listening, the instance must be terminated and a new one started.
I was reading about couldwatch, but i didn't found an alarm that i can customize like this (doing requests to ports). Is it possible to do this using cloudwatch ? i'm looking for a direction to create this monitoring, using internal aws services or develop a new solution (maybe a sheel script).
thanks!
I'm not aware of any AWS provided EC2 healthcheck monitoring system for custom checks.
You could write an AWS lambda function which sends requests to the ports on the EC2 instance you require. You can then schedule that lambda to run periodically with whatever frequency you want with Cloudwatch Events. The lambda function could publish this as a metric to cloudwatch which would then make it possible for you to use it in an alarm and thus take action when whatever threshold you deem reasonable to spin up a new replacement instance.
One part of AWS that does have basically what you are looking for built-in though is ECS. Instead of an EC2 instance, you'd have a Docker instance (running on an EC2 instance or Fargate) which can have healthchecks defined.
There are many ways to do what you are asking for.
Simplest solution: I will write a boto3/shell script to monitor the port and call TerminateInstance API or use AWS CLI to terminate the current instance. Needless to say, you need to pass AWS credentials or attach instance profile with sufficient privileges to terminate the instance.
Using Cloudwatch: Have a script to check port status and send 1 or 0 (Dimension: Count) to Cloudwatch. Set a threshold in Cloudwatch if there is consecutive 0s or NoData, then terminate the instance. Or do not send any data to Cloudwatch if the port is not available and NoData in Cloudwatch can trigger TerminateInstance. See: Cloudwatch - AddingTerminateActions
I'm trying to automate the creation of a self terminating AWS EC2 instance using AWS LAMBDA. Self-Terminating AWS EC2 Instance?
posted sometime back was helpful to do it via console but I was wondering if this is possible with AWS LAMBDA.
To give some additional context, I'm trying to create a workflow like this:
Get user registration through a unique link sent to them (A simple static intranet website)
POST the form details to AWS LAMBDA function and create an instance from a snapshot which will automatically terminate in say 12 hours
Send the user an email with the connection information (Possibly attaching a RDP file)
The issue I face now is with the second point. I also need to cap the registrations to 10 per day.
PS: Any recommendations or advice on this workflow will be helpful too. (I understand the security issues but this is for intranet and will never be visible to the public)
There are many ways to achieve what you want. Two ways I can think of:
When you launch the instance from Lambda, pass userdata to the instance that sets up a cronjob to terminate the instance using AWS CLI after 12 hours. You can get the instance's id from AWS metadata server. For this you need to have AWS CLI installed and the EC2 instance is attached an IAM role with sufficient privileges to terminate the instance. Or if Python is already preinstalled in that instance, you can write a small python script using Boto3 that terminates the instance 12 hours after the instance is launched.
Easiest and the cleanest option - Schedule a lambda event to fire 12 hours later. You need to pass the instance-id to the second lambda which can terminate the previously launched instance.
In both cases, you need to attach sufficient privileges to the lambda function or to the launched instance to terminate the instance.
These are the step I'd follow
Steps thinking on Microservice, however, you can implement this approach in one Lambda function only.
Create a either a DynamoDB table or RDS instance table to store the instance id TTL (Time to live), and the creation time.
Create a Lambda function with a specific purpose: Terminate instances by instance id.
Create a Lambda function to check the instances according their creation date.
Create a Schedule Cloudwatch event rule that will execute a Lambda function (Created Lambda in step 3) every 10, 30, or whatever N minutes (I recommend a good time interval regarding the amount of instances which will need to be removed at a specific time) to check the created date of instances.
If the creation date of a specific instance is greater or equals to TTL (Time to live), invoke a Lambda function (Created Lambda in step 2) for deletion of that instance.
Wait for Lambda invocation and then delete the table's row related to that instance.
Hope it helps to accomplish your scenario
Another way to do this is to create an EC2 instance from your lambda function. When you call the AWS API to create the EC2 instance, provide a userData script. (You'll probably need to anyway to install whatever software you need).
As part of your userData script, run shutdown -H +12, which will schedule a shutdown event of the machine via Old School Unix Commands.
If you want more detail, take a look at the blog post I stole the shutdown idea from
Just looking the way to start/stop a AWS EC2 instance in case of CPU utilization increase or decrease on another EC2 instacne. I know there is service available Auto Scaling in AWS but I have a scenario where I can't take advantage of this service.
So just looking if it is possible or anyone can help me on this.
Just detailing the concern like suppose I have 2 EC2 instance on AWS account by name EC21 and EC22. By default, EC22 instance is stopped.
Now I need to setup CloudWatch or any other service to check if load/CPU utilization increase on EC21 instance by 70% then need to start EC22 server and similarly if load decrease on EC21 instance by 30% then stop EC22 server.
Please advice!
When your CloudWatch alarm is triggered, it will notify an SNS topic. You can have that SNS topic then invoke a Lambda function, which can then start your EC2 instance.
Create an AWS Lambda function that starts your EC2 instance.
Configure your SNS topic to invoke your Lambda function when it receives messages. You can read about that here: Invoking Lambda functions using Amazon SNS notifications
Finally, ensure your CloudWatch alert sends messages to the SNS topic.
Yes this is possible for certain types of EC2 instances. Check this detailed guide using which you can set up the triggers in your EC2 instances based on AWS Cloud Watch metrics.
http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/UsingAlarmActions.html
I think your problem might fit the scenario which I'm also trying to solve now - I have some functionality which cannot be solved with Lambdas because of their low lifetime, so I need a relatively short-lived EC2 instance to accomplish the task.
The solution is similar to the one described by Matt, but without SNS, using AWS triggers to launch a lambda function to start the instance. Added benefit is that the lambda function can itself verify whether the EC2 start is really needed.
How do I stop and start Amazon EC2 instances at regular intervals using AWS Lambda?
Issue
I want to reduce my Amazon Elastic Cloud Compute (Amazon EC2) usage by
stopping and starting instances at predefined times or utilization
thresholds. Can I configure AWS Lambda and Amazon CloudWatch to help
me do that automatically?
Short Description
You can use a CloudWatch Event to trigger a Lambda function to start
and stop your EC2 instances at scheduled intervals.
Source: AWS Knowledge Center
I have launched an ec2 instance from an ami using lambda function.
I haven't enabled detailed monitoring. Now I want to keep track of the instances each time lambda set triggered to launch an instance. I want to get an email with instance id and status of that, when an instance is launched, stops/terminates and instances which running more than 2 hours. I tried cloudwatch, but is instance specific can't get configured for a newly launched instance. I can uses SNS, but how to keep track of these?
Use AWS Cloudtrail: http://aws.amazon.com/cloudtrail/
It gives more info than you are asking for. In cloudtrail, enable SNS Notifications of API activity and set a filter to notify you only when an instance creation/start/stop/terminate etc., For instances that are running for more than 2 hours, you can explore if cloudtrail provides it or it is very easy to write a script using Boto to fetch that information.
There are some AWS partners providing a similar service. Hope this helps.
I think you will be better off using an auto scale group and then use the auto scale life cycle hook to get notifications whenever any instance is added or removed.
http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/adding-lifecycle-hooks.html
When you use the life cycle hooks, you can have an SNS notification to not only get intimidated about the new instance but also take custom actions.
You can still use the aws lambda function to continue adding and removing new instances, only that you will increase and decrease the size of the auto scaling group.