I'm trying to automate the creation of a self terminating AWS EC2 instance using AWS LAMBDA. Self-Terminating AWS EC2 Instance?
posted sometime back was helpful to do it via console but I was wondering if this is possible with AWS LAMBDA.
To give some additional context, I'm trying to create a workflow like this:
Get user registration through a unique link sent to them (A simple static intranet website)
POST the form details to AWS LAMBDA function and create an instance from a snapshot which will automatically terminate in say 12 hours
Send the user an email with the connection information (Possibly attaching a RDP file)
The issue I face now is with the second point. I also need to cap the registrations to 10 per day.
PS: Any recommendations or advice on this workflow will be helpful too. (I understand the security issues but this is for intranet and will never be visible to the public)
There are many ways to achieve what you want. Two ways I can think of:
When you launch the instance from Lambda, pass userdata to the instance that sets up a cronjob to terminate the instance using AWS CLI after 12 hours. You can get the instance's id from AWS metadata server. For this you need to have AWS CLI installed and the EC2 instance is attached an IAM role with sufficient privileges to terminate the instance. Or if Python is already preinstalled in that instance, you can write a small python script using Boto3 that terminates the instance 12 hours after the instance is launched.
Easiest and the cleanest option - Schedule a lambda event to fire 12 hours later. You need to pass the instance-id to the second lambda which can terminate the previously launched instance.
In both cases, you need to attach sufficient privileges to the lambda function or to the launched instance to terminate the instance.
These are the step I'd follow
Steps thinking on Microservice, however, you can implement this approach in one Lambda function only.
Create a either a DynamoDB table or RDS instance table to store the instance id TTL (Time to live), and the creation time.
Create a Lambda function with a specific purpose: Terminate instances by instance id.
Create a Lambda function to check the instances according their creation date.
Create a Schedule Cloudwatch event rule that will execute a Lambda function (Created Lambda in step 3) every 10, 30, or whatever N minutes (I recommend a good time interval regarding the amount of instances which will need to be removed at a specific time) to check the created date of instances.
If the creation date of a specific instance is greater or equals to TTL (Time to live), invoke a Lambda function (Created Lambda in step 2) for deletion of that instance.
Wait for Lambda invocation and then delete the table's row related to that instance.
Hope it helps to accomplish your scenario
Another way to do this is to create an EC2 instance from your lambda function. When you call the AWS API to create the EC2 instance, provide a userData script. (You'll probably need to anyway to install whatever software you need).
As part of your userData script, run shutdown -H +12, which will schedule a shutdown event of the machine via Old School Unix Commands.
If you want more detail, take a look at the blog post I stole the shutdown idea from
Related
I would like to investigate that whether it is possible or not that if someone tries to hit the application url then instance should be re-enabled and remain active as long as there is active use. If the resources are inactive for 10 to 20 mins they should automatically disable themselves i.e. instance should get disabled.
Here there are Multi host Application deployed on ec2 instance and configured record set dns in Route53.
Pls suggest
Create a ec2 start lambda function that gets called when you hit the url hosted on APIGateway backed by this lambda, once the instance is up redirect to the actual ec2 instance url(So the lambda will have to keep checking the status of the ec2 instance once running redirect to this url).
On starting the ec2 instance trigger another lambda on the event pattern on the state as running basis which will attach a cloudwatch alarm to the instance.
The cloudwatch alarm will check on the cpu usage and if it goes below 10% for 3 consecutive times it will stop the instance.
The lambda should have the role with policy having full access to the EC2 instance(later on change it to the required method privileges).
blog on stop/start ec2 instance
aws knowledge centre
aws Instance Scheduler
Create an AWS Alarm to monitor usage or activity.
Use an SNS topic to trigger a Lambda function based on the Alarm
Turn off the ec2 instance based using Python in Lambda
This should help with the code:
https://medium.com/geekculture/terraform-setup-for-automatically-turning-off-ec2-instances-upon-inactivity-d7f414390800
I want to create a web app for my organization where users can schedule in advance at what times they'd like their EC2 instances to start and stop (like creating events in a calendar), and those instances will be automatically started or stopped at those times. I've come across four different options:
AWS Datapipeline
Cron running on EC2 instance
Scheduled scaling of Auto Scaling Group
AWS Lambda scheduled events
It seems to me that I'll need a database to store the user's scheduled times for autostarting and autostopping an instance, and that I'll have to pull that data from the database regularly (to make sure that's the latest updated schedule). Which would be the best of the four above options for my use case?
Edit: Auto Scaling only seems to be for launching and terminating instances, so I can rule that out.
Simple!
Ask users to add a tag to their instance(s) indicating when they should start and stop (figure out some format so they can easily specify Mon-Fri or Every Day)
Create an AWS Lambda function that scans instances for their tags and starts/stops them based upon the tag content
Create an Amazon CloudWatch Event rule that triggers the Lambda function every 15 minutes (or whatever resolution you want)
You can probably find some sample code if you search for AWS Stopinator.
Take a look at ParkMyCloud if you're looking for an external SaaS app that can help your users easily schedule (or override that schedule) your EC2, RDS, and ASG instances. It also connects to SSO, provides an API, and shows you all of your resources across regions/accounts/clouds. There's a free trial available if you want to test it out.
Disclosure: I work for ParkMyCloud.
How can I create a policy that will allow a single user in a group to create an ec2 instance in a particular VPC and AZ?
I need it to also be destroyable by the same user or destroyed when idle for more than 24 hours.
You can't. What you can do, however, is script the logic yourself to provide equivalent functionality. A good starting point would be to google AWS SDK.
This is not possible via IAM policies nor other configurations.
You would have to write your own application with this logic, which would launch the instance on the user's behalf and then terminate it later (although that depends on your definition of 'when idle'). The user would interact with this application when they want to launch the instance, rather than interacting directly with AWS.
To be more specific:
It is possible to limit the number of EC2 instances that an AWS Account can have running simultaneously (the default is 20)
It is not possible to limit the number of instances a single user may launch. Either they have permissions to launch an instance with certain attributes, or they don't, but this cannot be limited by whether/what instances are already running.
Some ways you could create a self-terminating instance:
A CloudWatch Alarm could terminate an instance when CPU or Network drops below a certain level for a period of time (eg when Network Out is below 10KB over a half-our period).
A script on the instance itself could decide when to Shutdown the instance. If the instance is launched with Shutdown Behaviour set to Terminate, then shutting down the instance from within the instance will cause it to terminate.
You could create an application that regularly looks at running instances and terminates them 24 hours after being launched. This could be triggered by a Scheduled Task (Windows) or cron job (Linux).
A variation on the previous option is to add a Tag to the instance to indicate when to terminate it. An application could regularly check the tag to determine which instances to terminate.
I have launched an ec2 instance from an ami using lambda function.
I haven't enabled detailed monitoring. Now I want to keep track of the instances each time lambda set triggered to launch an instance. I want to get an email with instance id and status of that, when an instance is launched, stops/terminates and instances which running more than 2 hours. I tried cloudwatch, but is instance specific can't get configured for a newly launched instance. I can uses SNS, but how to keep track of these?
Use AWS Cloudtrail: http://aws.amazon.com/cloudtrail/
It gives more info than you are asking for. In cloudtrail, enable SNS Notifications of API activity and set a filter to notify you only when an instance creation/start/stop/terminate etc., For instances that are running for more than 2 hours, you can explore if cloudtrail provides it or it is very easy to write a script using Boto to fetch that information.
There are some AWS partners providing a similar service. Hope this helps.
I think you will be better off using an auto scale group and then use the auto scale life cycle hook to get notifications whenever any instance is added or removed.
http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/adding-lifecycle-hooks.html
When you use the life cycle hooks, you can have an SNS notification to not only get intimidated about the new instance but also take custom actions.
You can still use the aws lambda function to continue adding and removing new instances, only that you will increase and decrease the size of the auto scaling group.
I'm trying to determine how to remove an instance from several applications (freeIPA, Chef, service discovery) from within an AWS autoscaling group but I'm finding that there's no reliable way to determine if an instance is simply stopping (sometimes our admins will take an instance out of the ASG for analysis) vs terminating. If the instance is stopped then I would like to retain the ability to have it stay connected to our LDAP and other systems. Anyone know a good way to do this?
Is the instance EBS backed or using instance store?
if instance store, you cannot really stop it (only terminate it)
Have you looked at the EC2 API (via aws-sdk maybe)? (looks like describe-instances and looking at reservations should do the trick here)
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html
I determined that the best way for me to do this was to use the ASG alarms (specifically the EC2_TERMINATE alarm). This effectively allows me to take no action if an instance is stopping and fire off a script if it is determined that the instance is terminating.