I'd like to run a very light ruby script once a day and then shutdown my Elastic Cloud instance on AWS in order to reduce its costs.
Is it possible to schedule an instance to auto-start once a day, run a script for one hour and then shut-down?
This article covers how to use auto-scaling to launch your instance for a short time to run a job.
As for the job itself, you can trigger it's execution on startup by any of the methods suitable to your OS/environment. Some are listed on this page.
You can do this with the help of Lambda and CloudWatch.
Please refer: https://aws.amazon.com/premiumsupport/knowledge-center/start-stop-lambda-cloudwatch/
Related
I want to have my ec2 instances to start running some automation script within the server once they are launched in the auto scaling group. This script will need to run every minute so userdata is probably not a good option. I wonder if there is a way to do this with ssm document. Any ideas would be much appreciated!
Do you need to store the logs of the script on either S3 or Cloudwatch? Then use an EventBridge rule and a SSM Run Command document.
Do you not care about such logs? Then use your operating system native tools: cron on Linux or task scheduler on Windows.
I have scheduled 2 cronjobs for my application.
My Application server is in an autoscaling group and I kept a minimum of 2 instances because of High availability. Everything working is fine but cron job is running multiple times because of 2 instances in autoscaling.
I could not limit the instance size to 1 because already my application in the production environment I prefer to have HA.
How should I have to limit execute cron job on a single instance? or should i have to use other services like AWS Lamda or AWS ELasticBeanstalk
Firstly you should consider whether running the crons on these instances is suitable. If you're trying to keep this highly available and it is directly interacted via customers what will the impact of the crons performance be?
Perhaps consider using a separate autoscaling group or instance with a total of 1 instances to run these crons? You could launch the instance or update the autoscaling group just before the cron needs to run and then automate the shutdown after it has completed.
Otherwise you would need to consider using a locking mechanism for your script. By using this your script write a lock to confirm that it is in process, at the beginning of the script run it would check whether there was any script lock in progress. To further prevent the chance of a collision between multiple servers consider adding jitter (random seconds of sleep) to the start of your script.
Suitable technologies for writing a lock are below:
DynamoDB using strongly consistent reads.
EFS for a Linux application, or FSX for a Windows application.
S3 using strong consistency.
Solutions suggested by Chris Williams sound reasonable if using lambda function is not an option.
One way to simulate cron job is by using CloudWatch Events (now known as EventBridge) in conjunction with AWS Lambda.
First you need to write a Lambda function with the code that needs to be executed on a schedule. Lambda supports cron expressions.
You can then use Schedule Expressions with EventBridge/CloudWatch Event in the same way as a cron tab and mention the Lambda function as target.
you can enable termination protection on of the instance. Attach necessary role & permission for system manager. once the instance is available under managed instance under system manager you can create a schedule event in cloudwatch to run ssm documents. if you are running a bash script convert that to ssm document and set this doc as targate. or you can use shellscript document for running commands
I have a Docker image containing Python code and third-party binary executables. There are only outbound network requests. The image must run hourly and each execution lasts ~3 minutes.
I can:
Use an EC2 instance and schedule hourly execution via cron
Create a CloudWatch Event/Rule to run an ECS Task Defintion hourly
Setup an Elastic Beanstalk environment and schedule hourly deployment of the image
In all of these scenarios, an EC2 instance is running 24/7 and I am being charged for extended periods of no usage.
How do I accomplish scheduling the starting of an existing EC2 instance hourly and the stopping of said instance after the completion of my docker image?
Here's one approach I can think of. It's very high-level, and omits some details, but conceptually it would work just fine. You'll also need to consider the Identity & Access Management (IAM) Roles used:
CloudWatch Event Rule to trigger the Step Function
AWS Step Function to trigger the Lambda function
AWS Lambda function to start up EC2 instances
EC2 instance polling the Step Functions service for Activity Tasks
Create a CloudWatch Event Rule to schedule a periodic task, using a cron expression
The Target of the CloudWatch Event Rule is an AWS Step Function
The AWS Step Function State Machine starts by triggering an AWS Lambda function, which starts the EC2 instance
The next step in the Step Functions State Machine invokes an Activity Task, representing the Docker container that needs to execute
The EC2 instance has a script running on it, which polls the Activity Task for work
The EC2 instance executes the Docker container, waits for it to finish, and sends a completion message to the Step Functions Activity Task
The script running on the EC2 instance shuts itself down
The AWS Step Function ends
Keep in mind that a potentially better option would be to spin up a new EC2 instance every hour, instead of simply starting and stopping the same instance. Although you might get better startup performance by starting an existing instance vs. launching a new instance, you'll also have to spend time to maintain the EC2 instance like a pet: fix issues if they crop up, or patch the operating system periodically. In today's world, it's a commonly accepted practice that infrastructure should be disposable. After all, you've already packaged up your application into a Docker container, so you most likely don't have overly specific expectations around which host that container is actually being executed on.
Another option would be to use AWS Fargate, which is designed to run Docker containers, without worrying about spinning up and managing container infrastructure.
AWS Step Functions
AWS Fargate
Blog: AWS Fargate: An Overview
Creating a CloudWatch Event Rule that triggers on a schedule
I would like to know how to create a script that Automatically stop and start a Google Compute Engine instance. and how can I configure him to run every day and choose to run it only 5 days a week?
because we are not using the server it nights so i can save 9 hours a day.
can it be done?
thank you.
You can use gcloud command line tool for that (of course from another machine), it provides all controls, including starting and stopping instances. Setup cron on your local machine for:
gcloud compute instances stop INSTANCE_NAMES
gcloud compute instances start INSTANCE_NAMES
See more:
https://cloud.google.com/sdk/gcloud/reference/compute/instances/stop
https://cloud.google.com/sdk/gcloud/reference/compute/instances/start
As far as I know, GCE doesn't provide scheduled VM stop/start as a managed feature, it has to be triggered outside of the VM. For example, you can a GAE scheduled task which uses gcloud or GCE Python SDK to start and stop your VM.
You can use Google Cloud Scheduler in conjunction with Cloud Functions to run lightweight cronjobs which start/stop GCE VM instances based on a schedule that you control.
You can find a step-by-step tutorial in the official docs, but the general flow is:
Use Cloud Scheduler to publish start/stop messages to a Cloud Pub/Sub topic at the desired times (ex: every weekday at 9am, write a start VM event, every weekday at 5pm, write a stop VM event)
Create a Cloud Function which subscribes to the Pub/Sub topic, and makes the appropriate calls to the GCE APIs to trigger start/stop VM.
I'm running a instance in amazon AWS and it runs non-stop everyday. I'm using ubuntu ec2 instance which is running Apache, Mirthconnect tool and LAMP server. I want to run this instance only on particular time duration of a day. I prefer not use any additional AWS services such as cloud-watch . Is there a way we could acheive this?.
The major purpose is for using Mirthconnect fetching data from mysql database
There are 3 solutions.
AWS Data Pipeline - You can schedule the instance start/stop just like cron. It will cost you one hour of t1.micro instance for every start/stop
AWS Lambda - Define a lambda function that gets triggered at a pre defined time. Your lambda function can start/stop instances. Your cost will be very minimal or $0
Write a shell script and run it as a cron job or run it on demand. The script will have AWS CLI command to start and stop the instance.
I used Data Pipeline for a long time before moving to Lambda. Data Pipeline is very trivial. Just paste the AWS CLI commands to stop and start instances. Lambda is more involved.
I guess for that you'll need another machine which is on 24x7. On which you can write cron job in python using boto or any other language like bash.
I don't see how you start a instance in stopped state without using any other machine.
Or you can have a simple raspberry pi on at your home which does the ON-OFF work for you using AWS CLI or simple Python. How about that? ;)