Running Python script on AWS EC2 - amazon-web-services

Question

Apologies if it is repeat but i couldn't find anything worthwhile to accomplish my task.
I have an instance and i have figured out starting and stopping it using boto3 and it works but the real problem is running the script when instance is up. I would like to wait for script to finish and then stop the instance.
python /home/ubuntu/MyProject/TechInd/EuropeRun.py &
python /home/ubuntu/FTDataCrawlerEU/EuropeRun.py &
Reading quite a few post leads to the direction of Lambda and AWS Beanstalk but those don't appear simple.
Any suggestion is greatly appreciated.
Regards
DC

Answer 1

You can use the following code.
import boto3
import botocore
import os
from termcolor import colored
import paramiko
def stop_instance(instance_id, region_name):
client = boto3.client('ec2', region_name=region_name)
while True:
try:
client.stop_instances(
InstanceIds=[
instance_id,
],
Force=False
)
except Exception, e:
print e
else:
break
# Waiter to wait till instance is stopped
waiter = client.get_waiter('instance_stopped')
try:
waiter.wait(
InstanceIds=[
instance_id,
]
)
except Exception, e:
print e
def ssh_connect(public_ip, cmd):
# Join the paths using directory name and file name, to avoid OS conflicts
key_path = os.path.join('path_to_aws_pem', 'file_name.pem')
key = paramiko.RSAKey.from_private_key_file(key_path)
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
# Connect/ssh to an instance
while True:
try:
client.connect(hostname=public_ip, username="ubuntu", pkey=key)
# Execute a command after connecting/ssh to an instance
stdin, stdout, stderr = client.exec_command(cmd)
print stdout.read()
# close the client connection once the job is done
client.close()
break
except Exception, e:
print e
# Main/Other module where you're doing other jobs:
# Get the public IP address of EC2 instance, I assume you should have handle to the ec2 instance already
# You can use any alternate method to fetch/get public ip of your ec2 instance
public_ip = ec2_instance.public_ip_address
# Get the instance ID of EC2 instance, I assume you should have handle to the ec2 instance already
instance_id = ec2_instance.instance_id
# Command to Run/Execute python scripts
cmd = "nohup python /home/ubuntu/MyProject/TechInd/EuropeRun.py & python /home/ubuntu/FTDataCrawlerEU/EuropeRun.py &"
ssh_connect(public_ip, cmd)
print colored('Script execution finished !!!', 'green')
# Shut down/Stop the instance
stop_instance(instance_id, region_name)

Answer 2

You can execute your shutdown command through python code once your script is done.
An example of using ls
from subprocess import call
call(["ls", "-l"])
But for something that simple lambda is much easier and resource efficient. You only need to upload your script to s3 and the execute the lambda function through boto3.
Actually you can just copy paste your script code to the lambda console if you don't have any dependencies.

Answer 3

Some options for running the script automatically at system startup:
Call the script via the EC2 User-Data
Configure the AMI to start the script on boot via an init.d script, or an #reboot cron job.
To shutdown the instance after the script is complete, add some code at the end of the script to either initiate a OS shutdown, or call the AWS API (via Boto) to shutdown the instance.