I have followed ec2 run scripts every boot and tried the same approach for per-instance but my script still doesn't get executed.
I also followed Execute User Data After the Initial Launch of Your EC2 Instance and yet the script doesn't get executed. Content of the script:
nohup /home/ubuntu/anaconda3/envs/tensorflow_p36/bin/python3 -u medi_app.py &> nohup_medi &
Why is it proving so hard to run a script every time an instance restarts? Can someone please help out here?
If you are using an Amazon EC2 Linux instance to run a batch job on startup, simply install the script in this directory:
/var/lib/cloud/scripts/per-boot/
This will work on any system that has cloud-init installed, which also runs User Data scripts.
For more details, see: Auto-Stop EC2 instances when they finish a task - DEV Community
Related
I have a python scraper script and I wish to deploy and run it on 100 ec2 (spot?) instances at the same time. It should run them for an hour and then stop all of them. How should i do it? Is there any aws service that will allow me to orchestrate it easily?
When launching an Amazon EC2 instance (including those on Spot pricing), you can nominate the number of instances to launch. So, you can launch 100 spot instances with a single command.
One hour later, you could then issue a command to terminate the instances using a list of the instances returned by the first command.
Or, you could simply add a command in the startup script of each instance (issued via the User Data field) and waits for one hour and then issues a command to the operating system to shutdown (sudo shutdown now -h). This User Data script can also be used to install and run your scraper code on the instance.
I suggest that you experiment by launching just a single instance until you have things working the way you want.
This question already has answers here:
Aws Ec2 run script program at startup
(5 answers)
Closed 1 year ago.
I am trying to run a python script on ec2 instance . The python file is residing on s3.
I am able to run manually from ec2 instance using iam role which allow access to s3 folder and files.
The question is , how can i automate the start and stop of ec2 instance whenever needed and how to invoke /pass a python file to run upon starting the ec2 instance and stop the instance once the python files completes the execution.
Thanks,
Nikhil
Your requirements seem to be:
Schedule an Amazon EC2 instance to start at a specific time every day
The instance should run a Python script after starting
When the Python script has finished running, Stop the instance
Start EC2 instance on a schedule
You can use Amazon EventBridge to trigger an AWS Lambda function on a schedule.
You can code the Lambda function to call StartInstances() on the EC2 instance to Start it.
Run a script on startup
Install a script into the /var/lib/cloud/scripts/per-boot/ directory. This script can download the Python program from S3 and then run it.
When the EC2 instance starts up, it will automatically run any script in that directory.
Stop the instance when the script is finished
At the end of the script, add the command:
shutdown -h now
This will turn off the instance and place it in the Stopped state.
(This assume that the script is running as root. If it is running as another user, it will need to use sudo shutdown -h now.)
EC2 instances use cloudinit which you can customize to run a given script on each boot. You can use use regular os tools from python to shutdown your instance (e.g. shutdown -h now).
Here another alternative could be to use lambda function instead of EC2 instance to run the python script if maximum execution time of script is less than 15 minutes. Go serverless with AWS lambda rather than EC2. just add your script code in AWS lambda and schedule lambda function from AWS event bridge to invoke it.
I want to run a shell script just before EC2 terminates or stops, i am using Amazon2 AMI. There is no support for Traditional SysVinit in Amazon2 Image, it relies on Daemon services . So i want to make use of systemD services.
My shell script will just gracefully shutdown the tomcat and make some DB changes, should not get executed for more than 2m.
Please suggest.
P.S - I do not want to use LifeCycle Hooks.
Unfortunately by the time you have stopped/terminated the EC2 instance it has entered a shutting down state which would stop this script from running.
If you're hesitant to trigger a script via lifecycle hooks then you will need to script this action.
Look at using Systems Manager Run Command programatically.
Add to the script the terminate or shutdown logic.
I am creating a AWS EC2 launch template that includes commands within the User Data field to perform actions when the instance is first launched (package updates, install software, format EBS volumes, etc). In addition to this I also want to perform tasks on reboot or subsequent starting of the instance, such as mounting existing EBS volumes and configuring and mounting volatile SSD volumes. I see that I can use MIME-type to have code run when instance restarts here:
https://aws.amazon.com/premiumsupport/knowledge-center/execute-user-data-ec2/
So, I can clearly modify User Data after I initially launch the instance, but this is cumbersome as it likely needs manual intervention or requires waiting for the instance to have executed the initial User Data code that runs on initialization of the instance.
My question is:
Can the multi-part MIME format be configured to run code that will execute on initialization of the instance and other code that will run every time the instance restarts?
I understand that what you're trying to achieve is passing two sets of commands using Userdata. One set which will be executed on the instance creation and another set which should be executed every reboot. Please lemme know if I misunderstood it.
For the first part, you can use Userdata itself as you already know. For the commands that should run on every reboot, you can leverage rc.local .
The script /etc/rc.local is for use by the system administrator. It is traditionally executed after all the normal system services are started, at the end of the process of switching to a multiuser runlevel, etc. You might use it to start a custom service or for mounting additional Volumes.
To write into /etc/rc.local , you can download the command set from S3 and copy into the file or you can simply echo it. Example:
echo 'echo HelloWorld' >> /etc/rc.local
Hope this helps.
My question is how do you run a boot script on a windows EC2 (similar to this question but for windows rather than linux)
Also, does a 'stopped' instance being restarted count as a 'boot', or must the instance be terminated in order for the script to run. I ask because I would like the script to run whenever a lambda starts a stopped (not terminated) instance
A script can be passed in the User Data property.
If you are using a Windows AMI, and the first line of the script begins with <powershell> or <script> (for normal DOS commands), then the script will be executed the first time that the instance is started.
For details, see: Running Commands on Your Windows Instance at Launch - Amazon Elastic Compute Cloud
Such a script only runs the first time the instance is started. More accurately, it is only once per Instance ID. This means that if you make an AMI of the instance, then any new instances launched from the AMI will run the User Data script.
If you wish a script to run whenever the virtual machine is turned on, then use the capabilities of the operating system. It has nothing to do with the fact that it is an Amazon EC2 instance.