I want to set up AWS Batch running few python scripts to do some batch operations on file fetched from S3 and post processing they need to be saved to a volume.
For this I want to configure compute environments in AWS batch.
I wish to use spot instances but i need my EBS volume to be there even after instance is terminated and if new instance is spin up it has to mount same volume as used before.
Create a instance-template, provide a bootstrap script, for the mentioned case something like:
sudo mkdir -p /<any directory name where volume will be mounted eg: dir>
aws ec2 attach-volume --volume-id <volume_id> --instance-id $(wget -q -O - http://169.254.169.254/latest/meta-data/instance-id) --device /dev/sdf
sudo mount /dev/sdf /<above mentioned dir rg :dir>
in AWS batch definition, use the above template to launch your ec2 machine.
Related
My directory is looking like it is mounted but if I terminate the instance and mount this volume to the new instance I can not see data from the old instance. Idk maybe I am missing some parts of the mounting. here is how I mount in the new instance.
'aws ec2 attach-volume --device /dev/xvdc --instance-id --volume-id --region=us-east-1',
'sudo mkfs -t ext4 /dev/xvdc',
'sudo mkdir /backup',
'sudo mount /dev/xvdc /backup/'
To Clarify my Autoscaling group removes all instances and their root EBS volumes during inactive hours, then once inside active hours recreates and installs all necessary base programs. However I have a smaller EBS volume that is persistent and holds code and data I do not want getting wiped out during down times. I am currently manually attaching via the console and mounting every time I am working inside active hours using the commands below.
sudo mkdir userVolume
sudo mount /dev/xvdf userVolume
How can I automatically attach and mount this volume to a folder? This is all for the sake of minimizing cost and uptime to when I can actually be working on it.
Use this code:
#!/bin/bash
OUTPUT=$(curl http://169.254.169.254/latest/meta-data/instance-id)
aws ec2 attach-volume --volume-id vol-xxxxxxxxxxxx --device /dev/xvdf --instance-id $OUTPUT --region ap-southeast-1
Set your volume ID and region.
Refer this link for further details: https://aws.amazon.com/premiumsupport/knowledge-center/ec2-linux-spot-instance-attach-ebs-volume/
I want to test my Python library in GPU machine once a day.
I decided to use AWS EC2 for testing.
However, the fee of gpu machine is very high, so I want to stop the instance after the test ends.
Thus, I want to do the followings once a day automatically
Start EC2 instance (which is setup manually)
Execute command (test -> push logs to S3)
Stop EC2 (not remove)
How to do this?
It is very simple...
Run script on startup
To run a script automatically when the instance starts (every time it starts, not just the first time), put your script in this directory:
/var/lib/cloud/scripts/per-boot/
Stop instance when test has finished
Simply issue a shutdown command to the operating system at the end of your script:
sudo shutdown now -h
You can push script logs to custom coudwatch namespaces. Like when the process ends publish a state to cloudwatch. In cloudwatch create alarms based on the state of process, so if it has a completed state trigger an AWS lambda function that will stop instance after completion of your job.
Also if you want to start and stop on specific time you can use ec2 instance scheduler to start/stop instances. It just works like a cron job at specific intervals.
You can use the aws cli
To start an instance you would do the following
aws ec2 start-instances --instance-ids i-1234567890abcdef0
and to stop the instance you would do the following
aws ec2 stop-instances --instance-ids i-1234567890abcdef0
To execute commands inside the machine, you will need to ssh into it and run the commands that you need, then you can use the aws cli to upload files to s3
aws s3 cp test.txt s3://mybucket/test2.txt
I suggest reading the aws cli documentation, you will find most if not all what you need to automate aws commands there.
I created a shell script to start an EC2 instance -if not already running,- connect via SSH and, if you want, run a command.
https://gist.github.com/jotaelesalinas/396812f821785f76e5e36cf928777a12
You can use it in three different ways:
./ec2-start-and-ssh.sh -i <instance id> -s
will show status information about your instance: running state and private and public IP addresses.
./ec2-start-and-ssh.sh -i <instance id>
will connect and leave you inside the default shell.
./ec2-start-and-ssh.sh -i <instance id> <command>
will run whatever command you specify, e.g.:
./ec2-start-and-ssh.sh -i <instance id> ./run.sh
./ec2-start-and-ssh.sh -i <instance id> sudo poweroff
I use the last two commands to run periodic jobs minimizing billing costs.
I hope this helps!
I would like to attach an EBS volume not a snapshot as a persistent store for my spot instances. I understand how to manually attach the volume, mount it and get it to survive reboots but how would I get it to automatically get attached at startup?
Is there something I could do in the user data at launching the instance?
Presently I have a ami that I run as a spot instance. I have a separate volume that persists and is used for both input to the instance and to save results. I only ever have one instance up at a time. The ami mounts the drive at /data. For the mount to survive reboots, I have edited /etc/fstab to include:
UUID=MY_VOLUME_UUID /data xfs defaults,nofail 0 2
Again Edited to show Passatizhi's Solution
I added the following to the Configure Instance Details > Advanced Details > User data part of the EC2 launch wizard:
#!/bin/bash
INSTANCE_ID=$(curl 169.254.169.254/latest/meta-data/instance-id)
export AWS_DEFAULT_REGION=$(curl 169.254.169.254/latest/meta-data/placement/availability-zone | sed 's/[a-z]$//')
/home/ubuntu/miniconda3/bin/aws ec2 attach-volume --volume-id vol-myVol12345 --instance-id $INSTANCE_ID --device /dev/sdf
sleep 10
sudo mkdir -p /data
sudo mount /dev/nvme1n1 /data
Note:
I needed to add the full path to aws to get it to work. Also as the ami already has the /data setup I don't need the sudo mkdir -p /data
#!/bin/bash INSTANCE_ID=$(curl 169.254.169.254/latest/meta-data/instance-id)
export AWS_DEFAULT_REGION=$(curl 169.254.169.254/latest/meta-data/placement/availability-zone | sed 's/[a-z]$//')
/bin/aws ec2 attach-volume --volume-id vol-0fdb738415896f8f6 --instance-id $INSTANCE_ID --device /dev/sdf
sleep 10
sudo mkdir -p /data
sudo mount /dev/nvme1n1 /data
Try /bin/aws instead aws. I used t3.small, so /dev/nvme1n1
I am using auto-scaling with desired count as 1 for master node. In case the instance terminates, in order to maintain high availability we need to attach the same EBS volume from previously terminated instance with the newly created one.
Provided CLI is configured on my AMI, I tried each of the followings in user data however it did not work.
#!/bin/bash
EC2_INSTANCE_ID=$(ec2metadata --instance-id)
aws ec2 attach-volume --volume-id vol-777099d8 --instance-id $EC2_INSTANCE_ID --device /dev/sdk
#!/bin/bash
echo "aws ec2 attach-volume --volume-id vol-777099d8 --instance-id $(ec2metadata --instance-id) --device /dev/sdk" > /tmp/xyz.sh
sudo chmod 755 /tmp/xyz.sh
sudo sh /tmp/xyz.sh 2>>
#!/bin/bash
var='ec2 attach-volume --volume-id vol-777099d8 --instance-id $(ec2metadata --instance-id) --device /dev/sdk'
aws "$var"
aws ec2 attach-volume --volume-id vol-777099d8 --instance-id $(ec2metadata --instance-id) --device /dev/sdk
Appreciate your help!
It probably did not work because an EBS volume can only be attached to a single instance at one time. If it did not work you should have error messages in response to the CLI commands to help you understand why it did not work so check the instance's log.
I think you should revisit your architecture a bit because trying to do this sends up a red flag for me. First, a HA architecture should not have a single instance running. A good architecture would remain HA as instances are scaled up and down. If you have data that needs to be available to more than one instance then you should use S3 or EFS to store that data and not an EBS volume.