Perform Root Valume Swap for EC2 via the AWS console - amazon-web-services

I've recently been using the Swap root volume approach for creating a persistent Spot Instance, as described here (Approach 2). Typically it takes 2-5 minutes for my Spot Instance to be fulfilled and the Swap to complete. However, some days, the process never finishes (or at least I get impatient after waiting 20 minutes to an hour!).
To be clear, the Instance is created, but the Swap never happens: I can ssh into the server but my persistent files are not there. I also can see this by going to my AWS console and noting that "spotter" (my persistent storage) has no attachment information:
As the swapping script which I'm using never gives me any errors, it's hard to see what's failing. So, I'm wondering if based on my screenshot I can just use the AWS EC2 Management Console to "manually" perform the swap, and if so, how would I accomplish this.
And, if it helps #Vorsprung,
I initiate the process by running the following script:
# The config file was created in ondemand_to_spot.sh
export config_file=my.conf
cd "$(dirname ${BASH_SOURCE[0]})"
. ../$config_file || exit -1
export request_id=`../ec2spotter-launch $config_file`
echo Spot request ID: $request_id
echo Waiting for spot request to be fulfilled...
aws ec2 wait spot-instance-request-fulfilled --spot-instance-request-ids $request_id
export instance_id=`aws ec2 describe-spot-instance-requests --spot-instance-request-ids $request_id --query="SpotInstanceRequests[*].InstanceId" --output="text"`
echo Waiting for spot instance to start up...
aws ec2 wait instance-running --instance-ids $instance_id
echo Spot instance ID: $instance_id
echo 'Please allow the root volume swap script a few minutes to finish.'
if [ "x$ec2spotter_elastic_ip" = "x" ]
then
# Non elastic IP
export ip=`aws ec2 describe-instances --instance-ids $instance_id --filter Name=instance-state-name,Values=running --query "Reservations[*].Instances[*].PublicIpAddress" --output=text`
else
# Elastic IP
export ip=`aws ec2 describe-addresses --allocation-ids $ec2spotter_elastic_ip --output text --query 'Addresses[0].PublicIp'`
fi
export name=fast-ai
if [ "$ec2spotter_key_name" = "aws-key-$name" ]
then function aws-ssh-spot {
ssh -i ~/.ssh/aws-key-$name.pem ubuntu#$ip
}
function aws-terminate-spot {
aws ec2 terminate-instances --instance-ids $instance_id
}
echo Jupyter Notebook -- $ip:8888
fi
where my.conf is:
# Name of root volume.
ec2spotter_volume_name=spotter
# Location (zone) of root volume. If not the same as ec2spotter_launch_zone,
# a copy will be created in ec2spotter_launch_zone.
# Can be left blank, if the same as ec2spotter_launch_zone
ec2spotter_volume_zone=us-west-2b
ec2spotter_launch_zone=us-west-2b
ec2spotter_key_name=aws-key-fast-ai
ec2spotter_instance_type=p2.xlarge
# Some instance types require a subnet to be specified:
ec2spotter_subnet=subnet-c9cba8af
ec2spotter_bid_price=0.55
# uncomment and update the value if you want an Elastic IP
# ec2spotter_elastic_ip=eipalloc-64d5890a
# Security group
ec2spotter_security_group=sg-2be79356
# The AMI to be used as the pre-boot environment. This is NOT your target system installation.
# Do Not Modify this unless you have a need for a different Kernel version from what's supplied.
# ami-6edd3078 is ubuntu-xenial-16.04-amd64-server-20170113
ec2spotter_preboot_image_id=ami-bc508adc
and the ec2spotter-launch script is:
#!/bin/bash
# "Phase 1" this is the user-facing script for launching a new spot istance
if [ "$1" = "" ]; then echo "USER ERROR: please specify a configuration file"; exit -1; fi
cd $(dirname $0)
. $1 || exit -1
# New instance:
# Desired launch zone
LAUNCH_ZONE=$ec2spotter_launch_zone
# Region is LAUNCH_ZONE minus the last character
LAUNCH_REGION=$(echo $LAUNCH_ZONE | sed -e 's/.$//')
PUB_KEY=$ec2spotter_key_name
# Existing Volume:
# If no volume zone
if [ "$ec2spotter_volume_zone" = "" ]
then # Use instance zone
ec2spotter_volume_zone=$LAUNCH_ZONE
fi
# Name of volume (find it by name later)
ROOT_VOL_NAME=$ec2spotter_volume_name
# zone of volume (needed if different than instance zone)
ROOT_ZONE=$ec2spotter_volume_zone
# Region is Zone minus the last character
ROOT_REGION=$(echo $ROOT_ZONE | sed -e 's/.$//')
#echo "ROOT_VOL_NAME=${ROOT_VOL_NAME}; ROOT_ZONE=${ROOT_ZONE}; ROOT_REGION=${ROOT_REGION}; "
#echo "LAUNCH_ZONE=${LAUNCH_ZONE}; LAUNCH_REGION=${LAUNCH_REGION}; PUB_KEY=${PUB_KEY}"
AWS_ACCESS_KEY=`aws configure get aws_access_key_id`
AWS_SECRET_KEY=`aws configure get aws_secret_access_key`
aws ec2 describe-volumes \
--filters Name=tag-key,Values="Name" Name=tag-value,Values="$ROOT_VOL_NAME" \
--region ${ROOT_REGION} --output=json > volumes.tmp || exit -1
ROOT_VOL=$(jq -r '.Volumes[0].VolumeId' volumes.tmp)
ROOT_TYPE=$(jq -r '.Volumes[0].VolumeType' volumes.tmp)
#echo "ROOT_TYPE=$ROOT_TYPE; ROOT_VOL=$ROOT_VOL";
if [ "$ROOT_VOL_NAME" = "" ]
then
echo "root volume lacks a Name tag";
exit -1;
fi
cat >user-data.tmp <<EOF
#!/bin/sh
echo AWSAccessKeyId=$AWS_ACCESS_KEY > /root/.aws.creds
echo AWSSecretKey=$AWS_SECRET_KEY >> /root/.aws.creds
apt-get update
apt-get install -y jq
apt-get install -y python-pip python-setuptools
apt-get install -y git
pip install awscli
cd /root
git clone --depth=1 https://github.com/slavivanov/ec2-spotter.git
echo Got spotter scripts from github.
cd ec2-spotter
echo Swapping root volume
./ec2spotter-remount-root --force 1 --vol_name ${ROOT_VOL_NAME} --vol_region ${ROOT_REGION} --elastic_ip $ec2spotter_elastic_ip
EOF
userData=$(base64 user-data.tmp | tr -d '\n');
cat >specs.tmp <<EOF
{
"ImageId" : "$ec2spotter_preboot_image_id",
"InstanceType": "$ec2spotter_instance_type",
"KeyName" : "$PUB_KEY",
"EbsOptimized": true,
"Placement": {
"AvailabilityZone": "$LAUNCH_ZONE"
},
"BlockDeviceMappings": [
{
"DeviceName": "/dev/sda1",
"Ebs": {
"DeleteOnTermination": true,
"VolumeType": "gp2",
"VolumeSize": 128
}
}
],
"NetworkInterfaces": [
{
"DeviceIndex": 0,
"SubnetId": "${ec2spotter_subnet}",
"Groups": [ "${ec2spotter_security_group}" ],
"AssociatePublicIpAddress": true
}
],
"UserData" : "${userData}"
}
EOF
SPOT_REQUEST_ID=$(aws ec2 request-spot-instances --launch-specification file://specs.tmp --spot-price $ec2spotter_bid_price --output="text" --query="SpotInstanceRequests[*].SpotInstanceRequestId" --region ${LAUNCH_REGION})
echo $SPOT_REQUEST_ID
# Clean up
rm user-data.tmp
rm specs.tmp
rm volumes.tmp

This is not an exact answer, but it may help you to find the way to debug the issue.
As I understand, this is the part of your setup is in the ec2spotter-launch script responsible for volume swap:
...
cat >specs.tmp <<EOF
{
"ImageId" : "$ec2spotter_preboot_image_id",
...
"UserData" : "${userData}"
}
EOF
SPOT_REQUEST_ID=$(aws ec2 request-spot-instances --launch-specification file://specs.tmp --spot-price $ec2spotter_bid_price --output="text" --query="SpotInstanceRequests[*].SpotInstanceRequestId" --region ${LAUNCH_REGION})
The specs.tmp is used as instance launch specification: --launc-specification file:://specs.tmp.
And the "UserData" inside the launch specification is a script which is also generated in es2spotter-launch:
cat >user-data.tmp <<EOF
#!/bin/sh
echo AWSAccessKeyId=$AWS_ACCESS_KEY > /root/.aws.creds
echo AWSSecretKey=$AWS_SECRET_KEY >> /root/.aws.creds
apt-get update
...
cd /root
git clone --depth=1 https://github.com/slavivanov/ec2-spotter.git
echo Got spotter scripts from github.
cd ec2-spotter
echo Swapping root volume
./ec2spotter-remount-root --force 1 --vol_name ${ROOT_VOL_NAME} --vol_region ${ROOT_REGION} --elastic_ip $ec2spotter_elastic_ip
EOF
The actual work to swap the root volume is performed by the ec2spotter-remount-root script which is downloaded from github.
There are many echo statements in that script, so I think if you find where the output goes, you'll be able to understand what was wrong.
So when you have the issue, you'll ssh to the instance and check the log file.
The question is what file to check (and if the script output is being logged into some file).
Here is what I suggest to try:
Check standard logs under /var/log generated when the instance is starting (cloud-init.log, syslog, etc) to see if you can find the ec2spotter-remount-root output
Try to enable logging by yourself, something similar is discussed here
I would try modifying the user-data.tmp part in es2spotter-launch this way:
#!/bin/bash
set -x
exec > >(tee /var/log/user-data.log|logger -t user-data ) 2>&1
echo AWSAccessKeyId=$AWS_ACCESS_KEY > /root/.aws.creds
...
echo Swapping root volume
./ec2spotter-remount-root --force 1 --vol_name ${ROOT_VOL_NAME} --vol_region ${ROOT_REGION} --elastic_ip $ec2spotter_elastic_ip
EOF
Here I've changed first three lines to enable logging into /var/log/user-data.log.
If 1 and 2 don't work, I would try asking script author on github. As there are lots of echos in the script, the author should know where to look for that output.
Hope that helps, also you don't need to wait for the issue to appear to try this out, instead look for the script output on successful runs too.
Or, if you are able to make few test runs, then do that and make sure you can find the log with script output.

Related

EC2 user-data Script Not Redirect `user-data.log` Output to Console

Description
Looking at this AWS EC2 Doc It should be possible to add exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1 to a user-data script that is run on EC2 initialization.
When running aws ec2 --region eu-west-1 get-console-output --instance-id i-<id> | grep "user-data" (or searching for other patterns that should be present) none are found after the ec2 initialization.
Goal
To read the results and debug information from this initialization script without needing to SSH into the ec2 instance and poll the logs for the shutdown statement. Using the Instance shutdown as the "finished" state has a significant simplification on the deployment process for this repository.
Question
What about this particular setup is not correct such that we are not getting logs out of the aws ec2 get-console-output command.
Alternative answer: What's a better method of retrieving the logs from an EC2 instance.
User-Data Script
#!/bin/bash -xe
# redirect output to log file and console
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
if [ -n "postgresql,jq" ]
then
yum -y -q update
echo "complete: yum update"
IFS=', ' read -r -a REQS <<< "postgresql,jq"
for REQ in "${REQS[#]}"
do
yum -y -q install $REQ
echo "complete: yum install $REQ"
done
fi
echo "EC2P: ENTERING BOOTSTRAP SCRIPT....."
export PGPASSWORD=$( aws secretsmanager get-secret-value --secret-id <secret_arn> --query SecretString --region eu-west-1 | jq -r . | jq -r .password)
aws s3 cp s3://<query_object> /tmp/postgres-query.sql
echo "EC2P: Starting PSQL Execution"
psql -h <host> \
-p 5432 \
-U <user> \
-o /tmp/postgres-query-result.txt \
<db_name>_db \
< /tmp/postgres-query.sql \
> /tmp/postgres-query-output.txt 2>&1
echo "psql exit code is $?"
echo "EC2P: PSQL Execution Complete"
# since instance_initiated_shutdown_behavior = "terminate"
echo "shutdown"
shutdown
Terraform Declaration Of EC2 Instance
resource "aws_instance" "ec2_provisioner" {
count = var.enabled ? 1 : 0
ami = data.aws_ami.ec2_provisioner.id
iam_instance_profile = var.iam_instance_profile
instance_initiated_shutdown_behavior = "terminate"
instance_type = var.instance_type
root_block_device {
volume_type = "gp2"
volume_size = "16"
delete_on_termination = "true"
}
subnet_id = var.subnet_id
tags = merge(
{
"es:global:component-name" = "${var.component_name}-ec2-provisioner",
"Name" = var.name
},
jsondecode(var.additional_tags)
)
user_data = templatefile(
"${path.module}/user_data.tpl",
{
BASH_SCRIPT = var.bash_script,
PACKAGES = join(",", var.packages)
}
)
volume_tags = {
"Name" = var.name
}
vpc_security_group_ids = [var.security_group_id]
}
The core issue here is that our initialization of the instance is producing too much console output
74295 # > 64 KBs
Which is hitting a size limit built into the get-console-output command
By default, the console output returns buffered information that was posted shortly after an instance transition state (start, stop, reboot, or terminate). This information is available for at least one hour after the most recent post. Only the most recent 64 KB of console output is available
The solution we're going with to fix this issue to to enable SSM and to log into parse the log output.

How to make Terraform wait for cloudinit to finish?

In my Terraform AWS Docker Swarm module I use cloud-init to initialize the EC2 instance. However, Terraform says the resource is ready before cloud-init finishes. Is there a way of making it wait for cloud-init to finish ideally without SSHing or checking for a port to be up using a null resource.
Your managers and workers both use template_cloudinit_config. They also have ec2:CreateTags.
You can use an EC2 resource tag like trajano/terraform-docker-swarm-aws/cloudinit-complete to indicate that the cloudinit has finished.
You could add this final part to each to invoke a tagging script:
part {
filename = "tag_complete.sh"
content = local.tag_complete_script
content_type = "text/x-shellscript"
}
And declare tag_complete_script be the following:
locals {
tag_complete_script = <<-EOF
#!/bin/bash
instance_id="${TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/instance-id}"
aws ec2 create-tags --resources "$instance_id" --tags 'Key=trajano/terraform-docker-swarm-aws/cloudinit-complete,Value=true'
EOF
}
Then with a null_resource, you wait for the tag to appear (wrote this on my phone, so use it for a general idea, but I don't expect that it will work without testing and edits):
resource "null_resource" "wait_for_cloudinit" {
provisioner "local-exec" {
command = <<-EOF
#!/bin/bash
poll_tags="aws ec2 describe-tags --filters 'Name=resource-id,Values=${join(",", aws_instance.managers[*].id)}' 'Name=key,Values=trajano/terraform-docker-swarm-aws/cloudinit-complete' --output text --query 'Tags[*].Value'"
expected='${join(",", formatlist("true", aws_instance.managers[*].id))}'
$tags="$($poll_tags)"
while [[ "$tags" != "$expected" ]] ; do
$tags="$($poll_tags)"
done
EOF
}
}
This way you can have dependencies on null_resource.wait_for_cloudinit on any resources that need to run after cloudinit has completed.
Another possible approach is using AWS Systems Manager Run Command, if available on your AMI.
You create an SSM Document with Terraform that uses the cloud-init status --wait command, then you trigger the command from a local provisioner, and wait for it to complete. In this way, you don't have to play around with tags, and you are 100% sure cloud-init has been completed.
This is an example of the document you can create with Terraform:
resource "aws_ssm_document" "cloud_init_wait" {
name = "cloud-init-wait"
document_type = "Command"
document_format = "YAML"
content = <<-DOC
schemaVersion: '2.2'
description: Wait for cloud init to finish
mainSteps:
- action: aws:runShellScript
name: StopOnLinux
precondition:
StringEquals:
- platformType
- Linux
inputs:
runCommand:
- cloud-init status --wait
DOC
}
and then you can use a local-provisioner inside the EC2 instance block, or in a null resource, up to what you have to do with it.
The provisioner would be more or less like this:
provisioner "local-exec" {
interpreter = ["/bin/bash", "-c"]
command = <<-EOF
set -Ee -o pipefail
export AWS_DEFAULT_REGION=${data.aws_region.current.name}
command_id=$(aws ssm send-command --document-name ${aws_ssm_document.cloud_init_wait.arn} --instance-ids ${self.id} --output text --query "Command.CommandId")
if ! aws ssm wait command-executed --command-id $command_id --instance-id ${self.id}; then
echo "Failed to start services on instance ${self.id}!";
echo "stdout:";
aws ssm get-command-invocation --command-id $command_id --instance-id ${self.id} --query StandardOutputContent;
echo "stderr:";
aws ssm get-command-invocation --command-id $command_id --instance-id ${self.id} --query StandardErrorContent;
exit 1;
fi;
echo "Services started successfully on the new instance with id ${self.id}!"
EOF
}

How to Execute bash code in EC2 instance from Automation Document deployed with CloudFormation

I'm trying to execute bash code via aws:runcommand. I have taken and adapted the below snippet from the AWS Repo to deploy a Golden Image pipeline
What you see below is deployed via a CloudFormation Stack. An AWS::SSM::Document object is formed, passing various inputs. This is one of the mainSteps of my automation document. I'm trying to update the OS of my instance.
{
"name": "updateOSSoftware",
"action": "aws:runCommand",
"maxAttempts": 3,
"timeoutSeconds": 3600,
"onFailure": "Abort",
"inputs": {
"DocumentName": "AWS-RunShellScript",
"InstanceIds": [
"{{startInstances.InstanceIds}}"
],
"Parameters": {
"commands": [
"export https_proxy=http://myproxy.com:myport",
"export https_proxy='{{OutBoundProxy}}'",
"set -e",
"[ -x \"$(which wget)\" ] && get_contents='wget $1 -O -'",
"[ -x \"$(which curl)\" ] && get_contents='curl -s -f $1'",
"eval $get_contents https://aws-ssm-downloads-eu-west-1.s3.amazonaws.com/scripts/aws-update-linux-instance > /tmp/aws-update-linux-instance",
"chmod +x /tmp/aws-update-linux-instance",
"/tmp/aws-update-linux-instance --pre-update-script '{{PreUpdateScript}}' --post-update-script '{{PostUpdateScript}}' --include-packages '{{IncludePackages}}' --exclude-packages '{{ExcludePackages}}' 2>&1 | tee /tmp/aws-update-linux-instance.log"
]
}
}
}
Once I execute the document from the Systems Manger, I SSH into the EC2 instance and trying and echo $http_proxy, the variable is unset, showing the code wasn't launched.
How can I run bash code?
The environment variable exported using 'export' command is for the current shell only.
If you login to the hosts via ssh, they will be in a different shell, I don't think you could see the env variable set via 'RunCommand'.
A workaround could be add the command in bash profile. e.g.
echo "export https_proxy=http://myproxy.com:myport" >> ~/.bash_profile

AWS CLI commands on JENKINS

I'm trying to write a script that with the help of Jenkins will look at the updated files in git, download it and will encrypt them using AWS KMS. I have a working script that does it all and the file is downloaded to the Jenkins repository on local server. But my problem is encrypting this file in Jenkins repo. Basically, when I encrypt files on local computer, I use the command:
aws kms encrypt --key-id xxxxxxx-xxx-xxxx-xxxx-xxxxxxxxxx --plaintext fileb://file.json --output text --query CiphertextBlob | base64 --decode > Encrypted-data.json
and all is ok, but if I try to do it with Jenkins I get an error that the AWS command not found.
Does somebody know how to solve this problem and how do I make it run AWS through Jenkins?
Here is my working code which breaks down on the last line:
bom_sniffer() {
head -c3 "$1" | LC_ALL=C grep -qP '\xef\xbb\xbf';
if [ $? -eq 0 ]
then
echo "BOM SNIFFER DETECTED BOM CHARACTER IN FILE \"$1\""
exit 1
fi
}
check_rc() {
# exit if passed in value is not = 0
# $1 = return code
# $2 = command / label
if [ $1 -ne 0 ]
then
echo "$2 command failed"
exit 1
fi
}
# finding files that differ from this commit and master
echo 'git fetch'
check_rc $? 'echo git fetch'
git fetch
check_rc $? 'git fetch'
echo 'git diff --name-only origin/master'
check_rc $? 'echo git diff'
diff_files=`git diff --name-only $GIT_PREVIOUS_COMMIT $GIT_COMMIT | xargs`
check_rc $? 'git diff'
for x in ${diff_files}
do
echo "${x}"
cat ${x}
bom_sniffer "${x}"
check_rc $? "BOM character detected in ${x},"
aws configure kms encrypt --key-id xxxxxxx-xxx-xxxx-xxxx-xxxxxxxxxx --plaintext fileb://${x} --output text --query CiphertextBlob | base64 --decode > Encrypted-data.json
done
After discussion with you this is how this issue was resolved :
First corrected the command by removing configure from it.
Installed the awscli for the jenkins user :
pip install awscli --user
Used the absolute path of aws in your script
for eg. say if it's in ~/.local/bin/ use ~/.local/bin/aws kms encrypt --key-id xxxxxxx-xxx-xxxx-xxxx-xxxxxxxxxx --plaintext fileb://${x} --output text --query CiphertextBlob | base64 --decode > Encrypted-data.json in your script.
Or add the path of aws in PATH.

Query EC2 tags from within instance

Amazon recently added the wonderful feature of tagging EC2 instances with key-value pairs to make management of large numbers of VMs a bit easier.
Is there some way to query these tags in the same way as some of the other user-set data? For example:
$ curl http://169.254.169.254/latest/meta-data/placement/availability-zone
us-east-1d
Is there some similar way to query the tags?
The following bash script returns the Name of your current ec2 instance (the value of the "Name" tag). Modify TAG_NAME to your specific case.
TAG_NAME="Name"
INSTANCE_ID="`wget -qO- http://instance-data/latest/meta-data/instance-id`"
REGION="`wget -qO- http://instance-data/latest/meta-data/placement/availability-zone | sed -e 's:\([0-9][0-9]*\)[a-z]*\$:\\1:'`"
TAG_VALUE="`aws ec2 describe-tags --filters "Name=resource-id,Values=$INSTANCE_ID" "Name=key,Values=$TAG_NAME" --region $REGION --output=text | cut -f5`"
To install the aws cli
sudo apt-get install python-pip -y
sudo pip install awscli
In case you use IAM instead of explicit credentials, use these IAM permissions:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [ "ec2:DescribeTags"],
"Resource": ["*"]
}
]
}
Once you've got ec2-metadata and ec2-describe-tags installed (as mentioned in Ranieri's answer above), here's an example shell command to get the "name" of the current instance, assuming you have a "Name=Foo" tag on it.
Assumes EC2_PRIVATE_KEY and EC2_CERT environment variables are set.
ec2-describe-tags \
--filter "resource-type=instance" \
--filter "resource-id=$(ec2-metadata -i | cut -d ' ' -f2)" \
--filter "key=Name" | cut -f5
This returns Foo.
You can use a combination of the AWS metadata tool (to retrieve your instance ID) and the new Tag API to retrieve the tags for the current instance.
You can add this script to your cloud-init user data to download EC2 tags to a local file:
#!/bin/sh
INSTANCE_ID=`wget -qO- http://instance-data/latest/meta-data/instance-id`
REGION=`wget -qO- http://instance-data/latest/meta-data/placement/availability-zone | sed 's/.$//'`
aws ec2 describe-tags --region $REGION --filter "Name=resource-id,Values=$INSTANCE_ID" --output=text | sed -r 's/TAGS\t(.*)\t.*\t.*\t(.*)/\1="\2"/' > /etc/ec2-tags
You need the AWS CLI tools installed on your system: you can either install them with a packages section in a cloud-config file before the script, use an AMI that already includes them, or add an apt or yum command at the beginning of the script.
In order to access EC2 tags you need a policy like this one in your instance's IAM role:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1409309287000",
"Effect": "Allow",
"Action": [
"ec2:DescribeTags"
],
"Resource": [
"*"
]
}
]
}
The instance's EC2 tags will available in /etc/ec2-tags in this format:
FOO="Bar"
Name="EC2 tags with cloud-init"
You can include the file as-is in a shell script using . /etc/ec2-tags, for example:
#!/bin/sh
. /etc/ec2-tags
echo $Name
The tags are downloaded during instance initialization, so they will not reflect subsequent changes.
The script and IAM policy are based on itaifrenkel's answer.
If you are not in the default availability zone the results from overthink would return empty.
ec2-describe-tags \
--region \
$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone | sed -e "s/.$//") \
--filter \
resource-id=$(curl --silent http://169.254.169.254/latest/meta-data/instance-id)
If you want to add a filter to get a specific tag (elasticbeanstalk:environment-name in my case) then you can do this.
ec2-describe-tags \
--region \
$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone | sed -e "s/.$//") \
--filter \
resource-id=$(curl --silent http://169.254.169.254/latest/meta-data/instance-id) \
--filter \
key=elasticbeanstalk:environment-name | cut -f5
And to get only the value for the tag that I filtered on, we pipe to cut and get the fifth field.
ec2-describe-tags \
--region \
$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone | sed -e "s/.$//") \
--filter \
resource-id=$(curl --silent http://169.254.169.254/latest/meta-data/instance-id) \
--filter \
key=elasticbeanstalk:environment-name | cut -f5
You can alternatively use the describe-instances cli call rather than describe-tags:
This example shows how to get the value of tag 'my-tag-name' for the instance:
aws ec2 describe-instances \
--instance-id $(curl -s http://169.254.169.254/latest/meta-data/instance-id) \
--query "Reservations[*].Instances[*].Tags[?Key=='my-tag-name'].Value" \
--region ap-southeast-2 --output text
Change the region to suit your local circumstances. This may be useful where your instance has the describe-instances privilege but not describe-tags in the instance profile policy
I have pieced together the following that is hopefully simpler and cleaner than some of the existing answers and uses only the AWS CLI and no additional tools.
This code example shows how to get the value of tag 'myTag' for the current EC2 instance:
Using describe-tags:
export AWS_DEFAULT_REGION=us-east-1
instance_id=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
aws ec2 describe-tags \
--filters "Name=resource-id,Values=$instance_id" 'Name=key,Values=myTag' \
--query 'Tags[].Value' --output text
Or, alternatively, using describe-instances:
aws ec2 describe-instances --instance-id $instance_id \
--query 'Reservations[].Instances[].Tags[?Key==`myTag`].Value' --output text
For Python:
from boto import utils, ec2
from os import environ
# import keys from os.env or use default (not secure)
aws_access_key_id = environ.get('AWS_ACCESS_KEY_ID', failobj='XXXXXXXXXXX')
aws_secret_access_key = environ.get('AWS_SECRET_ACCESS_KEY', failobj='XXXXXXXXXXXXXXXXXXXXX')
#load metadata , if = {} we are on localhost
# http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AESDG-chapter-instancedata.html
instance_metadata = utils.get_instance_metadata(timeout=0.5, num_retries=1)
region = instance_metadata['placement']['availability-zone'][:-1]
instance_id = instance_metadata['instance-id']
conn = ec2.connect_to_region(region, aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key)
# get tag status for our instance_id using filters
# http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-DescribeTags.html
tags = conn.get_all_tags(filters={'resource-id': instance_id, 'key': 'status'})
if tags:
instance_status = tags[0].value
else:
instance_status = None
logging.error('no status tag for '+region+' '+instance_id)
A variation on some of the answers above but this is how I got the value of a specific tag from the user-data script on an instance
REGION=$(curl http://instance-data/latest/meta-data/placement/availability-zone | sed 's/.$//')
INSTANCE_ID=$(curl -s http://instance-data/latest/meta-data/instance-id)
TAG_VALUE=$(aws ec2 describe-tags --region $REGION --filters "Name=resource-id,Values=$INSTANCE_ID" "Name=key,Values='<TAG_NAME_HERE>'" | jq -r '.Tags[].Value')
Starting January 2022, this should be also available directly via ec2 metadata api (if enabled).
curl http://169.254.169.254/latest/meta-data/tags/instance
https://aws.amazon.com/about-aws/whats-new/2022/01/instance-tags-amazon-ec2-instance-metadata-service/
Using the AWS 'user data' and 'meta data' APIs its possible to write a script which wraps puppet to start a puppet run with a custom cert name.
First start an aws instance with custom user data: 'role:webserver'
#!/bin/bash
# Find the name from the user data passed in on instance creation
USER=$(curl -s "http://169.254.169.254/latest/user-data")
IFS=':' read -ra UDATA <<< "$USER"
# Find the instance ID from the meta data api
ID=$(curl -s "http://169.254.169.254/latest/meta-data/instance-id")
CERTNAME=${UDATA[1]}.$ID.aws
echo "Running Puppet for certname: " $CERTNAME
puppet agent -t --certname=$CERTNAME
This calls puppet with a certname like 'webserver.i-hfg453.aws' you can then create a node manifest called 'webserver' and puppets 'fuzzy node matching' will mean it is used to provision all webservers.
This example assumes you build on a base image with puppet installed etc.
Benefits:
1) You don't have to pass round your credentials
2) You can be as granular as you like with the role configs.
Jq + ec2metadata makes it a little nicer. I'm using cf and have access to the region. Otherwise you can grab it in bash.
aws ec2 describe-tags --region $REGION \
--filters "Name=resource-id,Values=`ec2metadata --instance-id`" | jq --raw-output \
'.Tags[] | select(.Key=="TAG_NAME") | .Value'
No jq.
aws ec2 describe-tags --region us-west-2 \
--filters "Name=resource-id,Values=`ec2-metadata --instance-id | cut -d " " -f 2`" \
--query 'Tags[?Key==`Name`].Value' \
--output text
Download and run a standalone executable to do that.
Sometimes one cannot install awscli that depends on python. docker might be out of the picture too.
Here is my implementation in golang:
https://github.com/hmalphettes/go-ec2-describe-tags
The Metadata tool seems to no longer be available, but that was an unnecessary dependency anyway.
Follow the AWS documentation to have the instance's profile grant it the "ec2:DescribeTags" action in a policy, restricting the target resources as much as you wish. (If you need a profile for another reason then you'll need to merge policies into a new profile-linked role).
Then:
aws --region $(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone | sed -e 's/.$//') ec2 describe-tags --filters Name=resource-type,Values=instance Name=resource-id,Values=$(curl http://169.254.169.254/latest/meta-data/instance-id) Name=key,Values=Name |
perl -nwe 'print "$1\n" if /"Value": "([^"]+)/;'
Well there are lots of good answers here but none quite worked for me exactly out of the box, I think the CLI has been updated since some of them and I do like using the CLI. The following single command works out of the box for me in 2021 (as long as the instance's IAM role is allowed to describe-tags).
aws ec2 describe-tags \
--region "$(ec2-metadata -z | cut -d' ' -f2 | sed 's/.$//')" \
--filters "Name=resource-id,Values=$(ec2-metadata --instance-id | cut -d " " -f 2)" \
--query 'Tags[?Key==`Name`].Value' \
--output text
AWS has recently announced support for instance tags in Instance Metadata Service: https://aws.amazon.com/about-aws/whats-new/2022/01/instance-tags-amazon-ec2-instance-metadata-service/
If you have have the tag metadata option enabled for an instance, you can simply do
$ TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 900"`
$ curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/tags/instance
It is possible to get Instance tags from within the instance via metadata.
First, allow access to tags in instance metadata as explained here
Then, run this command for IMDSv1, Refer
curl http://169.254.169.254/latest/meta-data/tags/instance/Name
or this command for IMDSv2
TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/tags/instance
Install AWS CLI:
curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
sudo apt-get install unzip
unzip awscli-bundle.zip
sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
Get the tags for the current instance:
aws ec2 describe-tags --filters "Name=resource-id,Values=`ec2metadata --instance-id`"
Outputs:
{
"Tags": [
{
"ResourceType": "instance",
"ResourceId": "i-6a7e559d",
"Value": "Webserver",
"Key": "Name"
}
]
}
Use a bit of perl to extract the tags:
aws ec2 describe-tags --filters \
"Name=resource-id,Values=`ec2metadata --instance-id`" | \
perl -ne 'print "$1\n" if /\"Value\": \"(.*?)\"/'
Returns:
Webserver
For those crazy enough to use Fish shell on EC2, here's a handy snippet for your /home/ec2-user/.config/fish/config.fish. The hostdata command now will list all your tags as well as the public IP and hostname.
set -x INSTANCE_ID (wget -qO- http://instance-data/latest/meta-data/instance-id)
set -x REGION (wget -qO- http://instance-data/latest/meta-data/placement/availability-zone | sed 's/.$//')
function hostdata
aws ec2 describe-tags --region $REGION --filter "Name=resource-id,Values=$INSTANCE_ID" --output=text | sed -r 's/TAGS\t(.*)\t.*\t.*\t(.*)/\1="\2"/'
ec2-metadata | grep public-hostname
ec2-metadata | grep public-ipv4
end