I am attempting to create my own bastion host for a VPC, and created an auto-scaling group with min/max instances of 1. In my launch configurations, I specify the following for the ec2 user data:
#!
INSTANCE_ID=`/usr/bin/curl -s http://169.254.169.254/latest/meta-data/instance-id`
aws ec2 associate-address --instance-id $INSTANCE_ID --allocation-id eipalloc-my-eip-id --allow-reassociation
The goal of this user data is to immediately associate an Elastic IP address with my newly created EC2 instance - I've read from other StackOverflow posts that this must be explicitly done when using ASGs.
However, after the ASG instance spins up and finishes initializing, I still do not see any Elastic IP in my console output for the instance:
I've confirmed that the user data is indeed being used by the instance:
I tried to look inside the system log to see if there were any error messages during the initialization, but I couldn't see anything at first that would suggest that the associate-address command failed (inside /var/log/cloud-init-output).
Edit: Attempt to debug:
However, I then manually associated the Elastic IP with my instance, SSHed, and attempted to run the user data commands above. Interestingly, when I got to the aws ec2 associate-address portion, I ran into
Unable to locate credentials. You can configure credentials by running
"aws configure".
This appears to be at the root of the issue - my AWS profile is not configured. However, I've always been under the impression that a default AWS instance profile is set up for you with access to the AWS CLI when the instance finishes initializing.
Could anyone point me in the direction of why my user data to associate elastic IP addresses might not be executing properly?
Thank you!
Are you running your EC2 instance with the amazon provided AMI?
If so, the profile should be set up indeed.
However, we've noticed the default region is not set in your profile. So when running aws cli commands, you'll either need to give the region using --region <your region> or set the AWS_DEFAULT_REGION environment variable.
We use the following script to provision our servers with an EIP at startup as part of our CloudFormation template:
#!/bin/bash
INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
MAXWAIT=3
ALLOC_ID=<your eip allocation id>
AWS_DEFAULT_REGION=<your region>
# Make sure the EIP is free
echo "Checking if EIP with ALLOC_ID[$ALLOC_ID] is free...."
ISFREE=$(aws ec2 describe-addresses --allocation-ids $ALLOC_ID --query Addresses[].InstanceId --output text)
STARTWAIT=$(date +%s)
while [ ! -z "$ISFREE" ]; do
if [ "$(($(date +%s) - $STARTWAIT))" -gt $MAXWAIT ]; then
echo "WARNING: We waited 30 seconds, we're forcing it now."
ISFREE=""
else
echo "Waiting for EIP with ALLOC_ID[$ALLOC_ID] to become free...."
sleep 3
ISFREE=$(aws ec2 describe-addresses --allocation-ids $ALLOC_ID --query Addresses[].InstanceId --output text)
fi
done
# Now we can associate the address
echo Running: aws ec2 associate-address --instance-id $INSTANCE_ID --allocation-id $ALLOC_ID --allow-reassociation}
aws ec2 associate-address --instance-id $INSTANCE_ID --allocation-id $ALLOC_ID --allow-reassociation}
We've added the part where we check if the EIP is free as this is used inside an autoscaling group and we've noticed sometimes the EIP is still in use even though the old instance is already terminated and the new instance is at the point of running the userdata script.
It looks the instance profile attached to this EC2 instance does not have permission to perform the above task.
Referring to https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-policies-ec2-console.html#ex-eip,
Can you make sure your Instance profile has the following action allowed?
ec2:AllocateAddress: To allocate an Elastic IP address.
ec2:ReleaseAddress: To release an Elastic IP address.
ec2:AssociateAddress: To associate an Elastic IP address with an instance or a network interface.
ec2:DescribeNetworkInterfaces and ec2:DescribeInstances: To work with the Associate address screen. The screen displays the available instances or network interfaces to which you can associate an Elastic IP address.
ec2:DisassociateAddress: To disassociate an Elastic IP address from an instance or a network interface.
Sample policy would look like:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:DescribeAddresses",
"ec2:AllocateAddress",
"ec2:DescribeInstances",
"ec2:AssociateAddress"
],
"Resource": "*"
}
]
}
Hope this helps.
A User Data script must start with #! to be recognized as a script. For example:
#!
INSTANCE_ID=`/usr/bin/curl -s http://169.254.169.254/latest/meta-data/instance-id`
aws ec2 associate-address --instance-id $INSTANCE_ID --allocation-id eipalloc-xxx --allow-reassociation
You can view the resulting log in: /var/log/cloud-init-output.log
Related
I have one instance in ASG, I need to assign an elastic ip that instance. Now when the instance health check fails, the newly launched instance should have the same elastic IP. The IAM role and everything is in the correct order.
INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
MAXWAIT=3
ALLOC_ID=${IPAddresses}
echo "Checking if EIP with ALLOC_ID[$ALLOC_ID] is free...."
ISFREE=$(aws ec2 describe-addresses --allocation-ids $ALLOC_ID --query Addresses[].InstanceId --output text --region ${AWS::Region})
STARTWAIT=$(date +%s)
while [ ! -z "$ISFREE" ]; do
if [ "$(($(date +%s) - $STARTWAIT))" -gt $MAXWAIT ]; then
echo "WARNING: We waited 30 seconds, we're forcing it now."
ISFREE=""
else
echo "Waiting for EIP with ALLOC_ID[$ALLOC_ID] to become free...."
sleep 3
ISFREE=$(aws ec2 describe-addresses --allocation-ids $ALLOC_ID --query Addresses[].InstanceId --output text --region ${AWS::Region})
fi
done
echo Running: aws ec2 associate-address --instance-id $INSTANCE_ID --allocation-id $ALLOC_ID --allow-reassociation --region ${AWS::Region}}
aws ec2 associate-address --instance-id $INSTANCE_ID --allocation-id $ALLOC_ID --allow-reassociation --region ${AWS::Region}
yum install jq -y
Not sure how to take that IP from the resource itself and pass it as a user data in Launch configuration.
In the CFN, it would look similar to the following:
Resources:
MyEIP:
Type: AWS::EC2::EIP
Properties: {}
MyLaunchTemplate:
Type: AWS::EC2::LaunchTemplate
Properties:
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
EIP_IP=${MyEIP}
echo ${!EIP_IP}
# use aws cli to attach EIP_IP to the instance
Instance role would be required as well with permissions to attach the EIP.
From docs about !Ref which will be used when EIP_IP=${MyEIP}:
When you pass the logical ID of this resource to the intrinsic Ref function, Ref returns the Elastic IP address.
With EC2 & Auto scaling, You need using user data in EC2 to Auto Attach Elastic IP to EC2 Instance For Auto scaling
#!/bin/bash
aws configure set aws_access_key_id "XYZ..."
aws configure set aws_secret_access_key "ABC..."
aws configure set region "ap-..."
aws ec2 associate-address --instance-id "$(curl -X GET "http://169.254.169.254/latest/meta-data/instance-id")" --public-ip your_elastic_IP
Note: you should create new user & IAM have only permission associate-address to create/get aws key
Hope it be help you :)
How do I find the name of "this" Lightsail instance. "This" being the instance that the aws command is being executed. My below script isn't working, since I thought Lightsail is just another EC2 instance.
#!/bin/bash
InstanceId=`curl -s http://169.254.169.254/latest/meta-data/instance-id`
echo $InstanceId
Region=`aws configure get region`
echo $Region
InstanceName=$(aws ec2 describe-tags --region $Region --filters "Name=resource-id,Values=$InstanceId" "Name=key,Values=Name" --output text | cut -f5)
echo $InstanceName
The name of a Lightsail instance can be obtained with:
aws lightsail get-instances --query instances[].name
In my case, this was the auto-assigned name when I started the instance via the Lightsail management console. I couldn't see a way to change the name during launch.
Interestingly, I could not find a way to use the AWS CLI to list tags associated with a Lightsail instance. For example, I could not retrieve the Name tag that I manually added to an instance, and which appears in the Lightsail console.
Update:
After discussion in comments, I got this working:
aws lightsail get-instances --query "instances[?contains(supportCode,'`curl -s http://169.254.169.254/latest/meta-data/instance-id`')].name" --output text
I frequently need to start AWS EC2 instances to work with from the command line to work with over SSH, and would like to write a short script to do this, but I'm stuck at the most basic steps.
For example, I can get started with
aws ec2 start-instances --instance-ids i-84Sd8jdf
and would like to continue by grabbing the IP address assigned to the instance and using it as an environment variable or script variable to preform subsequent operations such as
ssh ubuntu#<theIP>
or
scp ubuntu#<theIP>:~/soruce_stuff/* ~/dest_folder/
but I can't figure out how to get the IP address from the start-instances command, or from any of the JSON emitted by other commands.
How do I script starting of EC2 instances, wearing for the IP to be assigned, and capturing the assigned IP address for subsequent use?
An example (based on this excellent answer) in bash where the instance is started, the script sleeps (specified in seconds), and the public ip address is saved to a local variable:
aws ec2 start-instances --instance-ids i-aaaa1111
sleep 10
ec2Address=$(aws ec2 describe-instances --instance-ids i-aaaa1111 --query "Reservations[*].Instances[*].PublicIpAddress" --output=text)
To verify:
echo $ec2Address
11.1.111.111
Amazon offers wait condition to wait for the instance to be ready before you can perform other tasks on it. Here is an example might help you
aws ec2 start-instances --instance-ids $instance_id
aws ec2 wait instance-running --instance-ids $instance_id
aws ec2 describe-instances --instance-ids $instance_id --output text|grep ASSOCIATION |awk '{print $3}'|head -1
I want to implement an automatic way of updating Route53 DNS based on EC2 tags of the instances.
Imagine that I can have a tag name named dns_name which is supposed to document the expected dns name of the machine.
If I shutdown the machine, upgrade it and start it again, I do want to get the DNS updated without having to change anything.
I don't want to install a DNS update service on the machine as this would require me to put AWS credentials on all the VMs, and to deploy this to them.
It seems much easier to use AWS EC2 tags to declare the expected names. A cron job could easily check for changes and update the DNS.
Did anyone implement this?
A DNS update service on the machine wouldn't require AWS credentials. You would preferably use AWS IAM instance profiles instead. I've used this method to run a script on server boot to update Route53 records with the latest IP address.
If you want to schedule a cron on a single server that handles updating the Route53 records for all servers, based on instance tags, then that is certainly possible. It would only be a few lines of code.
I would highly recommend using Elastic IP's. This is the exact purpose of them. For this implementation, go to EC2 => Network & Security => Elastic IP => Allocate New Address => Actions => Associate Address => (select the instance tag name you're looking for)
Copy the EIP to your clipboard then go to Route53 => Create Hosted Zone => go thought those steps => Go to Record Sets => Create Record Set => For Name: you can have your www. or not => Type: A - IPv4 => Alias: NO => TTL: Whatever you'd like => Value: your EIP address => Save Record Set
If you have an Elastic IP associated to your instance, you can reboot, swap, stop, update, change, shape shift, whatever your want! EIP's wont go anywhere.
Update:
You can make all this automated via the Route53 API.
These are the steps for doing it because you have to know the process in order to do it with the API calls.
I was trying to do exactly what I think your attempting to do.
What I've found is this script from this link which is a good starting point.
https://medium.com/#dbclin/a-bash-script-to-update-an-existing-aws-security-group-with-my-new-public-ip-address-d0c965d67f28
#!/bin/bash
# Extract information about the Instance
INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id/)
AZ=$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone/)
MY_IP=$(curl -s http://169.254.169.254/latest/meta-data/public-ipv4/)
# Extract tags associated with instance
ZONE_TAG=$(aws ec2 describe-tags --region ${AZ::-1} --filters "Name=resource-id,Values=${INSTANCE_ID}" --query 'Tags[?Key==`AUTO_DNS_ZONE`].Value' --output text)
NAME_TAG=$(aws ec2 describe-tags --region ${AZ::-1} --filters "Name=resource-id,Values=${INSTANCE_ID}" --query 'Tags[?Key==`AUTO_DNS_NAME`].Value' --output text)
# Update Route 53 Record Set based on the Name tag to the current Public IP address of the Instance
aws route53 change-resource-record-sets --hosted-zone-id $ZONE_TAG --change-batch '{"Changes":[{"Action":"UPSERT","ResourceRecordSet":{"Name":"'$NAME_TAG'","Type":"A","TTL":300,"ResourceRecords":[{"Value":"'$MY_IP'"}]}}]}'
This has details of the solution suggested by Mark B. Elastic IP doesn't work if you don't have any available and Amazon allows you only 5.
I used user14441273's pasted script as a starting point. The provided link for that is dead (redirects you to an article nothing to do with Route53), and the post gives no clue how to set up the required tags ZONE_TAG or NAME_TAG.
Regardless how you do it, you will need to allow the actions to the instance profile role.
Here's the productonlized working script. I get the instance name from local file system, which is much faster and more reliable than using a network service. Change the domain to the domain you want to use:
#!/usr/bin/env bash
set -e
shopt -s xpg_echo
set +u
DNS_DOMAIN=yourdomain.com
TTL_S=30 # You want this low or you will have to wait up to this long for changes to propagate
PATH="$PATH:/usr/local/bin"
# Extract information about the Instance
INSTNAME=$(ls -ld /var/lib/cloud/instance | sed 's#.*/##;')
[ -n "$INSTNAME" ] || {
echo 'Failed to determine Instance name' 1>&2
false
}
[ -f /tmp/date.txt ] && {
echo "date present" 1>&2
false
}
AWS_REGION=$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone | sed 's/.$//')
[ -n "$AWS_REGION" ] || {
echo 'Failed to determine AWS Region' 1>&2
false
}
HOSTNAME=$(aws ec2 describe-tags --region $AWS_REGION --filters Name=resource-type,Values=instance Name=resource-id,Values=$INSTNAME Name=key,Values=Name --query 'Tags[0].Value' --output text)
[ -n "$HOSTNAME" ] || {
echo 'Failed to determine DNS Hostname to update' 1>&2
false
}
MY_IP=$(curl -s http://169.254.169.254/latest/meta-data/public-ipv4/)
[ -n "$MY_IP" ] || {
echo 'Failed to determine my public IP address' 1>&2
false
}
ZONE_ID=$(aws route53 --region $AWS_REGION list-hosted-zones-by-name --region $AWS_REGION --dns-name $DNS_DOMAIN --query 'HostedZones[0].Id' --output text)
[ -n "$ZONE_ID" ] || {
echo 'Failed to determine my Route53 Zone ID' 1>&2
false
}
aws route53 change-resource-record-sets --hosted-zone-id $ZONE_ID --region $AWS_REGION --change-batch '{"Changes":[{"Action":"UPSERT","ResourceRecordSet":{"Name":"'${HOSTNAME}.$DNS_DOMAIN'","Type":"A","TTL":'$TTL_S',"ResourceRecords":[{"Value":"'$MY_IP'"}]}}]}'
enter code here
Assuming that you're running a late-model Linux variant, here's a working service script. N.b. I have it configured to run the shell script from /usr/local/sbin which I have NFS mounted:
[Unit]
Description=Update Route53 DNS Service with instance's current public IP addr
RequiresMountsFor=/usr/local
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/updateRoute53.bash
RemainAfterExit=no
StandardOutput=truncate:/var/log/updateRoute53.log
[Install]
WantedBy=multi-user.target
Is there any way to find out status of AWS EC2 instances, which are running on various different regions, from one EC2 instance which is present in any one of region by using EC2 API tool ?
How this is possible ?
I got the answer :-
ec2-describe-instances instance-ID --region region
Example :-
ec2-describe-instances i-f82d5ca0 --region eu-west-1
Where instance ID is EC2 instance ID which is located in region eu-west-1
Thats all .
Or in the new unified AWS CLI, this is slightly different:
aws ec2 describe-instances --instance-id i-f82d5ca0
You can also change the --output into json, text, or a table