failed to get ec2 instance public ip on instance startup (windows) - amazon-web-services

I tried the below powershell script on startup (passed it as a user data):
<powershell>
$instancePublicIp = (wget http://169.254.169.254/latest/meta-data/public-ipv4).Content
while ($instancePublicIp.length -eq 0)
{
Start-Sleep -s 40
$instancePublicIp = (wget http://169.254.169.254/latest/meta-data/public-ipv4).Content
Add-Content c:\debug.txt "public ip : $($instancePublicIp) - "
}
</powershell>
I verified the script is running because debug.txt exists on C: drive. but the public ip comes back empty.
I also tried sleeping for 40 seconds to allow the instance to start and the network adapter is connected with no luck.
the same script above runs and gives back proper ip address when running it manually though.
Any thoughts?

Related

EC2 Instance Metadata - Filter Private IPs based on Mac or Interface Index

I am writing a shell script that will run in an EC2 instance with 2 interfaces (Mgmt and Service NICs). The need to be able to grab the IP of the secondary interface (index1) via the metadata; however, I am only able to do it by filtering the mac address. The problem is that the mac addresses are never in the same order, so depending on the exact order of the Mac addresses, it may return the IP of the primary interface (Index0). Here is how I am doing my filter
curl --silent http://169.254.169.254/latest/meta-data/network/interfaces/macs/ > macs
MAC=$(awk '{if(NR==2) print $0}' macs )
SMNET_IP=$(curl --silent http://169.254.169.254/latest/meta-
data/network/interfaces/macs/$MAC/local-ipv4s)
echo {$SMNET_IP}
How can I query both interfaces private IP addresses via the metadata based on either the interface index or some other way?
You can query each network interface's device number, using the device-number fragment under the MAC. From there, it's just a matter of getting the IP address for the first network interface, the one with a number of 0.
#!/bin/bash
URL_BASE="http://169.254.169.254/latest/meta-data/network/interfaces/macs"
for MAC in $(curl -s ${URL_BASE}/); do
if [[ "$(curl -s ${URL_BASE}/${MAC}device-number)" == "0" ]]; then
LOCAL_IP=$(curl -s ${URL_BASE}/${MAC}local-ipv4s)
echo "Found device 0 with MAC: ${MAC}, IP of ${LOCAL_IP}"
fi
done

Is there a way to have an AWS spot instance register with route53 on boot?

We have some test, dev and ci servers that we have setup as long running persistent spot instances mapped to specific domains using route53. This works great - we get the savings, we can allocate these without too much concern about cost but every now and then we loose the instance due to availability. When they come back - they come back with different IP addresses which breaks the route.
Is there a good way to have these instances automatically remap to the new IP address when they come back online (usually within a minute or two)?
Caution: I'm not convinced this approach is working after all. While I can confirm everything runs as I expected it to - the new routes didn't get setup correctly after these machines were assigned new spot instances. I'm not sure if this is because some service is not started by the time this script runs or if Amazon specifically prohibits this behavior. I'd be curious to hear what others have found.
--
N.B. The right answer here might well be using elastic IP addresses which as I understand allow you to have a single static IP address avoiding this issue altogether. I've not done the cost calculation on this but it might well be cheaper than the solution offered below.
What we ended up coming up with is a script that uses the AWS instance metadata and cli to make a route53 call upon reboot. This did NOT work on our old Ubuntu 14.04 instances but appears to on our newer Ubuntu 20.04 instances.
Here's how it works:
We built a little script called setupRoute53.sh that knows how to make a single call to route53.
We added a job to cron to run this on each reboot.
(Bonus) we also created an ansible script to add additional lines to the crontab for each virtual host we're running locally - we reverse proxy multiple services using nginx.
We're currently running this within the ubuntu user - the crontab looks like this:
# m h dom mon dow command
#reboot /home/ubuntu/setupRoute53.sh example.com test.example.com
And setupRoute53.sh looks like this:
#!/bin/sh
# setupRoute53.sh
export ROOT_DOMAIN="$1"
export ROUTE53_HOSTNAME="$2"
export IP="$3"
if [ -z "$1" ]; then
echo "Usage: $1 <route53 domain> <this hostname> ";
echo "";
echo "Example: $1 tokenoftrust.com test.tokenoftrust.com";
echo;
exit;
fi
if [ -z "$3" ]; then
echo "IP not given...trying EC2 metadata...";
IP=$( curl http://169.254.169.254/latest/meta-data/public-ipv4 )
fi
echo "Updating $ROUTE53_HOSTNAME to : $IP"
HOSTED_ZONE_ID=$( aws route53 list-hosted-zones-by-name | grep -B 1 -e "$ROOT_DOMAIN" | sed 's/.*hostedzone\/\([A-Za-z0-9]*\)\".*/\1/' | head -n 1 )
echo "Hosted zone being modified: $HOSTED_ZONE_ID"
INPUT_JSON=$(echo '{
"Comment": "Update the A record set",
"Changes": [
{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "HOSTNAME",
"Type": "A",
"TTL": 60,
"ResourceRecords": [
{
"Value": "127.0.0.1"
}
]
}
}
]
}' | sed "s/127\.0\.0\.1/$IP/" | sed "s/HOSTNAME/$ROUTE53_HOSTNAME/" )
# http://docs.aws.amazon.com/cli/latest/reference/route53/change-resource-record-sets.html
# We want to use the string variable command so put the file contents (batch-changes file) in the following JSON
INPUT_JSON="{ \"ChangeBatch\": $INPUT_JSON }"
aws route53 change-resource-record-sets --hosted-zone-id "$HOSTED_ZONE_ID" --cli-input-json "$INPUT_JSON"
exit 0;

Ansible "connection closed" using AWS .pem file with Dynamic Inventory

Ansible is failing with a "connection closed" error when using an AWS .pem file. Yet I can connect OK via ssh using the same file. Does anyone know what the problem might be?
I have a couple of servers in AWS set-up with dynamic inventory (ignore the one without a public IP I know that won't work):
$ ansible-inventory -i aws_ec2.yaml --graph
#all:
|--#aws_ec2:
| |--ec2-34-244-225-4.eu-west-1.compute.amazonaws.com
| |--ip-10-128-1-129.eu-west-1.compute.internal
|--#ungrouped:
I can happily connect to the server with a public ip as follows:
$ ssh -i $HOME/.ssh/my-dev.pem openvpnas#ec2-34-244-225-4.eu-west-1.compute.amazonaws.com
Welcome to OpenVPN Access Server Appliance 2.8.5
System information as of Wed Aug 5 18:17:25 UTC 2020
etc.
But when I try to use Ansible with the same .pem file it fails with a connection closed error, and I don't know why.
$ ansible all -i aws_ec2.yaml -a ‘uptime’ \
--private-key=$HOME/.ssh/my-dev.pem \
--become-user=openvpnas
.
.
.
ec2-34-244-225-4.eu-west-1.compute.amazonaws.com | UNREACHABLE! => {
"changed": false,
"msg": "Failed to connect to the host via ssh: Connection closed by 34.244.225.4 port 22",
"unreachable": true
}
Is this a .pem file funny, or something to do with dynamic inventory. Any ideas?
--become-user is to run operation as that user. Try with --user openvpnas in the command or you can set it ansible.cfg file as remote_user = openvpnas

Running expect script on EC2 hangs, but runs successfully when manually invoked

I'm writing an expect script to start an SSH tunnel.
It gets run on EC2 when the instance starts, as part of the deployment which creates the script from a .ebextensions config file.
When the script is run, it always gets stuck at this point:
Enter passphrase for key '/home/ec2-user/id_data_app_rsa':
If I run the same script manually on the server it succeeds and i can see the tunnel process running.
ps aux | grep ssh
root 19046 0.0 0.0 73660 1068 ? Ss 16:58 0:00 ssh -i /home/ec2-user/id_data_app_rsa -p222 -vfN -L 3306:X.X.X.X:3306 root#X.X.X.X
I can verify that the script is reading the SSH_PASSPHRASE correctly by printing it to the console.
set password $::env(SSH_PASSPHRASE)
send_user "retrieved env variable : $password "
This is the debug output I get from the EC2 logs:
Enter passphrase for key '/home/ec2-user/id_data_app_rsa':
interact: received eof from spawn_id exp0
I'm baffled as to why it's getting no further here when the EC2 deployer runs, but it continues normally when run manually.
This is the script in .ebextensions, the script itself starts at #!/usr/bin/expect:
files:
"/scripts/createTunnel.sh" :
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/expect
exp_internal 1
set timeout 60
# set variables
set password $::env(SSH_PASSPHRASE)
send_user "retrieved env variable : $password "
spawn -ignore HUP ssh -i /home/ec2-user/id_data_app_rsa -p222 -vfN -L 3306:X.X.X.X:3306 root#X.X.X.X
expect {
"(yes/no)?" { send "yes\n" }
-re "(.*)assphrase" { sleep 1; send -- "$password\n" }
-re "(.*)data_app_rsa" { sleep 1; send -- "$password\n" }
-re "(.*)assword:" { sleep 1; send -- "$password\n" }
timeout { send_user "un-able to login: timeout\n"; return }
"denied" { send_user "\nFatal Error: denied \n"}
eof { send_user "Closed\n" ; return }
}
interact
We finally resolved this. There were two things that seemed to be at issue:
Changing the final interact to expect eof.
Trimming down the
expect pattern matching as much as possible.
We noticed in testing that expect seemed to be matching falsely, sending a password, for example, when it should have been sending a 'yes' matching on the 'yes/no' prompt.
This is the final script we ended up with in case it's useful to anyone else:
#!/usr/bin/expect
exp_internal 1
set timeout 60
# set variables
set password $::env(SSH_TUNNEL_PASSPHRASE)
spawn -ignore HUP ssh -i /home/ec2-user/id_data_rsa -p222 -vfN -L 3306:X.X.X.X:3306 root#X.X.X.X
expect {
"(yes/no)?" { send "yes\r" }
"Enter passphrase" { sleep 2; send -- "$password\r"; sleep 2; exit }
}
expect eof
Your problem is here:
set password $::env(SSH_PASSPHRASE)
and the way shell works with environment variables. When the script is invoked, you assume your environment variables are set. Depending on how the script is invoked, $::env(SSH_PASSPHRASE) may not be set, resulting in the variable to be null/blank. When init scripts (or cloud-init) are run, they are not run with the environment of a login shell. So you should not assume that .profile or /etc/profile environment variables are set, but rather source or set them explicitly.
A possible solution may be
. ~ec2-user/.profile /path/to/above.script

Powershell and Psession

I have a powershell script that pulls in a server list with the following headers. computername,optype,opname. I then loop through and if I have a format of server,service,service name then start/stop the service. If I have a format of server,script,script_path_name then I am trying to use an invoke-command to run that script. However I need somehow establish a remote session ... Here is the error txt that I get.
[c1399] Connecting to remote server c1399 failed with the following error message : The client cannot connect to the destination specified in the
request. Verify that the service on the destination is running and is accepting requests. Consult the logs and documentation for the WS-Management
service running on the destination, most commonly IIS or WinRM. If the destination is the WinRM service, run the following command on the destination to
analyze and configure the WinRM service: "winrm quickconfig". For more information, see the about_Remote_Troubleshooting Help topic.
+ CategoryInfo : OpenError: (c1399:String) [], PSRemotingTransportException
+ FullyQualifiedErrorId : CannotConnect,PSSessionStateBroken
Here is the current code I am using...
elseif($computer.optype -eq "script")
{
write-host running script $computer.opname on $computer.computername
$Status = Invoke-Command -ComputerName $computer.computername -ScriptBlock { . $computer.opname }
Write-Output "Script execution status for $($computer.name) is $Status" >> C:\Scripts\RMLog.txt
}
Enabled power shell remoting on the server.
To enable PowerShell Remoting, run the following command,
Enable-PSRemoting -Force
Please refer below link for more details
http://www.howtogeek.com/117192/how-to-run-powershell-commands-on-remote-computers/
http://powershelltutorial.net/V2/Powershell-With-TFS
this will solve your issue..