Bash for loop prints unexpected output, why is that? - regex

I wrote a script which queries a dns zone on AWS Route53 and returns IP's of specific servers.
The aim is to run this script after starting a CloudFormation stack and that the script will update my local ~/.ssh/config file with the updated servers IP's.
I want to "catch" the line in the config which starts with HostName but related to the hostname of the server I would like to edit and so I've ran the following command:
$ grep -A2 dev-api2.company-private ~/.ssh/config
Host dev-api2.company-private
User ec2-user
HostName 5.5.5.5
Next, I've ran:
$ for line in $(grep -A2 dev-api2.company-private ~/.ssh/config); do echo $line ; done
Host
dev-api2.company-private
User
ec2-user
HostName
5.5.5.5
For some reason the output is printed not as I expected, it broke each word to a separate line, but why does it happen?
Edit #1:
My idea was to do something like that:
for line in $(grep -A2 dev-api2.company-private ~/.ssh/config)
do
if [[ $line == "^HostName.*" ]]
then
sed -i 's!$line!HostName 1.1.1.1!g' ~/.ssh/config
fi
done
Edit #2:
Ok, so the relevant portion of the ~/.ssh/config file looks like that:
Host as-bastion
User ec2-user
HostName ec2-34.us-west-2.compute.amazonaws.com
IdentityFile ~/.ssh/company/bastion.pem
Host dev-api1.company-private
User ec2-user
HostName 172.51.24.171
IdentityFile ~/.ssh/company/company.pem
ProxyCommand ssh ec2-user#as-bastion -W %h:%p
Host dev-api2.company-private
User ec2-user
HostName 172.51.25.248
IdentityFile ~/.ssh/company/company.pem
ProxyCommand ssh ec2-user#as-bastion -W %h:%p
The file contains many blocks like the ones above, but I want to sed only the "HostName" lines of blocks which contain the string I've used in the grep (dev-api2.company-private).
#anubhava: The code in your answer changed lines containing the string "HostName" even in blocks which are not related to "dev-api2.company-private".
How can it be done?

You should be using this while read loop using process substitution instead of a for loop using command substitution:
while IFS= read -r line; do
echo "$line"
done < <(grep -A2 dev-api2.company-private ~/.ssh/config)
Based on your comments I suggest it is better to use awk like this:
awk -v ORS='\n\n' -v RS= -v api='^Host dev-api2' -v ip='1.1.1.1' '
$0 ~ api "\\.company-private$" {sub(/\nHostName [^\n]+/, "\nHostName " ip)} 1' ~/.ssh/config
To save changes use this awk command:
awk -v ORS='\n\n' -v RS= -v api='^Host dev-api2' -v ip='1.1.1.1' '
$0 ~ api "\\.company-private$" {sub(/\nHostName [^\n]+/, "\nHostName " ip)} 1
' ~/.ssh/config > $$.tmp && mv $$.tmp ~/.ssh/config
Here is one liner sed to do the same job:
sed -i.bak '/dev-api2\.company-private/{N;p;N;s/.*/HostName 1.1.1.1/;}' ~/.ssh/config

Related

bash sript to check script file extension and adding an extension

I have written the following Bash script. Its role is to check its own name, and in case of nonexistent extension , to amend ".sh" with sed. Still I have error "missing target file..."
#!/bin/bash
FILE_NAME="$0"
EXTENSION=".sh"
FILE_NAME_MOD="$FILE_NAME$EXTENSION"
if [[ "$0" != "FILE_NAME_MOD" ]]; then
echo mv -v "$FILENAME" "$FILENAME$EXTENSION"
cp "$0" | sed 's/\([^.sh]\)$/\1.sh/g' $0
fi
#!/bin/bash
file="$0"
extension=".sh"
if [ $(echo -n $file | tail -c 3) != $extension ]; then
mv -v "$file" "$file$extension"
fi
Important stuff:
-n flag suppress the new line at the end, so we can test for 3 chars instead of 4
When in doubt, always use set -x to debug your scripts.
Try this Shellcheck-clean code:
#! /bin/bash -p
file=${BASH_SOURCE[0]}
extension=.sh
[[ $file == *"$extension" ]] || mv -i -- "$file" "$file$extension"
See choosing between $0 and BASH_SOURCE for details of why ${BASH_SOURCE[0]} is better than $0.
See Correct Bash and shell script variable capitalization for details of why file is better than FILE and extension is better than EXTENSION. (In short, ALL_UPPERCASE names are dangerous because there is a danger that they will clash with names that are already used for something else.)
The -i option to mv means that you will be prompted to continue if the new filename is already in use.
See Should I save my scripts with the .sh extension? before adding .sh extensions to your shell programs.
Just for fun, here is a way to do it just with GNU sed:
#!/usr/bin/env bash
sed --silent '
# match FILENAME only if it does not end with ".sh"
/\.sh$/! {
# change "FILENAME" to "mv -v FILENAME FILENAME.sh"
s/.*/mv -v & &.sh/
# execute the command
e
}
' <<<"$0"
You can also make the above script output useful messages:
#!/usr/bin/env bash
sed --silent '
/\.sh$/! {
s/.*/mv -v & &.sh/
e
# exit with code 0 immediately after the change has been made
q0
}
# otherwise exit with code 1
q1
' <<<"$0" && echo 'done' || echo 'no changes were made'

Hard regex with sed

In a script.sh file, I have the following line:
ExecStart=ssh -nN -R 46:192.168.0.1:56 192.168.0.2
I try to replace with sed the second port (56 here) knowing that its value can vary between 1 and 65535.
So I tried that without success :
sed -i -e "s/:.*[[:space/]]/other port number/2g' script.sh
Could you help me solve my regex?
You may use:
sed -i "s/:[0-9]\{1,5\} /:other port number /" script.sh
$ other_port_number="123"
$ echo "ExecStart=ssh -nN -R 46:192.168.0.1:56 192.168.0.2" | sed "s/:[0-9]\{1,5\} /:$other_port_number /"
ExecStart=ssh -nN -R 46:192.168.0.1:123 192.168.0.2

grep regex to output only multiple matches on a single line

I wanted to search for multiple matches in a single line and output only the matched words using grep
alias host='echo -e "Connecting to host 10.10.11.120\n===================";ssh root#10.10.11.120'
alias host1='echo -e "Connecting to host 10.10.11.121\n===================";ssh root#10.10.11.121'
alias host2='echo -e "Connecting to host 10.10.11.122\n===================";ssh root#10.10.11.122'
alias host3='echo -e "Connecting to host 10.10.11.123\n===================";ssh root#10.10.11.123'
I want grep to output only the host name and IP address like
host 10.10.11.120
host1 10.10.11.121
host2 10.10.11.122
host3 10.10.11.123
With grep and pcre, required strings can be extracted
$ grep -oP 'alias \K[^=]+|#\K[0-9.]+' ip.txt
host
10.10.11.120
host1
10.10.11.121
host2
10.10.11.122
host3
10.10.11.123
However, each extracted string would be in separate line, so one can use other commands to join them, for ex:
$ grep -oP 'alias \K[^=]+|#\K[0-9.]+' ip.txt | paste - -
host 10.10.11.120
host1 10.10.11.121
host2 10.10.11.122
host3 10.10.11.123
Or, a single perl command can also be used:
$ perl -pe 's/alias (host\d*).*#([\d.]+).*/$1 $2/' ip.txt
host 10.10.11.120
host1 10.10.11.121
host2 10.10.11.122
host3 10.10.11.123
$ grep -oP 'alias \K[^=]+|#\K[0-9.]+' ip.txt | paste - -
host 10.10.11.120
host1 10.10.11.121
host2 10.10.11.122
host3 10.10.11.123
this is great!
1st solution: With awk it could be much simpler, with your shown samples please try following awk code.
awk -F" |=|'|#" '{print $2,$(NF-1)}' Input_file
Explanation: Simple explanation would be, setting different field separators eg: space, = OR # for lines and printing 2nd and second last fields values for each line.
2nd solution: using field separator and match function of awk. Set space and = as field separators AND use match function to get values needed by OP.
awk -F' |=' 'match($0,/root#([0-9]+\.){3}[0-9]+/){print $2,substr($0,RSTART+5,RLENGTH-5)}' Input_file
3rd solution(Generic one): Adding pure Generic solution here, this will work irrespective of where alias host and root#... are placed in lines, in case anyone requires it.
awk '
match($0,/alias host[^=]*/){
firstVal=substr($0,RSTART+6,RLENGTH-6)
match($0,/root#([0-9]+\.){3}[0-9]+/)
print firstVal,substr($0,RSTART+5,RLENGTH-5)
}
' Input_file
grep matches. It doesn't modify.
Try sed:
sed 's/.*ssh root#/host /' 's/'//' myscript.sh

sed replace does not come up with my desired result

sudo sed -i 's!# dbdir /var/lib/munin!dbdir /var/lib/munin!g' /etc/munin/munin.conf
sudo sed -i 's!localhost 127.0.0.0/8 ::1!all!g' /etc/munin/apache.conf
Why does # dbdir /var/lib/munin does not get replace with dbdir /var/lib/munin
and
why does localhost 127.0.0.0/8 ::1 not get replaced with all?
sudo sed -i 's!# dbdir!dbdir!g' /etc/munin/munin.conf
gives a satisfactory result, only the localhost replacement question remaining.
In my munin.conf there are multiple space between dbdir and /var/lib/munin so unless you have exact info this replace would not work.
You search for only part of the text then replace the line:
awk '/dbdir/ {$0="dbdir /var/lib/munin"}1' /etc/munin/munin.conf > temp ; mv temp /etc/munin/munin.conf
or remove then # in front of the line
awk '/dbdir/ {sub(/^#/,x)}1' /etc/munin/munin.conf
EDIT:
awk '/Allow from local/ {sub(/localhost 127.0.0.0\/8 ::1/,"all")}1' /etc/munin/apache.conf

Using the eval command to create a command alias

For the life of me I cannot get the bash script to execute the alias command to set the hostname of a workstation the alias name to the WOL (Wakeup On Lan) equivalent command. I figure there must be an issue with quoting somewhere that I am missing.
#!/bin/bash
WOLHosts=`nvram get wol_hosts`
WOLList=($(echo "$WOLHosts" | grep -o '[A-F0-9]\{2\}:[A-F0-9]\{2\}:[A-F0-9]\{2\}:[A-F0-9]\{2\}:[A-F0-9]\{2\}:[A-F0-9]\{2\}=[^=]*=[0-9]*[.][0-9]*[.][0-9]*[.][0-9]*' ))
if [ "${#WOLList[#]}" -gt 0 ]
then
for Match in ${WOLList[#]}
do
Command=`echo "$Match" | sed -r "s/([A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2})=([^=]*)=([0-9]*[.][0-9]*[.][0-9]*[.][0-9]*)/alias \2='\/usr\/sbin\/wol -i \3 \1'/"`
Name=`echo "$Match" | sed -r "s/([A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2})=([^=]*)=([0-9]*[.][0-9]*[.][0-9]*[.][0-9]*)/\2/"`
Com=`echo "$Match" | sed -r "s/([A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2})=([^=]*)=([0-9]*[.][0-9]*[.][0-9]*[.][0-9]*)/\/usr\/sbin\/wol -i \3 \1/"`
alias $Name="$Com"
eval $Command
echo "$Command"
done
fi
exit 0
Here is some sample data and output that I am currently receiving with the script:
Input (into WOLHosts):
00:1F:D0:26:72:53=Justin-PC=192.168.1.255 00:16:17:DD:12:7B=Justin-HTPC=192.168.1.255 00:1C:25:BC:C3:85=justinlaptop=192.168.1.255
The output produced by the vi WOecho "$Command" is:
alias Justin-PC='/usr/sbin/wol -i 192.168.1.255 00:1F:D0:26:72:53'
alias Justin-HTPC='/usr/sbin/wol -i 192.168.1.255 00:16:17:DD:12:7B'
alias justinlaptop='/usr/sbin/wol -i 192.168.1.255 00:1C:25:BC:C3:85'
Since you appear to be running this as a script, your current shell will not receive the aliases -- the aliases will disappear then the bash process driving the script ends.
Try: . script.sh or source script.sh