Bash break down IP range into variables - regex

I'm trying to write a bash script which will do the following
Read a file that has IP addresses stored in it
ips.txt
Take those IP addresses and break them into 3 variables that write to their own files
Files
thrid_octave.txt
last_digit_third_octave.txt
last_octive.txt
Variables
thrid_octave
last_digit_third_octave
last_octive
EX: 12.34.117.88
thrid_octave = 111
last_digit_third_octave = 7
last_octive = 88
Below is how i intend to use those variables
#!/bin/bash
domain="$(cat domains.txt)"
ip="$(cat ips.txt)"
last_number_third_octete="$(cat last_number_third_octete.txt)"
thrid_octive="$(cat thrid_octive.txt)"
last_octive="$(cat last_octive.txt)"
while read -r domain <&3 && read ip <&4 && read last_number_third_octete <&5 && read thrid_octive <&6 && read last_octive <&7 &&;
do
echo "processing $domain";
"##################################### $domain ######################################################"
smtp-listener $ip\n
<virtual-mta $domain.c$last_number_third_octete.$last_octive>
smtp-source-host $ip mailer1-vmta-plat-$thrid_octive-$last_octive.$domain
</virtual-mta>
<virtual-mta-pool $domain.p>
virtual-mta $domain.c$thrid_octive-$last_octive
</virtual-mta-pool>
\n\n
done 3<domains.txt 4<ips.txt 5<last_number_third_octete.txt 6<thrid_octive.txt 7<last_octive.txt
echo "All Set"

You could try to use cut, like this:
ip="12.34.117.88"
thrid_octave=`echo $ip | cut -d\. -f3`
last_digit_third_octave=`echo $ip | cut -d\. -f3 | cut -c 3`
last_octive=`echo $ip | cut -d\. -f4`

Related

Creating an alert function in Bash

I wanted to create a function in bash similar to a default alias I got in Ubuntu, looking like:
alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//'\'')"'
This creates a simple notification after a command has been issued with it.
For example, using
history | grep vim; sleep 5; alert
gives a notification after the sleep is done, simply saying
history | grep vim; sleep 5;
I would like to write the alert into a bash function instead, which have given some trouble with the regex.
I have tried:
function alert2 () {
ICON=$([ $? = 0 ] && echo terminal || echo error)
MSG=$(history | tail -n1 | sed -e s/^\s*[0-9]\+\s*//\;s/[\;\&\|]\s*alert$//)
notify-send --urgency=low -i $ICON $MSG
}
which would output both the linenumber in history when called itself, and give an Invalid number of options when called such as the first example.
Is this possible, and if so, how? Is it simply my regex that is faulty?
I'm running on WSL, so don't have notify-send installed:
function alert2 () {
ICON=$([ $? = 0 ] && echo terminal || echo error);
MSG=$(history | tail -n1| sed -e 's/^\s*[0-9]\+\s*//;s/[;&|]\s*alert2$//');
echo $ICON $MSG;
}
jadams#Temp046317:~/code/data-extract$ cat /etc/apt/sources.list > /dev/null ; alert2
terminal cat /etc/apt/sources.list > /dev/null
I'm hoping that this would work for you (instead of the echo):
notify-send --urgency=low -i "$ICON $MSG"

AWK catching a regular expression

I have been using this little script for months now with success. Today I realize there is one output it cant seem to catch, screen comes up blank with a new prompt:
user#computer ~]$ myscan ipsFile 23
user#computer ~]$
Here is the code
#!/bin/bash
sudo nmap -v -Pn -p T:$2 -reason -i $1 | awk ' {
if (/syn-ack/) {
print "Yes"
c++
}
else if (/no-response|reset|host-unreach/) {
print "No"
c++
}
}
END { print c} '
If I run the nmap against one of the IPs then it returns
Starting Nmap 5.51 ( http://nmap.org ) at 2017-09-26 11:44 CDT
Initiating Parallel DNS resolution of 1 host. at 11:44
Completed Parallel DNS resolution of 1 host. at 11:44, 0.00s elapsed
Initiating Connect Scan at 11:44
Scanning 1.1.1.1 [1 port]
Completed Connect Scan at 11:44, 0.20s elapsed (1 total ports)
Nmap scan report for 1.1.1.1
Host is up, received user-set (0.20s latency).
PORT STATE SERVICE REASON
23/tcp filtered telnet host-unreach
Read data files from: /usr/share/nmap
Nmap done: 1 IP address (1 host up) scanned in 0.26 seconds
How can I catch the 'host-unreach' portion?
Let's try and debug this. Execute this:
nmap -v -Pn -p T:23 -reason -i ipsFile | awk '{print $0}/syn-ack/{print "Yes";c++}/no-response|reset|host-unreach/{print "No";c++}END {print c}' > out.txt
The only difference here is that the awk script prints $0 (i.e. the output of your nmap calls) to file out.txt. Try to grep your unreach value.
I tried this myself and found that instead of a host-unreach I got a net-unreach. Might be the same thing in your case.
Have you tried piping stderr to stdout like
#!/bin/bash
sudo nmap -v -Pn -p T:$2 -reason -i $1 2>&1 | awk ' {
if (/syn-ack/) {
print "Yes"
c++
}
else if (/no-response|reset|host-unreach/) {
print "No"
c++
}
}
END { print c} '

Create conditional bash script code

I use the following code to load some text file with emails
and create users in the system with user password.
the text file contain emails like following
abc#gmail.com
BDD#gmail.com
ZZZ#gmail.com
In case the name is coming with upper case I convert it to lower case, I was able to make it work.
Now I need to support another input instead of email
e.g.
P123456
Z877777
but now I dont want for this type of input to convert it to lower case
someting like
if(emailpattern )
convert to lower
else
Not
This is the code which works but I failed to make it work...
for user in $(cat ${users} | awk -F";" '{ print $1 }'); do
user=$(echo ${user} | tr "[:upper:]" "[:lower:]")
log "cf create-user ${user} ${passwd}"
#Here we are creating email user in the sys
cf create-user ${user} ${passwd} 2>&1 |
tee -a ${dir}/${scriptname}.log ||
{ log "ERROR cf create-user ${user} failed" ;
errorcount=$[errorcount + 1]; }
done
You can use:
while IFS= read -r user; do
# convert to lowercase only when $user has # character
[[ $user == *#* ]] && user=$(tr "[:upper:]" "[:lower:]" <<< "$user")
log "cf create-user ${user} ${passwd}"
cf create-user ${user} ${passwd} 2>&1 |
tee -a ${dir}/${scriptname}.log ||
{ log "ERROR cf create-user ${user} failed" ;
errorcount=$[errorcount + 1]; }
done < <(awk -F ';' '{ print $1 }' "$users")
Assumptions:
input file consists of email addresses or names, each on a separate line
email addresses are to be converted to lower case
names are to be left as is (ie, no conversion to lower case)
all of the log/cf/tee/errorcount code functions as desired
Sample input file:
$ cat userlist
abc#gmail.com
BDD#gmail.com
ZZZ#gmail.com
P123456
Z877777
We'll start by using awk to conditionally convert email addresses to lower case:
$ awk '/#/ {$1=tolower($1)} 1' userlist
abc#gmail.com
bdd#gmail.com
zzz#gmail.com
P123456
Z877777
first we'll run the input file (userlist) through awk ...
/#/ : for lines that include an email address (ie, contains #) ...
$1=tolower($1) : convert the email address (field #1) to all lowercase, then ...
1 : true for all records and implies print all inputs to output
Now pipe the awk output to a while loop to perform the rest of the operations:
awk '/#/ {$1=tolower($1} 1}' userlist | while read user
do
log "cf create-user ${user} ${passwd}"
#Here we are creating email user in the sys
cf create-user ${user} ${passwd} 2>&1 |
tee -a ${dir}/${scriptname}.log ||
{ log "ERROR cf create-user ${user} failed" ;
errorcount=$((errorcount + 1)) ;
}
done
updated to correctly increment errorcount by 1
bash can lower-case text:
while IFS= read -r line; do
[[ $line == *#* ]] && line=${line,,}
# do stuff with "$line"
done

Bash variable substitution in a function

I have a function read_command defined as:
function read_command {
local __newline __lines __input __newstr __tmp i;
exec 3< "$*";
__newline=$'\n';
__lines=();
while IFS= read <&3 -r __line && [ "$__line" != '####' ]; do
echo "$__line";
__lines+=("$__line");
done
while IFS= read <&3 -r __line && [ "$__line" != '####' ]; do
read -e -p "${__line#*:}$PS2" __input;
local ${__line%%:*}="$__input";
done
__command="";
for i in "${__lines[#]}"; do
__tmp=$(echo "${i}");
__command="${__command} ${__newline} ${__tmp}";
done
echo -e "$__command";
}
In the current directory there is a file named "test", with the following
content:
greet ${a:-"Bob"}
greet ${b:-"Jim"}
####
a: name of friend a
b: name of friend b
####
In the terminal, the command executed is
read_command test
With no input, I am expecting the output of the last statement to be:
greet Bob
greet Jim
But what I get is:
greet ${a:-"Bob"}
greet ${b:-"Jim"}
What is wrong here?
Edit: As suggested by David, adding eval works in some cases except the following one.
j=1;i="export DISPLAY=:0 && Xephyr :${j}&";k=$(eval echo "$i");
echo $k
export DISPLAY=:0
I am expecting k to be "export DISPLAY=:0 && Xephyr :1&", what's wrong here?
Edit: I tried with the following
k=$(eval "echo \"$i\"")
This is the link to the script I am working on.
https://gist.github.com/QiangF/565102ba3b6123942b9bf6b897c05f87
During the first while loop, in echo "$__line", you have __line='greet ${a:-"Bob"}'. When you try to print that, Bash won't be expanding ${a:-"Bob"} into Bob. (Even if you remove the quotes around $__line this won't happen.) To get that effect, you need to add eval, as in, e.g., eval echo "$__line". Unfortunately eval comes with its can of worms, you have to start worrying about interactions between quoting levels and such.

Accumulo Overview console not reachable outside of VirtualBox VM

I am running Accumulo 1.5 in an Ubuntu 12.04 VirtualBox VM. I have set the accumulo-site.xml instance.zookeeper.host file to the VM's IP address, and I can connect to accumulo and run queries from a remote client machine. From the client machine, I can also use a browser to see the hadoop NameNode, browse the filesystem, etc. But I cannot connect to the Accumulo Overview page (port 50095) from anywhere else than directly from the Accumulo VM. There is no firewall between the VM and the client, and besides the Accumulo Overview page not being reachable, everything else seems to work fine.
Is there a config setting that I need to change to allow outside access to the Accumulo Overview console?
thanks
I was able to get the Accumulo monitor to bind to all network interfaces by manually applying this patch:
https://git-wip-us.apache.org/repos/asf?p=accumulo.git;a=commit;h=7655de68
In conf/accumulo-env.sh add:
# Should the monitor bind to all network interfaces -- default: false
export ACCUMULO_MONITOR_BIND_ALL="true"
In bin/config.sh add:
# ACCUMULO-1985 provide a way to use the scripts and still bind to all network interfaces
export ACCUMULO_MONITOR_BIND_ALL=${ACCUMULO_MONITOR_BIND_ALL:-"false"}
And modify bin/start-server.sh to match:
SOURCE="${BASH_SOURCE[0]}"
while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
bin="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
SOURCE="$(readlink "$SOURCE")"
[[ $SOURCE != /* ]] && SOURCE="$bin/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
done
bin="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
# Stop: Resolve Script Directory
. "$bin"/config.sh
HOST="$1"
host "$1" >/dev/null 2>/dev/null
if [ $? -ne 0 ]; then
LOGHOST="$1"
else
LOGHOST=$(host "$1" | head -1 | cut -d' ' -f1)
fi
ADDRESS="$1"
SERVICE="$2"
LONGNAME="$3"
if [ -z "$LONGNAME" ]; then
LONGNAME="$2"
fi
SLAVES=$( wc -l < ${ACCUMULO_HOME}/conf/slaves )
IFCONFIG=/sbin/ifconfig
if [ ! -x $IFCONFIG ]; then
IFCONFIG='/bin/netstat -ie'
fi
# ACCUMULO-1985 Allow monitor to bind on all interfaces
if [ ${SERVICE} == "monitor" -a ${ACCUMULO_MONITOR_BIND_ALL} == "true" ]; then
ADDRESS="0.0.0.0"
fi
ip=$($IFCONFIG 2>/dev/null| grep inet[^6] | awk '{print $2}' | sed 's/addr://' | grep -v 0.0.0.0 | grep -v 127.0.0.1 | head -n 1)
if [ $? != 0 ]
then
ip=$(python -c 'import socket as s; print s.gethostbyname(s.getfqdn())')
fi
if [ "$HOST" = "localhost" -o "$HOST" = "`hostname`" -o "$HOST" = "$ip" ]; then
PID=$(ps -ef | egrep ${ACCUMULO_HOME}/.*/accumulo.*.jar | grep "Main $SERVICE" | grep -v grep | awk {'print $2'} | head -1)
else
PID=$($SSH $HOST ps -ef | egrep ${ACCUMULO_HOME}/.*/accumulo.*.jar | grep "Main $SERVICE" | grep -v grep | awk {'print $2'} | head -1)
fi
if [ -z $PID ]; then
echo "Starting $LONGNAME on $HOST"
if [ "$HOST" = "localhost" -o "$HOST" = "`hostname`" -o "$HOST" = "$ip" ]; then
#${bin}/accumulo ${SERVICE} --address $1 >${ACCUMULO_LOG_DIR}/${SERVICE}_${LOGHOST}.out 2>${ACCUMULO_LOG_DIR}/${SERVICE}_${LOGHOST}.err &
${bin}/accumulo ${SERVICE} --address ${ADDRESS} >${ACCUMULO_LOG_DIR}/${SERVICE}_${LOGHOST}.out 2>${ACCUMULO_LOG_DIR}/${SERVICE}_${LOGHOST}.err &
MAX_FILES_OPEN=$(ulimit -n)
else
#$SSH $HOST "bash -c 'exec nohup ${bin}/accumulo ${SERVICE} --address $1 >${ACCUMULO_LOG_DIR}/${SERVICE}_${LOGHOST}.out 2>${ACCUMULO_LOG_DIR}/${SERVICE}_${LOGHOST}.err' &"
$SSH $HOST "bash -c 'exec nohup ${bin}/accumulo ${SERVICE} --address ${ADDRESS} >${ACCUMULO_LOG_DIR}/${SERVICE}_${LOGHOST}.out 2>${ACCUMULO_LOG_DIR}/${SERVICE}_${LOGHOST}.err' &"
MAX_FILES_OPEN=$($SSH $HOST "/usr/bin/env bash -c 'ulimit -n'")
fi
if [ -n "$MAX_FILES_OPEN" ] && [ -n "$SLAVES" ] ; then
if [ "$SLAVES" -gt 10 ] && [ "$MAX_FILES_OPEN" -lt 65536 ]; then
echo "WARN : Max files open on $HOST is $MAX_FILES_OPEN, recommend 65536"
fi
fi
else
echo "$HOST : $LONGNAME already running (${PID})"
fi
Check that the monitor is bound to the correct interface, and not the "localhost" loopback interface. You may have to edit the monitors file in Accumulo's configuration directory with the IP/hostname of the correct interface.