Shell Script to Detect if the IP Address Is Googlebot - amazon-web-services

I'm trying to do a bulk reverse DNS the IP's which I got from CloudFront Logs with help of below shell script, which I need to differentiate whether it's suspicious or genuine Googlebot IP. But I'm not getting the output. Not sure where I am going wrong. Is there any other option to do bulk reverse DNS?
#!/bin/sh
file="googlebots"
while read -r line
do
hostName=`host $line | cut -d" " -f 5`
domainName=`echo $hostName | cut -d"." -f2,3`
#echo $domainName
#echo "$hostName"
hostIp=`host $hostName | cut -d" " -f 4`
#echo "$hostIp"
if [ $line == $hostIp ] && [ $domainName == "googlebot.com" ]
then
echo "Googlebot: $hostIp -> $hostName"
fi
done < "$file"

Related

What gcloud command would list all the compute instances (for all gcp projects) with a network tag that contains a specific string?

I have put together the code below to find all resources with a network tag that contains -allowaccess however it doesn't seem to work...
for i in $(gcloud projects list | awk NR>1); do gcloud compute instances list --filter="tags.items:-allowaccess --project=$i; done
Any ideas?

A colleague of mine figured it out...here's the command - hope it's useful to others!
for i in $(gcloud projects list | awk '{print $1}' | awk 'NR>1'); do echo PROJECT: $i && echo "--" && gcloud compute instances list --project=$i --filter="(tags.items:allowaccess)" && echo ""; done
For each project, this outputs each VM with a network tag that contains the text 'allow access'

Try something alike --filter="label:(*allowaccess)" or --filter="labels.*allowaccess:*", because these are generally instance labels. See gcloud topic filters.

I think the code self explain :)
# indice of .csv
echo "project;machine;region;family;value1;value2;value3;value4;value5" >> export.csv
# loop projects
for p in $(gcloud projects list | awk '{print $1}' | awk 'NR>1')
do
# loop values of instance
for i in $(gcloud compute instances list --project=${p} | grep -v "TERMINATED" | grep -v "NAME")
do
if [ "${i}" == "RUNNING" ]
then
echo ${instance}
X=0
elif [[ $X -eq 0 ]]
then
echo -n ${i}
echo -n ";"
echo -n ${i}
echo -n ";"
X=$((X+1))
else
echo -n ${i}
echo -n ";"
X=$((X+1))
fi
done
done >> export.csv
# remove wrong ;
sed -i 's/,;/ /g' export.csv
sed -i 's/;vCPU/ vCPU/g' export.csv
sed -i 's/;GiB/ GiB/g' export.csv

Delete Mails from Postfix Queue with filter from and to

I try to delete some special mails from Postifx Queue.
I want to Filter with "Mail From" and Mail Domain "TO".
I have tried:
mailq|grep 'info#sendingdomain.com'|awk '/ #test\.com$/ { print $1 }' | tr -d '*!' | postsuper -d -
But it don't work.

You can use this bunch of code
#!/bin/bash
[ ! -n "$1" ] && echo "Usage : $0 mail" && exit 1
for mail in $( mailq|egrep '^[A-Z0-9]'|grep "$1"|awk '{print $1}'|sed 's/^\([0-9A-Z]*\).*$/\1/' )
do
/usr/sbin/postsuper -d $mail
done

ipv6 validation using regex

#!/bin/bash
echo "enter the ip address:"
read s
if [[ $s =~ ^(([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,7}:|([0-9a-fA-F]{1,4}:){1,6}:[0-9a-fA-F]{1,4}|([0-9a-fA-F]{1,4}:){1,5}(:[0-9a-fA-F]{1,4}){1,2}|([0-9a-fA-F]{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}|([0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}|([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4}){1,5}|[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4}){1,6})|:((:[0-9a-fA-F]{1,4}){1,7}|:)|fe80:(:[0-9a-fA-F]{0,4}){0,4}%[0-9a-zA-Z]{1,}|::(ffff(:0{1,4}){0,1}:){0,1}((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])|([0-9a-fA-F]{1,4}:){1,4}:((25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9])\.){3,3}(25[0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]))$ ]]; then
echo -e '\E[47;31m'"\033[1mIPv6 Format\033[0m"
echo -n "The IPv6 Address Expanded Form:"
EXPANDED=`sipcalc $s | fgrep Expand | cut -d '-' -f 2`
echo -e "\033[32m $EXPANDED\033[0m"
echo -n "IPv6 address Compress Form:"
Compress=`sipcalc $s | fgrep Comp | cut -d '-' -f 2`
echo -e "\033[32m$Compress\033[0m"
echo -n "Address Type of IPv6:"
type=`sipcalc $s | fgrep type | cut -d '-' -f 2,3,4`
comment=`sipcalc $s | fgrep Comment | cut -d '-' -f 2`
echo -e "\033[32m $type$comment\033[0m"
else
echo -e '\E[37;44m'"\033[1mNOT VALID IPv6 address\033[0m"
fi
Hello everyone. I am trying to validate IPv6 addresses using this script. It is working well. The problem is that it's also accepting IPs like 1111:2222:3333:4444::. Could you help me to avoid this types of IPs?

Why don't you use built-in linux utility,
ipcalc --ipv6 1111:2222:3333:4444::
If it returns nothing then you've provided correct IP
If you provide incorrect IP it returns something like,
ipcalc: bad IPv6 address:
Hope it helps.

This regular expression should do the trick :
([0-9a-fA-F]{1,4}:){7}[0-9a-fA-F]{1,4}
It matches 8 blocks of 1 to 4 hexadecimal values, joined by 7 :.

Continue a bash script

After the success view of the host ( IPs ) I need to ping them in order to check if they are up. SIDS file contains 2 columns with hostnames. Are there any suggestions on how to Iimprove the code below?
#!/bin/bash
LINES=`cat /home/marko/SIDS | sed "s!/!-!g" | wc -l`
for (( i=1; i<=${LINES}; i++))
do
FIRSTIP=CPE-`sed -n "${i}{p;q}" /home/marko/SIDS | awk '{print $1}'| sed "s!/!-!g"`
SECONDIP=CPE-`sed -n "${i}{p;q}" /home/marko/SIDS | awk '{print $2}'| sed "s!/!-!g"`
COUNT=$( host ${FIRSTIP} | grep address | wc -l )
if [ $COUNT -gt 0 ]
then
echo success
else
echo ${SECONDIP}
fi
done

You can just use dig, to avoid searching the output of host:
IP=$(dig +short $SERVERNAME)
Then to check, if the host is alive:
if ping -q -c $IP >/dev/null 2>&1
then
echo "OK"
fi

Using the eval command to create a command alias

For the life of me I cannot get the bash script to execute the alias command to set the hostname of a workstation the alias name to the WOL (Wakeup On Lan) equivalent command. I figure there must be an issue with quoting somewhere that I am missing.
#!/bin/bash
WOLHosts=`nvram get wol_hosts`
WOLList=($(echo "$WOLHosts" | grep -o '[A-F0-9]\{2\}:[A-F0-9]\{2\}:[A-F0-9]\{2\}:[A-F0-9]\{2\}:[A-F0-9]\{2\}:[A-F0-9]\{2\}=[^=]*=[0-9]*[.][0-9]*[.][0-9]*[.][0-9]*' ))
if [ "${#WOLList[#]}" -gt 0 ]
then
for Match in ${WOLList[#]}
do
Command=`echo "$Match" | sed -r "s/([A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2})=([^=]*)=([0-9]*[.][0-9]*[.][0-9]*[.][0-9]*)/alias \2='\/usr\/sbin\/wol -i \3 \1'/"`
Name=`echo "$Match" | sed -r "s/([A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2})=([^=]*)=([0-9]*[.][0-9]*[.][0-9]*[.][0-9]*)/\2/"`
Com=`echo "$Match" | sed -r "s/([A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2}:[A-F0-9]{2})=([^=]*)=([0-9]*[.][0-9]*[.][0-9]*[.][0-9]*)/\/usr\/sbin\/wol -i \3 \1/"`
alias $Name="$Com"
eval $Command
echo "$Command"
done
fi
exit 0
Here is some sample data and output that I am currently receiving with the script:
Input (into WOLHosts):
00:1F:D0:26:72:53=Justin-PC=192.168.1.255 00:16:17:DD:12:7B=Justin-HTPC=192.168.1.255 00:1C:25:BC:C3:85=justinlaptop=192.168.1.255
The output produced by the vi WOecho "$Command" is:
alias Justin-PC='/usr/sbin/wol -i 192.168.1.255 00:1F:D0:26:72:53'
alias Justin-HTPC='/usr/sbin/wol -i 192.168.1.255 00:16:17:DD:12:7B'
alias justinlaptop='/usr/sbin/wol -i 192.168.1.255 00:1C:25:BC:C3:85'

Since you appear to be running this as a script, your current shell will not receive the aliases -- the aliases will disappear then the bash process driving the script ends.
Try: . script.sh or source script.sh