I am trying to extract and collect some client IP addresses using the command tcpdump, but just can't figure out how to get grep/sed/awk to give me just the first IP address of the output. If I run the following
# tcpdump -t -i vlan4 -n ip dst host 192.168.2
tcpdump: listening on vlan4, link-type EN10MB
tcpdump: WARNING: compensating for unaligned libpcap packets
10.28.6.1.29447 > 192.168.2.62.22: P 3758838950:3758839034..
10.28.6.1.29447 > 192.168.2.62.22: P 84:136(52) ack 117..
10.28.6.1.29447 > 192.168.2.62.22: P 136:188(52) ack 233..
10.28.6.1.29447 > 192.168.2.62.22: . ack 349 win 251 (DF)..
I just want to get the first ip on each line without the port number. So from the above output I want to get just.....
10.28.6.1
10.28.6.1
10.28.6.1
10.28.6.1
Thanks
try this cut line:
cut -f1-4 -d "."
test with your data:
kent$ echo "10.28.6.1.29447 > 192.168.2.62.22: P 3758838950:3758839034..
10.28.6.1.29447 > 192.168.2.62.22: P 84:136(52) ack 117..
10.28.6.1.29447 > 192.168.2.62.22: P 136:188(52) ack 233..
10.28.6.1.29447 > 192.168.2.62.22: . ack 349 win 251 (DF).."|cut -f1-4 -d "."
10.28.6.1
10.28.6.1
10.28.6.1
10.28.6.1
perl -lane '$F[0]=~s/(.*)\..*/$1/g;print $F[0]' your_file
Related
I am receiving the response to a Unix cat /etc/hosts and would like to match on this data with Expect - so I can write it local disk.
I have
#!/usr/bin/expect -f
set timeout -1
spawn nc -lvnp 9090
match_max 100000
expect -re {.*(Connection received).*}
puts "Getting hosts file"
send -- "cat /etc/hosts\r"
expect -re "(.*)\n"
The host file is multiple lines and I am unable to construct the correct regular expression to match on it.
Help would be gratefully received
Update
When interact with when the remote service via netcat manually the response interaction looks like this :
nc -lvnp 9090 Listening on 0.0.0.0 9090 Connection received on
192.168.236.135 44686 cat /etc/hosts
127.0.0.1 localhost
127.0.1.1 ubuntu
# The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters
my Expect code is :
#!/usr/bin/expect -f
set timeout -1
spawn nc -lvnp 9090
match_max 100000
expect -re {.*(Connection received).*}
puts "******Getting hosts file*******"
log_user 0
send -- "cat /etc/hosts \r"
expect -re "(.*\[\r\n])+"
#expect -re "(.*\[\r|\n]).*"
#expect -re "(.*\[\t|\r|\n]).*"
expect -re "(.*\[\r\n]).*$"
puts "Output 0 string"
puts <<<<$expect_out(0,string)>>>>
puts ""
puts "Output 1 string"
puts <<<<$expect_out(1,string)>>>>
puts ""
puts "Output buffer"
puts <<<<$expect_out(buffer)>>>>
puts ""
puts [open hosts.txt w] $expect_out(buffer)
send -- "\x03"
Just seem to be unable to match on the whole response - I'm not 100% sure if I need to match on EOF or EOL or something else. The newlines and tab are also part of the problem I think.
I basically just to to capture the whole response from the 'send -- "cat /etc/hosts" and write that to disk
turns out he response from the remote server was too quick.
A simple 'sleep 1' was good enough to slow things down
#!/usr/bin/expect -f
set timeout -1
spawn nc -lvnp 9090
match_max 100000
expect -re {.*(Connection received).*}
puts "Getting hosts file"
send -- "cat /etc/hosts\r"
sleep 1
expect -re "(.*)\n"
I have been using this little script for months now with success. Today I realize there is one output it cant seem to catch, screen comes up blank with a new prompt:
user#computer ~]$ myscan ipsFile 23
user#computer ~]$
Here is the code
#!/bin/bash
sudo nmap -v -Pn -p T:$2 -reason -i $1 | awk ' {
if (/syn-ack/) {
print "Yes"
c++
}
else if (/no-response|reset|host-unreach/) {
print "No"
c++
}
}
END { print c} '
If I run the nmap against one of the IPs then it returns
Starting Nmap 5.51 ( http://nmap.org ) at 2017-09-26 11:44 CDT
Initiating Parallel DNS resolution of 1 host. at 11:44
Completed Parallel DNS resolution of 1 host. at 11:44, 0.00s elapsed
Initiating Connect Scan at 11:44
Scanning 1.1.1.1 [1 port]
Completed Connect Scan at 11:44, 0.20s elapsed (1 total ports)
Nmap scan report for 1.1.1.1
Host is up, received user-set (0.20s latency).
PORT STATE SERVICE REASON
23/tcp filtered telnet host-unreach
Read data files from: /usr/share/nmap
Nmap done: 1 IP address (1 host up) scanned in 0.26 seconds
How can I catch the 'host-unreach' portion?
Let's try and debug this. Execute this:
nmap -v -Pn -p T:23 -reason -i ipsFile | awk '{print $0}/syn-ack/{print "Yes";c++}/no-response|reset|host-unreach/{print "No";c++}END {print c}' > out.txt
The only difference here is that the awk script prints $0 (i.e. the output of your nmap calls) to file out.txt. Try to grep your unreach value.
I tried this myself and found that instead of a host-unreach I got a net-unreach. Might be the same thing in your case.
Have you tried piping stderr to stdout like
#!/bin/bash
sudo nmap -v -Pn -p T:$2 -reason -i $1 2>&1 | awk ' {
if (/syn-ack/) {
print "Yes"
c++
}
else if (/no-response|reset|host-unreach/) {
print "No"
c++
}
}
END { print c} '
I'm somewhat new to socket programming, and am confused about the concept of binding a socket to the address INADDR_LOOPBACK, or 127.0.0.1.
If I'm writing server code to listen for messages on a specific port, and I bind a socket to an address as in the following code exerpt...
int sd = socket( PF_INET, SOCK_DGRAM, 0 );
sockaddr_in si;
si.sin_family = AF_INET;
si.sin_addr.s_addr = inet_addr( "127.0.0.1" );
si.sin_port = htons( 9090 );
bind( sd, (sockaddr*)&si, sizeof si )
...my question is: who is able to send to this socket?
I know that other processes running on the same PC as the server process can reach the above socket, by doing a sendto() with a dest_addr argument specifying 127.0.0.1.
But can clients on other PCs on the same network also send to that socket if they know the server's "actual" address? What I mean is: if I run ifconfig on a Linux PC, I'll see an inet address, e.g. 10.138.19.27. Does this mean a client process on a different PC than the server, but on the same network, can send to the server's socket - which was bound to 127.0.0.1 - if the client specifies an address of 10.138.19.27?
Only connections to the loopback adapter (127.0.0.1), and those can only originate from the same machine as the listener since the other interfaces intentionally avoid rounding to that one.
When you don't bind or when you bind to INADDR_ANY (0.0.0.0), you accept connections from all interfaces.
Window 1 Window 2
------------------------------------------ ------------------------------------------
$ nc -l 5678
$ echo test-ip | nc 69.163.162.155 5678 test-ip
$ echo $?
0
$ nc -l 5678
$ echo test-localhost | nc localhost 5678 test-localhost
$ echo $?
0
When you bind to an IP address, you only accept connections directed to that IP address.
Window 1 Window 2
------------------------------------------ ------------------------------------------
$ nc -l 69.163.162.155 5678
$ echo test-localhost | nc localhost 5678
$ echo $?
1
$ echo test-ip | nc 69.163.162.155 5678 test-ip
$ echo $?
0
The same goes for addresses in 127.x.x.x.
Window 1 Window 2
------------------------------------------ ------------------------------------------
$ nc -l localhost 5678
$ echo test-ip | nc 69.163.162.155 5678
$ echo $?
1
$ echo test-localhost | nc localhost 5678 test-localhost
$ echo $?
0
The special thing about 127.x.x.x is that only your own machine can reach 127.x.x.x addresses.
I use elasticsearch a lot and I needed a way to see the request to elasticsearch server.
So I found this bash script:
sudo tcpdump -A -nn -s 0 'tcp dst port 9200 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)' -i lo
Works well, the problem is it outputs a lot of junk:
11:40:49.690863 IP 127.0.0.1.50270 > 127.0.0.1.9200: Flags [P.], seq 2594084191:2594084256, ack 2674569216, win 342, options [nop,nop,TS val 910075 ecr 910075], length 65
E..uw.#.#............^#...._.j.....V.i.....
..HEAD / HTTP/1.1
Host: localhost:9200
Connection: keep-alive
11:40:49.707181 IP 127.0.0.1.50271 > 127.0.0.1.9200: Flags [P.], seq 3009491178:3009491248, ack 3451175088, win 342, options [nop,nop,TS val 910080 ecr 910079], length 70
E..z..#.#.jc........._#..a0........V.n.....
..GET /_nodes HTTP/1.1
Host: localhost:9200
Connection: keep-alive
11:40:52.215800 IP 127.0.0.1.50272 > 127.0.0.1.9200: Flags [P.], seq 148869443:148869508, ack 2833610757, win 342, options [nop,nop,TS val 910707 ecr 910707], length 65
E..ut3#.#..M.........`#....C..x....V.i.....
.sHEAD / HTTP/1.1
Host: localhost:9200
Connection: keep-alive
I just want to keep the JSON.
So I though about going by elimination.
I added this regex:
sudo tcpdump -A -nn -s 0 'tcp dst port 9200 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)' -i lo | grep -v "[0-9]{2}[:][0-9]{2}[:][0-9]{2}\.[0-9]*[ ][I][P][ ][0-9]{1,3}\.[0 -9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]*[ ][\>].*"
Here is the regex in action: https://regex101.com/r/aS6kS2/1
Anyways, what I am trying to accomplish here is to remove:
11:40:49.707181 IP 127.0.0.1.50271 > 127.0.0.1.9200: Flags [P.], seq 3009491178:3009491248, ack 3451175088, win 342, options [nop,nop,TS val 910080 ecr 910079], length 70
E..z..#.#.jc........._#..a0........V.n.....
..GET /_nodes HTTP/1.1
Host: localhost:9200
Connection: keep-alive
So that only the body/JSON is kept.
I was trying to go line by line but for now it's not even removing the first one that was the target of my regex.
This will do what you've told us about so far (remove the middle chunk of text from the sequence of 3 chunks of text):
$ awk -v RS= -v ORS='\n\n' 'NR!=2' file
11:40:49.690863 IP 127.0.0.1.50270 > 127.0.0.1.9200: Flags [P.], seq 2594084191:2594084256, ack 2674569216, win 342, options [nop,nop,TS val 910075 ecr 910075], length 65
E..uw.#.#............^#...._.j.....V.i.....
..HEAD / HTTP/1.1
Host: localhost:9200
Connection: keep-alive
11:40:52.215800 IP 127.0.0.1.50272 > 127.0.0.1.9200: Flags [P.], seq 148869443:148869508, ack 2833610757, win 342, options [nop,nop,TS val 910707 ecr 910707], length 65
E..ut3#.#..M.........`#....C..x....V.i.....
.sHEAD / HTTP/1.1
Host: localhost:9200
Connection: keep-alive
If that's not what you want then edit your question to clarify as I suggested in my comment.
I have a string in a bash script that contains a line of a log entry such as this:
Oct 24 12:37:45 10.224.0.2/10.224.0.2 14671: Oct 24 2012 12:37:44.583 BST: %SEC_LOGIN-4-LOGIN_FAILED: Login failed [user: root] [Source: 10.224.0.58] [localport: 22] [Reason: Login Authentication Failed] at 12:37:44 BST Wed Oct 24 2012
To clarify; the first IP listed there "10.224.0.2" was the machine the submitted this log entry, of a failed login attempt. Someone tried to log in, and failed, from the machine at the 2nd IP address in the log entry, "10.224.0.58".
I wish to replace the first occurrence of the IP address "10.224.0.2" with the host name of that machine, as you can see presently is is "IPADDRESS/IPADDRESS" which is useless having the same info twice. So here, I would like to grep (or similar) out the first IP and then pass it to something like the host command to get the reverse host and replace it in the log output.
I would like to repeat this for the 2nd IP "10.224.0.58". I would like to find this IP and also replace it with the host name.
It's not just those two specific IP address though, any IP address. So I want to search for 4 integers between 1 and 3, separated by 3 full stops '.'
Is regex the way forward here, or is that over complicating the issue?
Many thanks.
Replace a fixed IP address with a host name:
$ cat log | sed -r 's/10\.224\.0\.2/example.com/g'
Replace all IP addresses with a host name:
$ cat log | sed -r 's/[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}/example.com/g'
If you want to call an external program, it's easy to do that using Perl (just replace host with your lookup tool):
$ cat log | perl -pe 's/(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/`host \1`/ge'
Hopefully this is enough to get you started.
There's variou ways to find th IP addresses, here's one. Just replace "printf '<<<%s>>>' " with "host" or whatever your command name is in this GNU awk script:
$ cat tst.awk
{
subIp = gensub(/\/.*$/,"","",$4)
srcIp = gensub(/.*\[Source: ([^]]+)\].*/,"\\1","")
"printf '<<<%s>>>' " subIp | getline subName
"printf '<<<%s>>>' " srcIp | getline srcName
gsub(subIp,subName)
gsub(srcIp,srcName)
print
}
$
$ gawk -f tst.awk file
Oct 24 12:37:45 <<<10.224.0.2>>>/<<<10.224.0.2>>> 14671: Oct 24 2012 12:37:44.583 BST: %SEC_LOGIN-4-LOGIN_FAILED: Login failed [user: root] [Source: <<<10.224.0.58>>>] [localport: 22] [Reason: Login Authentication Failed] at 12:37:44 BST Wed Oct 24 2012
googled this one line command together. but was unable to pass the founded ip address to the ssh command:
sed -n 's/\([0-9]\{1,3\}\.\)\{3\}[0-9]\{1,3\}/\nip&\n/gp' test | grep ip | sed 's/ip//' | sort | uniq
the "test" is the file the sed command is searching for for the pattern