I have text files with hundreds of entries like those below. They mostly come in pairs of 2 IPs. Sometimes they come as 3 IPs. I am trying to find that third IP that is always in the middle of the stack (syntax below). There are maximum 3 different IPs in each file at all times. It is possible that some text files won’t have that middle IP (its occurrence is quite rare). How do I write the search command to find the middle IP from mentioned stacks if there is one in the text file? OS: Win7.
Text file sample syntax:
- saving IP addresses
* 192.168.1.1
* 111.111.222.222
- over
- saving IP addresses
* 192.168.1.1
* 11.123.11.123
* 111.111.222.222
- over
- saving IP addresses
* 192.168.1.1
* 111.111.222.222
- over
I have tried findstr \-.*\*.*\*.*\- pathtofile.txt This should return the block of 3 IPs if there is such block in the file but it didn't work.
Assuming your real file isn't double-spaced like your sample, the following will output the first line (saving...) and line number of matching blocks. Your real problem is findstr will only output one line even if you are matching across lines, so you will never get the whole block output. You need a better tool.
Note: I am using the JPSoft Take Command escape character to put in CR and LF, but you can create them in real batch files as well, though it isn't easy.
findstr /n /R saving.*^r^n.*\..*\..*\..*^r^n.*\..*\..*\..*^r^n.*\..*\..*\..*^r^n sampleIPinput.txt
Related
I am trying to build a connection string that requires pulling 3 IP addresses from another config file. When I get those values, I need to replace the port on each. I plan to replace each port using simple Bash find and replace ${string/pattern/replacement} but my problem is I'm stuck on the best way to parse the pattern out of the IP.
Here is what I have so far:
myFile.config:
ip.1=ip-ip-1-address:1234:5678
ip.2=ip-ip-2-address:1234:5678
ip.3=ip-ip-3-address:1234:5678
Copying some other simple process, I found I can pull the value of each IP like this:
IP1=`grep "ip.1=" /path/to/conf/myFile.config | awk -F "=" '{print $2}'`
which gives me ip.1=ip-ip-1-address:1234:5678. However, I need to replace 1234:5678 with 6543 for example. I've been looking around and I found this awesome answer that detailed using Bash prefix substitution but that relies on knowing the parameter. for example, I would have to do it this way:
test=${ip1##ip-ip-1-address:}
which results in $test being 1234:5678. That's fine but maybe I don't know the IP address as the parameter, so I'm back to considering regex unless there's a way for me to use * as the parameter or something, but I have been unsuccessful so far. For regex, I have tried a bunch such as test=${ip1/(?<=:).*/}.
Note that ${ip1/(?<=:).*/} you tried is an example of string manipulation syntax that does not support regex, only specific patterns.
You seem to want
x='ip.1=ip-ip-1-address:1234:5678'
echo "${x%%:*}:6543" # => ip.1=ip-ip-1-address:6543
The ${x%%:*} takes the value of x and removes all chars from the end till the first : including it. :6543 is added to the result of this manipulation using "${x%%:*}:6543".
To extract that value, you may also use
awk '/^ip\.1=/{sub("^[^:]+:", "");print}' myFile.config
The awk command finds lines starting with ip.1= and then removes all text from the start till the first colon including the colon and only prints these values.
I have a report that has information about a list of servers. I am wanting to search this list for uptime over a certain amount, and also the IP of the server. I have been using notepad++ to do the searching, but sed syntax would be ok too. The report has data like this:
some.dns.com
up 720 days,
some version
several lines of disk space information, between 14 and 16 lines
Connection to 10.1.1.1 closed.
some.other.dns
up 132 days,
some version
several lines of disk space information, between 14 and 16 lines
Connection to 10.1.1.2 closed.
I've come up with the following so far, which gives me the uptime threshold I need:
up ([9-9]\d|\d{3,} days,)
But I also need the IP addresses to make sense of it, and haven't been able to figure out a way to get JUST the IPs related to the servers with high uptime.
I've found something like this to find IP addresses:
((?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?\.){3}(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)))
So, I was hoping to return something like the following:
up 720 days,
10.1.1.1
You may actually use awk:
awk -F"\n" -v RS="" '$0 ~ /up (9[0-9]|[0-9]{3,}) days/{gsub(/Connection to | closed\./, "", $NF); print $1 "\n" $NF}' file > newfile
See the online demo
The file is read paragraph by paragraph, and fields are separated with a newline. If a record matches up (9[0-9]|[0-9]{3,}) days pattern (up with a space, then 9 followed with any digit or any 3 digits followed with space and days, then the last field ($NF) is stripped from the static text and the first and last fileds are printed.
I want to grab all occurrences in configuration file where first line starts from 'object' and immediately second line starts with 'nat'
object network obj_any
nat (inside,outside) dynamic interface
object network obj-test
nat (DMZ1,outside) static 10.206.49.180
object network obj-192.168.236.200
nat (DMZ1,outside) static 10.206.74.60
object network obj-192.168.236.8
nat (DMZ1,outside) static 10.206.49.183 tcp 8080 80
object network obj-192.168.236.9
nat (DMZ1,outside) static 10.206.49.178 tcp 1002 22
object network obj-192.168.236.10
nat (DMZ1,outside) static 10.206.49.178 tcp 8080 80
object network obj-192.168.236.13
nat (DMZ1,outside) static 10.206.74.58 dns
I tried below but seems not working
object network .+? nat .+? static .+?
and selected 'match new line" but seems not matching
I believe that this cannot be done in one step with Notepad++. A multi-step process to copy the lines is as follows.
(1) Find the wanted pairs of lines and merge them into single lines with a marker string. (2) Bookmark all lines with the marker and copy them. (3) Paste the wanted lines into a new buffer and convert the marker strings back to newlines.
In more detail.
(Setup) Choose a marker string, something that does not occur anywhere in the buffer being searched or in the destination buffer. For this example I choose !!!.
(1) Do a regular expression replace of ^(object.*)\R+( nat.*)$ with `\1!!!\2'. This converts the wanted lines so the first pair shown in the question become:
object network obj_any!!! nat (inside,outside) dynamic interface
(2) Open the search window and select the Mark tab. Click on Clear all marks, tick Bookmark line, enter the marker string (i.e. !!!) into the Find what field and click on Mark all. Select menu => Search =>
Bookmark => Copy bookmarked lines.
(3) Select the place where the copied lines should be written and Paste in the copied lines. Do a regular expression search and replace of !!! with \r\n\r\n. (May need to alter the replacement string if you preferred line endings are not Windows.)
Notes
The above does not preserve the exact sequence of CRs and LFs between the two lines. The first replace uses \R+ to find any combination of CRs and LFs between the two lines. The final replace inserts a fixed CR and LF sequence.
Rather than using the Copy bookmarked lines it may be suitable to use Remove unmarked lines which then leaves only the wanted lines in the buffer. The Copy and Paste command are then not needed and the final search and replace can be done in the initial buffer.
I am working with a cloud environment and there is a command that will display all available information about VMs running. here is an example of some of the lines that pertain to one VM.
RESERVATION r-6D0F464B 170506678332 GroupD
INSTANCE i-E9B444A9 emi-376642D8 999.99.999.999 88.888.88.888 running lock_key 0 c1.xlarge 2013-06-17T18:40:56.270Z cluster01 eki-E7E242A3 monitoring-disabled 999.99.999.999 88.888.88.888 ebs
I need to be able to pull the i-********, emi-********, both IP address, its status, the lock_key, the c1.xlarge, and the monitoring-disabled/enabled.
I have been able to pull the whole line with some super simple regex but all of this is well beyond me. If there is another easier method of grabbing this data any suggestions are welcome.
Let's go by parts. Best way I can think of is redirecting the output to a file, in unix-like environments you do it like:
cat your-command > filename.txt
Second, you need to read the file line by line, I would use a python script or a perl script if you know any of those, or whatever language fits you.
Third, you can get values two different ways:
Read columns by position, you can get colums with a regex like: [^\s]+
Write regular expressions for every specific column, so for IP you could have something like this: ([0-9]{1,3}\.){4} for monitoring monitoring-([^\s]+) and so on.
As long as the fields will always be in the same order, all you need to is split on whitespace.
Pseudocode (well, it's ruby, but hopefully you get the idea):
vms = {}
File.open('vm-info').readlines.each do |line|
fields = line.split('\s+')
field_map = {}
vm_name = fields[<index_of_vm_name>]
field_map['emi'] = fields[<index_of_emi>]
field_map['ip_address'] = fields[<index_of_ip_address]
.
.
.
vms[vm_name] = field_map
end
After this, vms will be initialized to contain information about each vm. You can simply print them all out at this point, or continue running data manipulation on them.
I have a huge log file that has a structure like this:
ip=X.X.X.X
userAgent=Firefox
-----
Referer=hxxp://www.bla.org
I want to create a custom output like this:
ip:userAgent
for ex:
X.X.X.X:Firefox
and the pattern will ignore lines which don't start with ip= and userAgent=. (these two must form a pair as i mentioned above.)
I am a newbie administrator and our client needs a sorted file immediately.
Any help will be wonderful.
Thanks.
^ip=(\d+(?:\.\d+){3})[\r\n]+userAgent=(.+)$
Apply in global + multiline mode.
Group 1 will contain the IP, group 2 will contain the user agent string.
Edit: The above expression can be simplified a bit, we can remove the IP address format checking - assuming that there will be nothing but real IP addresses in the log file:
^ip=(\d+\.?)+[\r\n]+userAgent=(.+)$
You can use:
^ip=((?:[0-9]{1,3}\.){3}[0-9]{1,3})$
And
^userAgent=(.*)$
Get the group 1 for both and you will have the desired data.
give it a try (this is in no way robust if there are lines where your log file differs from the example snippet above):
sed -n -e '/^ip=/ {s///
N
s/\nuserAgent=/:/
p
}' HugeFile > customoutput