Grep regex treated as path - regex

I have a script, where I read strings from txt file, then assign it to $snmp_cred variable and then trying to strip ip address from strings, using grep, into another variable ($snmp_ip)
while read snmp_cred; do
echo appliance $ADDM_address and $snmp_cred
snmp_ip=$(echo $snmp_cred | grep "/((25[0-5]|2[0-4][0-9]|[01]?[1-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[1-9][0-9]?)\d(?=#)/g")
echo IP for snmp community is $snmp_ip
done </tmp/input.txt
Content of input.txt file is:
a10networks/generic/1.3.6.1.4.1.22610.1.3.27_thunder_series4430s/10.72.168.33#public
a10networks/generic/1.3.6.1.4.1.22610.1.3.23_thunder_series1030s/172.17.48.24#public
a10networks/generic/1.3.6.1.4.1.22610.1.3.16_ax3200_12/10.251.1.101#public
The regex works in online regex editor, but fails into bash script. Bash output is:
++ echo $'a10networks/generic/1.3.6.1.4.1.22610.1.3.27_thunder_series4430s/10.72.168.33#public\r'
++ grep '/((25[0-5]|2[0-4][0-9]|[01]?[1-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[1-9][0-9]?)\d(?=#)/g'
+ snmp_ip=
+ echo IP for snmp community is
IP for snmp community is
can anyone point, what an I doing wrong?

Since you are not getting the matched texts only, you do not really need the lookahead that the POSIX regex does not support. Also, note that \d is not supported by POSIX regex standard either. Also, grep pattern should not be placed inside regex delimiters.
If you still need to use your pattern (say, to also grab the matches), pass the -oP option use:
grep -oP "((25[0-5]|2[0-4]\d|[01]?[1-9]\d?)\.){3}(25[0-5]|2[0-4]\d|[01]?[1-9]\d?)\d(?=#)"
And the online demo

In this statement:
snmp_ip=$($snmp_cred | grep "/((25[0-5]|2[0-4][0-9]|[01]?[1-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[1-9][0-9]?)\d(?=#)/g")
you are just expanding the variable, without passing it to grep.
you need to either pass it to grep as an argument (in the form of a file redirection) or send it to greps STDIN.
this worked for me
#!/bin/bash
while read snmp_cred; do
#echo appliance $ADDM_address and $snmp_cred
snmp_ip=$(grep -E -o "((25[0-5]|2[0-4][0-9]|[01]?[1-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[1-9][0-9]?)#" <<< $snmp_cred)
echo IP for snmp community is $snmp_ip
done <input.txt
output:
IP for snmp community is 10.72.168.33#
IP for snmp community is 172.17.48.24#
IP for snmp community is

Related

How to extract value from shell and regex

I have a string "12G 39G 24% /dev" . I have to extract the value '24'. I have used the below regex
grep '[0-9][0-9]%' -o
But I am getting output as 24%. I want only 24 as output and don't want '%' character. How to modify the regex script to extract only 24 as value?
One option would be to just grep again for the digits:
grep -o '[0-9][0-9]%' | grep -o '[0-9][0-9]'
However, if you want to accomplish this with a single regex, you can use the following:
grep -Po '[0-9]{2}(?=%)'
Note the -P option in this case; vanilla grep doesn't seem to support the (?=%) "look-around" part.
The most common way not to capture something is using look-around assertions:
Use it like this
grep -oP '[0-9][0-9](?=%)'
It's worth noting that GNU grep support the -P option to enable Perl compatible regex syntax, however it is not included with OS X. On Linux, it will be available by default. A workaround would be to use ack instead.
But I'd still recommend to use GNU grep on OS X by default. It can be installed on OSX using Homebrew with the command brew grep install
Also, see How to match, but not capture, part of a regex?
You can use sed as an alternative:
sed -rn 's/(^.*)([[:digit:]]{2})(%.*$)/\2/p' <<< "12G 39G 24% /dev"
Enable regular expressions with -r or -E and then split the line into 3 sections represented through parenthesis. Substitute the line for the second section only and print.
Use awk:
awk '{print $3+0}'
The value you seek is in the third field, and adding a zero coerces the string to a number, so % is removed.

Use sed and regex to isolate data

Hehey
i'm on the way to learn about sed and regex and grouping and how to isolate data from a file with that.
Ok i wrote a sed command there give me, all the IPs in the auth.log and write the ips in the logips.log file, so i need to grouping the regex and take the second (ip) group.
sed 's/(.*)([0-9]{1,3}[.][0-9]{1,3}[.][0-9]{1,3}[.][0-9]{1,3})(.*)/\/2/g' /var/log/auth.log > logips.log
But i have every time the whole auth.log in my logips.log. After 2 hours of thinking and seeking, i'm here and asking.
i hope someone can push me in the right direction to solve this.
happy greetings
To get all IP from the auth.log, try grep instead.
grep -o -E "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" auth.log
-o output only match
-E extended regex
[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3} match IP

Get text between two patterns with egrep and awk

I'm trying to parse a command's help file to grab all the arguments the command excepts.
Here is some text from the help file:
* --digest:
Set the digest for fingerprinting (defaults to the digest used when
signing the cert). Valid values depends on your openssl and openssl ruby
extension version.
* --debug:
Enable full debugging.
* --help:
Print this help message
* --verbose:
Enable verbosity.
* --version:
Print the puppet version number
I want to just grab --argument and nothing else.
I almost got it with this command, but its still including the ":" which I want to exclude:
puppet cert --help | egrep '^* --(.*):$' | awk '{print $2}'
--all:
--allow-dns-alt-names:
--digest:
--debug:
--help:
--verbose:
--version:
Why is '^* --(.*):$' including the ":" shouldn't it be matching everything between '^* --' and ':$' ?
shouldn't it be matching everything between ^* -- and :$ ?
Actually, no. You're capturing a group, but it won't print just the group. I suggest using the -P flag to use Perl regex, and look arounds. In your case, this might be enough:
$ cert --help | grep -Po '^\* \K--\w+'
Note that I also used the -o option, to print only the matched content, not the whole line. This eliminates the usage of awk.
A more complete line based on your initial thoughts and more look arounds:
$ cert --help | grep -Po '^\* \K--.*(?=:)'
Edit: as noted in the comments and fine answer by mklement0, this requires GNU grep. You can however do the same with Perl itself, which certainly is probably already installed in your system.
$ cert --help | perl -nle 'print $1 if /^\* (--\w+)/'
This works like a line of code inside a loop. Which is automatically generated by the -nle. -n for the input look, -l for the auto line break, and -e to present the line of code.
The line of Perl code prints the first captured group if the line matches the regex. So it combines ideas from your original solution too.
For a complete POSIX compliant answer, check the answer by mklement0 here in this page.
To provide a POSIX-compliant alternative to sidyll's elegant GNU grep answer (which also explains why the OP's approach didn't work):
Update: Avinash Raj points out in a comment that sed is an option, which indeed allows for a POSIX-compliant single-tool solution: sed allows us to match entire lines of interest and replace them with the contents of a capture group (the part of the line of interest):
puppet cert --help | sed -n 's/^\* \(--.*\):$/\1/p'
Note that since sed is used without the - nonstandard - -r / -E option, a basic regular expression must be used, where ( and ) must be \-escaped to act as capture-group delimiters.
Original answer:
puppet cert --help | egrep '^\* --.+:$' | awk -F '\\* |:' '{print $2}'
Note:
^* was replaced with ^\* so as to ensure that * is matched as a literal, and (.*) was replaced with .+, because (a) there is nothing to be gained by a capture group here, and (b) it's fair to assume that at least one letter follows the --.
-F '\\* |:' uses either literal *<space> or : as the field separator, which ensures that only the --... token (the second field) is printed.

How to retrieve a captured group regex in busybox sed

I'm making a script to boot a TS-7400 ARM SBC and I want it to be able to read some arguments and optional kernel parameters passed via a configuration file found on a SDcard. I called my config file syscfg.conf and it is organized using KEYWORD=value pairs, but since kernel arguments themselves can have the same syntax, I thought of delimiting values like this:
CMDLINE_ARGS="elevator=noop scheduler=noop"
While testing in regular bash, I was able to isolate the kernel command line arguments using either one of these methods:
$ grep CMDLINE_ARGS syscfg.conf | sed 's/CMDLINE_ARGS="\(.*\)"/\1/'
elevator=noop scheduler=noop
$ grep CMDLINE_ARGS syscfg.conf | cut -d'"' -f2
elevator=noop scheduler=noop
$ awk -F'"' '/CMDLINE_ARGS/ {print $2}' syscfg.conf
elevator=noop scheduler=noop
but when it runs on TS-LINUX, which is a busybox-based stripped down Linux used to boot a custom kernel or application, it doesn't work like in regular bash. While the awk command doesn't even exist, the cut version worked fine, but the sed one returns this:
CMDLINE_ARGS="elevator=noop scheduler=noop"
Why is busybox's sed implementation behaving like this? Instead of returning the whole string, I expected it to output only the "\1" group of any characters between the " delimiters - the "(.*)" regex. Is there any way we can make it work like in bash?
sed '/CMDLINE_ARGS=/ {s/CMDLINE_ARGS=.//;s/.$//;}' syscfg.conf
try maybe this if quote are problem or the pipe (i suspect shell substitution in one of those part)

Strange behaviour with command-line perl

I have a file that I'm trying to modify using perl from the terminal in Ubuntu Linux (Natty).
The name of the file is vm.args and the first two lines are as follows:
## Name of the riak node
-name riak#127.0.0.1
I am trying to use perl to update the ip address. Below is my code:
riak_ip=`ifconfig eth1 | grep "inet addr" | cut -d ":" -f2 | cut -d " " -f1`
perl -0777 -i -pe "s/(\-name[\t ]*riak\#)[^\n]+/\1$riak_ip/g" vm.args
Let's assume the ip address I get is 10.181.106.32. The perl command gives me a result I can't understand. The resulting first two lines in the my file after I run the above in the terminal become:
## Name of the riak node
H.181.106.32
Which is the letter H and part of the ip address.
I can't seem to figure out what I'm doing wrong and will appreciate some assistance.
Thanks in advance.
This seems to work reliably:
perl -0777 -i -pe "s/(-name\\s*riak#).*/\${1}$riak_ip/g" vm.args
The "\\1$riak_ip" seems to cause some problems since perl was seeing it as "\1172.20.2.136" if $riak_ip was 172.20.2.136. My guess is that the back reference to "1172" was causing some weirdness. Anyway, switching to the ${1} form removes the possibility for misinterpretation (pun intended).
This really should all be done in Perl, which is much better at extracting data from text than shell script. Something like this should work, but I cannot test it at present.
perl -0777 -i -pe '($ip)=`ifconfig eth1`=~/inet addr:([\d.]+)/;s/-name\s+riak#\K[\d.]+/$ip/g;' vm.args
I would be grateful if someone could confirm whether this works OK. Beware that the \K construct in Perl regexes is a recent addition and may not be in any given installation of Perl.
Problem is that \1 gets concatenated with the first IP octet. To make it work despite concatenation, the ${1} syntax needs to be used and properly quoted. This works:
perl -0777 -i -pe "s/(\-name[\t ]*riak\#)[^\n]+/\${1}$riak_ip/g" vm.args
You might consider to use single quotes for the regex parts, to remove one layer of quoting:
perl -0777 -i -pe 's/(-name[\t ]*riak#)[^\n]+/${1}'"$riak_ip"'/g' vm.args
(Edited/corrected according to comments, my previous suggestion was wrong.)
Sounds like a good use for the \K sequence (v5.10). And [^\n] is actually ., unless the /s modifier is used. No need for /g option unless you intend to replace the string several times.
perl -0777 -i -pe "s/\-name[\t ]*riak\#\K.+/$riak_ip/" vm.args
This would be the correct regexp:
perl -0777 -i -pe "s/(-name\s*riak#)\S+/$1$riak_ip/g" vm.args
Result:
## Name of the riak node
10.181.106.32
Use \s for space characters, and \S (no space character) to match the whole IP address. In the replacement string, $1 is used instead \1. - and # are not special, so there is no need to escape them, although there is nothing bad with them.