$ reg='(\.js)|(\.txt)|(\.html)$'
$ [[ 'flight_query.jsp' =~ $reg ]]
$ echo $?
0
*.jsp should not be matched based on the regular expression, but actually doesn't.
Any suggestions?
A useful comment was deleted. The comment suggested that operator precedence was the reason why the regular expression was passing. He suggested the following regular expression as a fix.
$ reg='(\.js|\.txt|\.html)$'
$ if [[ 'flight_query.jsp' =~ $reg ]]; then echo 'matches'; else echo "doesn't match"; fi
doesn't match
$ if [[ 'flight_query.js' =~ $reg ]]; then echo 'matches'; else echo "doesn't match"; fi
matches
This regular expression works as well (\.js$)|(\.txt$)|(\.html$).
Related
The following code
number=1
if [[ $number =~ [0-9] ]]
then
echo matched
fi
works. If I try to use quotes in the regex, however, it stops:
number=1
if [[ $number =~ "[0-9]" ]]
then
echo matched
fi
I tried "\[0-9\]", too. What am I missing?
Funnily enough, bash advanced scripting guide suggests this should work.
Bash version 3.2.39.
It was changed between 3.1 and 3.2. Guess the advanced guide needs an update.
This is a terse description of the new
features added to bash-3.2 since the
release of bash-3.1. As always, the
manual page (doc/bash.1) is the place
to look for complete descriptions.
New Features in Bash
snip
f. Quoting the string argument to the
[[ command's =~ operator now forces
string matching, as with the other pattern-matching operators.
Sadly this'll break existing quote using scripts unless you had the insight to store patterns in variables and use them instead of the regexes directly. Example below.
$ bash --version
GNU bash, version 3.2.39(1)-release (i486-pc-linux-gnu)
Copyright (C) 2007 Free Software Foundation, Inc.
$ number=2
$ if [[ $number =~ "[0-9]" ]]; then echo match; fi
$ if [[ $number =~ [0-9] ]]; then echo match; fi
match
$ re="[0-9]"
$ if [[ $number =~ $re ]]; then echo MATCH; fi
MATCH
$ bash --version
GNU bash, version 3.00.0(1)-release (i586-suse-linux)
Copyright (C) 2004 Free Software Foundation, Inc.
$ number=2
$ if [[ $number =~ "[0-9]" ]]; then echo match; fi
match
$ if [[ "$number" =~ [0-9] ]]; then echo match; fi
match
Bash 3.2 introduced a compatibility option compat31 which reverts bash regular expression quoting behavior back to 3.1
Without compat31:
$ shopt -u compat31
$ shopt compat31
compat31 off
$ set -x
$ if [[ "9" =~ "[0-9]" ]]; then echo match; else echo no match; fi
+ [[ 9 =~ \[0-9] ]]
+ echo no match
no match
With compat31:
$ shopt -s compat31
+ shopt -s compat31
$ if [[ "9" =~ "[0-9]" ]]; then echo match; else echo no match; fi
+ [[ 9 =~ [0-9] ]]
+ echo match
match
Link to patch:
http://ftp.gnu.org/gnu/bash/bash-3.2-patches/bash32-039
GNU bash, version 4.2.25(1)-release (x86_64-pc-linux-gnu)
Some examples of string match and regex match
$ if [[ 234 =~ "[0-9]" ]]; then echo matches; fi # string match
$
$ if [[ 234 =~ [0-9] ]]; then echo matches; fi # regex natch
matches
$ var="[0-9]"
$ if [[ 234 =~ $var ]]; then echo matches; fi # regex match
matches
$ if [[ 234 =~ "$var" ]]; then echo matches; fi # string match after substituting $var as [0-9]
$ if [[ 'rss$var919' =~ "$var" ]]; then echo matches; fi # string match after substituting $var as [0-9]
$ if [[ 'rss$var919' =~ $var ]]; then echo matches; fi # regex match after substituting $var as [0-9]
matches
$ if [[ "rss\$var919" =~ "$var" ]]; then echo matches; fi # string match won't work
$ if [[ "rss\\$var919" =~ "$var" ]]; then echo matches; fi # string match won't work
$ if [[ "rss'$var'""919" =~ "$var" ]]; then echo matches; fi # $var is substituted on LHS & RHS and then string match happens
matches
$ if [[ 'rss$var919' =~ "\$var" ]]; then echo matches; fi # string match !
matches
$ if [[ 'rss$var919' =~ "$var" ]]; then echo matches; fi # string match failed
$
$ if [[ 'rss$var919' =~ '$var' ]]; then echo matches; fi # string match
matches
$ echo $var
[0-9]
$
$ if [[ abc123def =~ "[0-9]" ]]; then echo matches; fi
$ if [[ abc123def =~ [0-9] ]]; then echo matches; fi
matches
$ if [[ 'rss$var919' =~ '$var' ]]; then echo matches; fi # string match due to single quotes on RHS $var matches $var
matches
$ if [[ 'rss$var919' =~ $var ]]; then echo matches; fi # Regex match
matches
$ if [[ 'rss$var' =~ $var ]]; then echo matches; fi # Above e.g. really is regex match and not string match
$
$ if [[ 'rss$var919[0-9]' =~ "$var" ]]; then echo matches; fi # string match RHS substituted and then matched
matches
$ if [[ 'rss$var919' =~ "'$var'" ]]; then echo matches; fi # trying to string match '$var' fails
$ if [[ '$var' =~ "'$var'" ]]; then echo matches; fi # string match still fails as single quotes are omitted on RHS
$ if [[ \'$var\' =~ "'$var'" ]]; then echo matches; fi # this string match works as single quotes are included now on RHS
matches
As mentioned in other answers, putting the regular expression in a variable is a general way to achieve compatibility over different bash versions. You may also use this workaround to achieve the same thing, while keeping your regular expression within the conditional expression:
$ number=1
$ if [[ $number =~ $(echo "[0-9]") ]]; then echo matched; fi
matched
$
Using a local variable has slightly better performance than using command substitution.
For larger scripts, or collections of scripts, it might make sense to use a utility to prevent unwanted local variables polluting the code, and to reduce verbosity. This seems to work well:
# Bash's built-in regular expression matching requires the regular expression
# to be unqouted (see https://stackoverflow.com/q/218156), which makes it harder
# to use some special characters, e.g., the dollar sign.
# This wrapper works around the issue by using a local variable, which means the
# quotes are not passed on to the regex engine.
regex_match() {
local string regex
string="${1?}"
regex="${2?}"
# shellcheck disable=SC2046 `regex` is deliberately unquoted, see above.
[[ "${string}" =~ ${regex} ]]
}
Example usage:
if regex_match "${number}" '[0-9]'; then
echo matched
fi
For some reason, the following regular expression match doesn't seem to be working.
string="#Hello world";
[[ "$string" =~ 'ello' ]] && echo "matches";
[[ "$string" =~ 'el.o' ]] && echo "matches";
The first command succeeds (as expected), but the second one does not.
Shouldn't that period be treated by the regular expression as a single character?
Quoting the period causes it to be treated as a literal character, not a regular-expression metacharacter. Best practice if you want to quote the entire regular expression is to do so in a variable, where regular expression matching rules aren't in effect, then expand the parameter unquoted (which is safe to do inside [[ ... ]]).
regex='el.o'
[[ "$string" =~ $regex ]] && echo "matches"
string="#Hello world";
[[ "$string" =~ ello ]] && echo "matches";
[[ "$string" =~ el.o ]] && echo "matches";
Test
$ string="hh elxo fj"
$ [[ "$string" =~ el.o ]] && echo "matches";
matches
In a list of files:
javascript-custom-rules-plugin-1.0-SNAPSHOT.jar
README.txt
sonar-build-breaker-plugin-2.0.jar
sonar-javascript-plugin-2.11.jar
tmo-custom-rules-1.0.jar
I am attempting to match these filenames by regex.
My Script
#!/usr/bin/env bash
install_location=/usr/local/sonar/extensions/plugins
for f in $(ls -1 $install_location)
do
# remove any previous versions of this plugin
if [[ "$f" =~ ".*tmo-custom-rules-(.+)\.jar" ]]
then
echo "found $f. will remove"
else
echo "$f doesn't match"
fi
done
I've tried if [[ "$f" =~ ".*tmo-custom-rules-(.+)\.jar" ]] and if [[ "$f" == *"tmo-custom-rules" ]] to no avail.
I'm getting
javascript-custom-rules-plugin-1.0-SNAPSHOT.jar doesn't match
README.txt doesn't match
sonar-build-breaker-plugin-2.0.jar doesn't match
sonar-javascript-plugin-2.11.jar doesn't match
tmo-custom-rules-1.0.jar doesn't match
when I expect found tmo-custom-rules-1.0.jar. will remove
I've run my regular expression through many regular expression testers with the data above, and they all return the correct matches, but I can't get it to work here in my script.
How can I loop through, and check to see if any files matches this regular expression?
In BASH regex must be unquoted so this should work:
[[ $f =~ .*tmo-custom-rules-(.+)\.jar ]]
Or better:
re=".*tmo-custom-rules-(.+)\.jar"
[[ $f =~ $re ]]
However you don't even need regex and can use shell glob matching:
#!/usr/bin/env bash
install_location=/usr/local/sonar/extensions/plugins
for f in "$install_location"/*
do
# remove any previous versions of this plugin
if [[ $f == *tmo-custom-rules-*.jar ]]
then
echo "found $f. will remove"
else
echo "$f doesn't match"
fi
done
Note that you can avoid using output of ls which is not always fit for scripting.
You can do this with expr using the colon operator:
if expr "$f" : '.*tmo-custom-rules-.*\.jar' > /dev/null; then
echo matches
fi
Note that the regular expressions in this context are assumed to be anchored to the start of the line.
I can run a while loop with regex successfully
$ cat while.sh
#!/bin/sh
arr=(a1c a2c a3c b4c)
i=0
while [[ ${arr[i]} =~ a(.)c ]]
do
echo ${BASH_REMATCH[1]}
((i++))
done
$ ./while.sh
1
2
3
A for loop causes this error
$ cat for.sh
#!/bin/sh
arr=(a1c a2c a3c b4c)
for ((i=0; [[ ${arr[i]} =~ a(.)c ]]; i++))
do
echo ${BASH_REMATCH[1]}
done
$ ./for.sh
./for.sh: line 3: ((: [[ a1c =~ a(.)c ]]: syntax error: operand expected (error
token is "[[ a1c =~ a(.)c ]]")
To follow-up on my comment above, if you wanted to keep your formatting, more or less, this might do what you expected, because you need the return value of the evaluation as an expression in the for loop, not the output of it.
for ((i=0; `[[ ${arr[i]} =~ a(.)c ]] && echo -n 1`; i++)); do
# do whatever
done
Ugly, but worked for me, and should explain your error for you. The back-ticks evaluate the expression then output either a '1' for true or nothing for false. This leaves you the valid conditional for the loop in the middle.
I'm not sure your for loop construct is legal with regex. Double parenthesis are for Arithmetic expressions and that includes for loops. Regex matching is not arithmetic. I think if you were really set on using for for some reason, you would have to do something like:
arr=(a1c a2c a3c b4c)
for val in "${arr[#]}"; do
if [[ $val =~ a(.)c ]]; then
echo ${BASH_REMATCH[1]}
fi
done
The following code
number=1
if [[ $number =~ [0-9] ]]
then
echo matched
fi
works. If I try to use quotes in the regex, however, it stops:
number=1
if [[ $number =~ "[0-9]" ]]
then
echo matched
fi
I tried "\[0-9\]", too. What am I missing?
Funnily enough, bash advanced scripting guide suggests this should work.
Bash version 3.2.39.
It was changed between 3.1 and 3.2. Guess the advanced guide needs an update.
This is a terse description of the new
features added to bash-3.2 since the
release of bash-3.1. As always, the
manual page (doc/bash.1) is the place
to look for complete descriptions.
New Features in Bash
snip
f. Quoting the string argument to the
[[ command's =~ operator now forces
string matching, as with the other pattern-matching operators.
Sadly this'll break existing quote using scripts unless you had the insight to store patterns in variables and use them instead of the regexes directly. Example below.
$ bash --version
GNU bash, version 3.2.39(1)-release (i486-pc-linux-gnu)
Copyright (C) 2007 Free Software Foundation, Inc.
$ number=2
$ if [[ $number =~ "[0-9]" ]]; then echo match; fi
$ if [[ $number =~ [0-9] ]]; then echo match; fi
match
$ re="[0-9]"
$ if [[ $number =~ $re ]]; then echo MATCH; fi
MATCH
$ bash --version
GNU bash, version 3.00.0(1)-release (i586-suse-linux)
Copyright (C) 2004 Free Software Foundation, Inc.
$ number=2
$ if [[ $number =~ "[0-9]" ]]; then echo match; fi
match
$ if [[ "$number" =~ [0-9] ]]; then echo match; fi
match
Bash 3.2 introduced a compatibility option compat31 which reverts bash regular expression quoting behavior back to 3.1
Without compat31:
$ shopt -u compat31
$ shopt compat31
compat31 off
$ set -x
$ if [[ "9" =~ "[0-9]" ]]; then echo match; else echo no match; fi
+ [[ 9 =~ \[0-9] ]]
+ echo no match
no match
With compat31:
$ shopt -s compat31
+ shopt -s compat31
$ if [[ "9" =~ "[0-9]" ]]; then echo match; else echo no match; fi
+ [[ 9 =~ [0-9] ]]
+ echo match
match
Link to patch:
http://ftp.gnu.org/gnu/bash/bash-3.2-patches/bash32-039
GNU bash, version 4.2.25(1)-release (x86_64-pc-linux-gnu)
Some examples of string match and regex match
$ if [[ 234 =~ "[0-9]" ]]; then echo matches; fi # string match
$
$ if [[ 234 =~ [0-9] ]]; then echo matches; fi # regex natch
matches
$ var="[0-9]"
$ if [[ 234 =~ $var ]]; then echo matches; fi # regex match
matches
$ if [[ 234 =~ "$var" ]]; then echo matches; fi # string match after substituting $var as [0-9]
$ if [[ 'rss$var919' =~ "$var" ]]; then echo matches; fi # string match after substituting $var as [0-9]
$ if [[ 'rss$var919' =~ $var ]]; then echo matches; fi # regex match after substituting $var as [0-9]
matches
$ if [[ "rss\$var919" =~ "$var" ]]; then echo matches; fi # string match won't work
$ if [[ "rss\\$var919" =~ "$var" ]]; then echo matches; fi # string match won't work
$ if [[ "rss'$var'""919" =~ "$var" ]]; then echo matches; fi # $var is substituted on LHS & RHS and then string match happens
matches
$ if [[ 'rss$var919' =~ "\$var" ]]; then echo matches; fi # string match !
matches
$ if [[ 'rss$var919' =~ "$var" ]]; then echo matches; fi # string match failed
$
$ if [[ 'rss$var919' =~ '$var' ]]; then echo matches; fi # string match
matches
$ echo $var
[0-9]
$
$ if [[ abc123def =~ "[0-9]" ]]; then echo matches; fi
$ if [[ abc123def =~ [0-9] ]]; then echo matches; fi
matches
$ if [[ 'rss$var919' =~ '$var' ]]; then echo matches; fi # string match due to single quotes on RHS $var matches $var
matches
$ if [[ 'rss$var919' =~ $var ]]; then echo matches; fi # Regex match
matches
$ if [[ 'rss$var' =~ $var ]]; then echo matches; fi # Above e.g. really is regex match and not string match
$
$ if [[ 'rss$var919[0-9]' =~ "$var" ]]; then echo matches; fi # string match RHS substituted and then matched
matches
$ if [[ 'rss$var919' =~ "'$var'" ]]; then echo matches; fi # trying to string match '$var' fails
$ if [[ '$var' =~ "'$var'" ]]; then echo matches; fi # string match still fails as single quotes are omitted on RHS
$ if [[ \'$var\' =~ "'$var'" ]]; then echo matches; fi # this string match works as single quotes are included now on RHS
matches
As mentioned in other answers, putting the regular expression in a variable is a general way to achieve compatibility over different bash versions. You may also use this workaround to achieve the same thing, while keeping your regular expression within the conditional expression:
$ number=1
$ if [[ $number =~ $(echo "[0-9]") ]]; then echo matched; fi
matched
$
Using a local variable has slightly better performance than using command substitution.
For larger scripts, or collections of scripts, it might make sense to use a utility to prevent unwanted local variables polluting the code, and to reduce verbosity. This seems to work well:
# Bash's built-in regular expression matching requires the regular expression
# to be unqouted (see https://stackoverflow.com/q/218156), which makes it harder
# to use some special characters, e.g., the dollar sign.
# This wrapper works around the issue by using a local variable, which means the
# quotes are not passed on to the regex engine.
regex_match() {
local string regex
string="${1?}"
regex="${2?}"
# shellcheck disable=SC2046 `regex` is deliberately unquoted, see above.
[[ "${string}" =~ ${regex} ]]
}
Example usage:
if regex_match "${number}" '[0-9]'; then
echo matched
fi