Bash script with regex not behaving on Ubuntu - regex

I have a Bash script that is working on my OpenSuSE box, but when copied across to my Ubuntu box, is not working. The script reads in from a file. The file has fields separated by white space (tabs and spaces).
#!/bin/bash
function test1()
{
while read LINE
do
if [[ $LINE =~ "^$" || $LINE =~ "^#.*" ]] ; then
continue;
fi
set -- $LINE
local field1=$1
local field2=$2
done < test.file
}
test1
with test.file containing:
# Field1Header Field2Header
abcdef A-2
ghijkl B-3
There seem to be two problems:
(1) $field2, the one with the hyphen, is blank
(2) The regex to strip out the blank lines and lines that start with # is not working
Anyone know what's wrong? As I said, it works fine on OpenSuSE.
Thanks,
Paul

Apparently, as of bash 3.2 the regular expression should not be quoted. So this should work:
#!/bin/bash
while read LINE
do
if [[ $LINE =~ ^$ || $LINE =~ ^#.* ]] ; then
continue;
fi
set -- $LINE
local field1=$1
local field2=$2
done < test.file
Edit: you should probably use Jo So's answer as it's definitely cleaner. But I was explaining why the regex fails and the reason behind the different behavior between OpenSuse and Ubuntu(different version of bash, very probably)

Quoting is wrong, that probably accounts for the regex failing.
No need to use bashisms.
No need to use set
Try
while read field1 field2 dummy
do
if ! test "${field1%%#*}"
then
continue
fi
# do stuff here
done
EDIT: The obvious version using set
while read -r line
do
if ! test "${line%%#*}"
then
continue
fi
set -- $line
do_stuff_with "$#"
done

On my ubuntu there is no expresion like "=~" for test command. Just use this one:
if [[ $LINE = "" || ${LINE:0:1} = "#" ]] ; then
continue;
fi

Related

Bash regex not recognizing a single space " "

I'm trying to solve a problem that appeared in my script which doesn't let me match the date+time (YYYY-MM-DD HH:MM:SS) inside a for loop
list='"dt_txt":"2022-06-03 21:00:00"},'
regex_datehour='"dt_txt":"([0-9,-]*.[0-9,:]*)'
for i in $list; do
[[ $i =~ $regex_datehour ]] && echo "${BASH_REMATCH[1]}"
done
It seems that the "." between the two pair of brackets it's not recognizing the space! that's because inside of the list, if I replace the empty space between the date and the time by a _, it works as intended! list='"dt_txt":"2022-06-03_21:00:00"},'
desired output:
2022-06-03 21:00:00
what I get:
2022-06-03
The problem here is one that catches a lot of people, and that is whitespace breaking. In the for loop, your $list variable is not quoted, and it contains a space:
$ list='"dt_txt":"2022-06-03 21:00:00"},'
$ for i in $list ; do echo "i = $i" ; done ;
i = "dt_txt":"2022-06-03
i = 21:00:00"},
Make sure to put double-quotes around all strings that contain variables except regexes:
Using an array for list, which is what makes sense when using the for loop from your original code, it would look something like this:
#!/usr/bin/env bash
# filename: re.sh
list=(
'"dt_txt":"2022-06-03 21:00:00"},'
'"dt_txt":"2022-06-03 22:00:00"},'
'"dt_txt":"2022-06-03 23:00:00"},'
)
regex_datehour='"dt_txt":"([0-9,-]*.[0-9,:]*)'
for i in "${list[#]}" ; do
[[ "$i" =~ $regex_datehour ]] && echo "${BASH_REMATCH[1]}"
done
$ ./re.sh
2022-06-03 21:00:00
2022-06-03 22:00:00
2022-06-03 23:00:00

preg_match_all equivalent for BASH?

I have a string like this
foo:collection:indexation [options] [--] <text> <text_1> <text_2> <text_3> <text_4>
And i want to use bash regex to get an array or string that I can split to get this in order to check if the syntax is correct
["text", "text_1", "text_2", "text_3", "text_4"]
I have tried to do this :
COMMAND_OUTPUT=$($COMMAND_HELP)
# get the output of the help
# regex
ARGUMENT_REGEX="<([^>]+)>"
GOOD_REGEX="[a-z-]"
# get all the arguments
while [[ $COMMAND_OUTPUT =~ $ARGUMENT_REGEX ]]; do
ARGUMENT="${BASH_REMATCH[1]}"
# bad syntax
if [[ ! $ARGUMENT =~ $GOOD_REGEX ]]; then
echo "Invalid argument '$ARGUMENT' for the command $FILE"
echo "Must only use characters [a-z:-]"
exit 5
fi
done
But the while does not seem to be appropriate since I always get the first match.
How can I get all the matches for this regex ?
Thanks !
The loop doesn't work because every time you're just testing the same input string against the regexp. It doesn't know that it should start scanning after the match from the previous iteration. You'd need to remove the part of the string up to and including the previous match before doing the next test.
A simpler way is to use grep -o to get all the matches.
$COMMAND_HELP | grep -o "$ARGUMENT_REGEX" | while read ARGUMENT; do
if [[ ! $ARGUMENT =~ $GOOD_REGEX ]]; then
echo "Invalid argument '$ARGUMENT' for the command $FILE"
echo "Must only use characters [a-z:-]"
exit 5
fi
done
Bash doesn't have this directly, but you can achieve a similar effect with a slight modification.
string='foo...'
re='<([^>]+)>'
while [[ $string =~ $re(.*) ]]; do
string=${BASH_REMATCH[2]}
# process as before
done
This matches the regex we want and also everything in the string after the regex. We keep shortening $string by assigning only the after-our-regex portion to it on every iteration. On the last iteration, ${BASH_REMATCH[2]} will be empty so the loop will terminate.

Bash regex does not accept slash

i am pretty new to bash shell scripting (and linux too)... i try to do a simple script which involves some regex for a string given by keyboard from a user.
clear
read -p "Insert e-mail > "
if [[ $REPLY =~ ^[.] ]]
then
echo "ERROR (code 1): e-mail cannot start with \".\""
elif [[ $REPLY =~ .[.]$ ]]
then
echo "ERROR (code 2): e-mail cannot end with \".\""
else
if [[ $REPLY =~ ^[0-9][0-9a-zA-Z!#$%^\&\'*+-]+$ ]] #THIS IS WHERE I NEED HELP
then
echo "Good!"
else
echo "Bad!"
fi
fi
so what i want to do is to make a regex
so that the user cant start with . or end with . (i pretty much did that and its working)...
next what i wanted to do was make the string start with a number and i did that with ^[0-9] (i think this is correct)
and after that..string could be anything like a number 0-9 or letters a-z and A-Z or the next characters: !#$%^&'*+-/
so when user entered 1& (it starts with number and the rest is in the acceptable characters) but it didn't work.. because it need to be \& (at the regex formula).
next the same problem occurred to character ' what i did, was to add again a backslash to regex formula (\') and it worked..
then i tried to do the same with / character (slash character) so what i did was add a backslash / (backslash slash) but when user entered 1/ (it starts with number and the rest are acceptable characters) unfortunately it printed "Bad!" ... it should print Good!..
why is that happening?
i tried \/ and \\/ but still... cant understand why it doesn't work!
Problem is presence of ! in your character class that is doing history expansion.
I suggest declaring your regex beforehand like this:
re="^[0-9][0-9a-zA-Z\!#$%^&/*'+-]+$"
Then use it as:
s='1/'
[[ $s =~ $re ]] && echo "good" || echo "bad"
good
Actually, /s work in character classes just fine:
$ [[ "1/" =~ ^[0-9][/]+$ ]]; echo $?
0

Shell regex to end of line

I have a file like this little example:
# ...
# mode=dev
# ...
Somewhere in this file there is a "variable" within a comment. And i would like to get the value with regex in a Shell script.
My code so far:
#!/bin/bash
conf=$(<"/etc/test.conf") # Get the file content
regex='mode=(.*)$' # Set a regex
if [[ $conf =~ $regex ]]; then # Search for the regex in the file
# We found it, so ...
echo "${BASH_REMATCH[1]}" # ... here is the value
fi
My big problem is, that it will not find the value :(
I tried a lot of different regex expressions and tested them with https://regex101.com/ , but it seems, that the Shell regex interprator is different from pcre and python.
My best solution was to find the mode= and everything after it. So is there a way to get only the value? The start is easy ... find mode=. But how do I say the shell regex to get everything behind mode= until the next linebreak? and not beyond this linebreak?
Something with \n (unix linebreak) and $ (end of string) did not work for me :(
Thanks for the help,
greetings
You can use this regex to get your match:
conf=$(<"/etc/test.conf")
regex=$'mode=([^\n]*)'
[[ $conf =~ $regex ]] && echo "${BASH_REMATCH[1]}"
Output:
dev
Regex $'mode=([^\n]*)' will match literal text mode= followed by 0 or more of any character that is not \n.

In bash how do I match the a string of the form [SOME_ALPHA_NUM_WORD]?

I have tried stuff like =~ "\[[A-Za-z0-9]+\]" which I would expect would work but doesnt. I also tried "[[A-Za-z0-9]+]" and "\[[:alnum:]+\]". What am I doing wrong? Sample line I want to match: [RTNUT18] (I am iterating through a file, some lines are of this form)
This is my code snippet:
while read line;
do
if [[ $line =~ "^\[[A-Za-z0-9]+\]$" ]]; then
echo match
else
echo no match
fi
done < $1
This is a sample file:
[RBPAT7]
Whatever=foo,bla
Otherline
RRR
and I run:
./script.sh thefile.txt
I am not getting a hit on the [RBPAT7] line at all
Stuff like that isn't enough. You must use it in [[.
$ [[ [foo] =~ ^\[[A-Za-z0-9]+\]$ ]] ; echo $?
0
EDIT:
Unlike test, [[ does not need quotes around its arguments. Your code matches nothing, since you can't have " before the beginning of the line, nor " after the end. Remove the quotes.