shell test operator regular expressions - regex

#!/bin/bash
# This file will fix the cygwin vs linux paths and load programmer's notepad under windows.
# mail : <sandundhammikaperera#gmail.com>
# invokes the GNU GPL, all rights are granted.
# check first parameter is non empty.
# if empty then give a error message and exit.
file=${1:?"Usage: pn filename"};
if [[ "$file" == /*/* ]] ;then
#if long directory name.
# :FAILTHROUGH:
echo "$0: Executing pn.exe $file"
else
file="$(pwd)/$file";
fi
#check whether the filename starts with / if so replace it with appropriate prefix #
prefix="C:/cygwin/";
#check for the partterns starting with "/" #
echo $var | grep "^/*$"
if [[ "$?" -eq "0" ]] ;then
# check again whether parttern starts with /cygdrive/[a-z]/ parttern #
if [[ $file == /cygdrive/[a-z]/* ]] ; then
file=${file#/cygdrive/[a-z]/};
file="C:/"$file;
else
file="$prefix""$file";
fi
fi
#check for the appropriate file permissions #
# :TODO:
echo $file
exec "/cygdrive/c/Program Files (x86)/Programmer's Notepad/pn.exe" $file
as I in my program which convert path names between cygwin and windows and load
the pn.exe [ programmer's notepad in windows]. So my questions are,
There are built in regex expression for the "[[" or 'test' operator. (as well as
I used them in my above program). But why they don't work in here if I change,
echo $var | grep "^/*$"
if [[ "$?" -eq "0" ]] ;then
to this,
if [[ "$file" == ^/*$ ]] ;then
What is the reason for that? Is there any workaround?
I have already tried the second method [[ "$file" == ^/*$ ]] but it didn't work.
then , simple googling brought to me here: http://unix.com/shell-programming
How to find all the documentation about [[ operator or 'test' command? I have used
man test but :(. Which document specifies it's limitations on regex usage if there so.

First, grep "^/*$" will only match paths containing only slashes, like "/", "///", "////". You can use grep "^/" to match paths starting with a slash. If you want to use bash regexes:
var="/some"
#echo $var | grep "^/"
if [[ "$var" =~ ^/ ]] ;then
echo "yes"
fi

Related

Weird regex behavior in bash if condition

I have written a small script that loops through directories (starting from a given argument directory) and prompts directories that have an xml file inside. Here is my code :
#! /bin/bash
process()
{
LIST_ENTRIES=$(find $1 -mindepth 1 -maxdepth 1)
regex="\.xml"
if [[ $LIST_ENTRIES =~ $regex ]]; then
echo "$1"
fi
# Process found entries
while read -r line
do
if [[ -d $line ]]; then
process $line
fi
done <<< "$LIST_ENTRIES"
}
process $1
This code works fine. However, if I change the regex to \.xml$ to indicate that it should match at the end of the line, the result is different, and I do not get all the right directories.
Is there something wrong with this ?
Your variable LIST_ENTRIES may not have .xml as the last entry.
To validate, try echo $LIST_ENTRIES.
To overcome this, use for around your if:
process()
{
LIST_ENTRIES=$(find $1 -mindepth 1 -maxdepth 1)
regex="\.xml$"
for each in $LIST_ENTRIES; do
if [[ $each =~ $regex ]]; then
echo "$1"
fi
done
# Process found entries
while read -r line
do
if [[ -d $line ]]; then
process $line
fi
done <<< "$LIST_ENTRIES"
}
process $1

Using regular expressions in a ksh Script

I have a file (file.txt) that contains some text like:
000000000+000+0+00
000000001+000+0+00
000000002+000+0+00
and I am trying to check each line to make sure that it follows the format:
character*9, "+", character*3, "+", etc
so far I have:
#!/bin/ksh
file=file.txt
line_number=1
for line in $(cat $file)
do
if [[ "$line" != "[[.]]{9}+[[.]]{3}+[[.]]{1}+[[.]]{2} ]" ]]
then
echo "Invalid number ($line) check line $line_number"
exit 1
fi
let "line_number++"
done
however this does not evaluate correctly, no matter what I put in the lines the program terminates.
When you want line numbers of the mismatches, you can use grep -vn. Be careful with writing a correct regular expression, and you will have
grep -Evn "^.{9}[+].{3}[+].[+].{2}$" file.txt
This is not in the layout that you want, so change the layout with sed:
grep -Evn "^.{9}[+].{3}[+].[+].{2}$" file.txt |
sed -r 's/([^:]*):(.*)/Invalid number (\2) check line number \1./'
EDIT:
I changed .{1} into ..
The sed is also over the top. When you need spme explanation, you can start with echo "Linenr:Invalid line"
I'm having funny results putting the regex in the condition directly:
$ line='000000000+000+0+00'
$ [[ $line =~ ^.{9}\+.{3}\+.\+..$ ]] && echo ok
ksh: syntax error: `~(E)^.{9}\+.{3}\+.\+..$ ]] && echo ok
' unexpected
But if I save the regex in a variable:
$ re="^.{9}\+.{3}\+.\+..$"
$ [[ $line =~ $re ]] && echo ok
ok
So you can do
#!/bin/ksh
file=file.txt
line_number=1
re="^.{9}\+.{3}\+.\+..$"
while IFS= read -r line; do
if [[ ! $line =~ $re ]]; then
echo "Invalid number ($line) check line $line_number"
exit 1
fi
let "line_number++"
done < "$file"
You can also use a plain glob pattern:
if [[ $line != ?????????+???+?+?? ]]; then echo error; fi
ksh glob patterns have some regex-like syntax. If there's an optional space in there, you can handle that with the ?(sub-pattern) syntax
pattern="?????????+???+?( )?+??"
line1="000000000+000+0+00"
line2="000000000+000+ 0+00"
[[ $line1 == $pattern ]] && echo match || echo no match # => match
[[ $line2 == $pattern ]] && echo match || echo no match # => match
Read the "File Name Generation" section of the ksh man page.
Your regex looks bad - using sites like https://regex101.com/ is very helpful. From your description, I suspect it should look more like one of these;
^.{9}\+.{3}\+.{1}\+.{2}$
^[^\+]{9}\+[^\+]{3}\+[^\+]{1}\+[^\+]{2}$
^[0-9]{9}\+[0-9]{3}\+[0-9]{1}\+[0-9]{2}$
From the ksh manpage section on [[ - you would probably want to be using =~.
string =~ ere
True if string matches the pattern ~(E)ere where ere is an extended regular expression.
Note: As far as I know, ksh regex doesn't follow the normal syntax
You may have better luck with using grep:
# X="000000000+000+0+00"
# grep -qE "^[^\+]{9}\+[^\+]{3}\+[^\+]{1}\+[^\+]{2}$" <<<"${X}" && echo true
true
Or:
if grep -qE "^[^\+]{9}\+[^\+]{3}\+[^\+]{1}\+[^\+]{2}$" <<<"${line}"
then
exit 1
fi
You may also prefer to use a construct like below for handling files:
while read line; do
echo "${line}";
done < "${file}"

Bash pattern matching inside [[ test block

I am making a script that can do the same task on several machines, like this:
#!/bin/bash
if [[ hostname = login.machine1 ]]; then
#do things how they are done on machine1
...
elif [[ hostname = login.machine2 ]]; then
#etc.
fi
Problem: some machines have several login nodes, like login1.machine2, login2.machine2, etc. I have tried shell pattern matching and regex but none of them work:
if [[ hostname = login?.machine2 ]]
if [[ hostname == login?.machine2 ]]
if [[ hostname =~ login[0-9].machine2 ]]
if [[ hostname =~ login[0-9]\.machine2 ]]
and at least a dozen similar patterns, with or without quotes, etc. What is the correct way to make a test that will return true if hostname = login1.machine2 OR login2.machine2, etc.? Bonus points for something that works with [ instead of [[.
You need a $ before hostname to use the variable.
if [[ $hostname =~ login[0-9]\.machine2 ]]; then
echo "ok"
fi
And for the single [ (sh), the regexp operator =~ was added on bash, so it should not be possible to do it natively with sh.
Is [[ hostname =~ ... ]] supposed to call the command hostname or do you have a variable called hostname? Either use $HOSTNAME or the command substitution $(hostname)
If concerned about portability AWK can be used, with old school command substitutions:
if printf "%s\n" "`hostname`" | awk '!/login[[:digit:]]\.machine2/{exit 1}'; then
printf "Matches...\n"
else
printf "Doesn't match\n"
fi
If not I would highly recommend the answer by naab
The command hostname needs to be wrapped:
if [[ $(hostname) = login?.machine2 ]]

How to write and match regular expressions in /bin/sh script?

I am writing a shell script for a limited unix-based microkernel which doesn't have bash! the /bin/sh can't run the following lines for some reasons.
if [[ `uname` =~ (QNX|qnx) ]]; then
read -p "what is the dev prefix to use? " dev_prefix
if [[ $dev_prefix =~ ^[a-z0-9_-]+#[a-z0-9_-"."]+:.*$ ]]; then
For the 1st and 3rd lines, it complains about missing expression operator, and for the 2nd line it says no coprocess! Can anyone shed light on differences between /bin/bash and /bin/sh scripts?
You can use this equivalent script in /bin/sh:
if uname | grep -Eq '(QNX|qnx)'; then
printf "what is the dev prefix to use? "
read dev_prefix
if echo "$dev_prefix" | grep -Eq '^[a-z0-9_-]+#[a-z0-9_-"."]+:'; then
...
fi
fi
You can use shellcheck to detect non-Posix features in a script:
Copy/Paste this into https://www.shellcheck.net/:
#!/bin/sh
if [[ `1uname` =~ (QNX|qnx) ]]; then
read -p "what is the dev prefix to use? " dev_prefix
if [[ $dev_prefix =~ ^[a-z0-9_-]+#[a-z0-9_-"."]+:.*$ ]]; then
: nothing
fi
fi
Or install shellcheck locally, and run shellcheck ./check.sh,
and it will highlight the non-posix features:
In ./check.sh line 2:
if [[ `1uname` =~ (QNX|qnx) ]]; then
^-- SC2039: In POSIX sh, [[ ]] is not supported.
^-- SC2006: Use $(..) instead of deprecated `..`
In ./check.sh line 4:
if [[ $dev_prefix =~ ^[a-z0-9_-]+#[a-z0-9_-"."]+:.*$ ]]; then
^-- SC2039: In POSIX sh, [[ ]] is not supported.
You either have to rewrite the expressions as globs (not realistic), or use external commands (grep/awk), a explained by #anubhava

bash if [[ =~ regex compare not working?

I have a value in a variable that may be absolute or relative url, and I need to check which one it is.
I have found that there's a =~ operator in [[, but I can't get it to work. What am I doing wrong?
url="http://test"
if [[ "$url" =~ "^http://" ]];
then echo "absolute.";
fi;
You need to use regex without quote:
url="http://test"
if [[ "$url" =~ ^http:// ]]; then
echo "absolute."
fi
This outputs `absolute. as regex needs to be without quote in newer BASH (after BASH v3.1)
Or avoid regex and use glob matching:
if [[ "$url" == "http://"* ]]; then
echo "absolute."
fi