/bin/sh: 1: Syntax error: "(" unexpected after running Makefile [duplicate] - c++

I often find Bash syntax very helpful, e.g. process substitution like in diff <(sort file1) <(sort file2).
Is it possible to use such Bash commands in a Makefile? I'm thinking of something like this:
file-differences:
diff <(sort file1) <(sort file2) > $#
In my GNU Make 3.80 this will give an error since it uses the shell instead of bash to execute the commands.

From the GNU Make documentation,
5.3.2 Choosing the Shell
------------------------
The program used as the shell is taken from the variable `SHELL'. If
this variable is not set in your makefile, the program `/bin/sh' is
used as the shell.
So put SHELL := /bin/bash at the top of your makefile, and you should be good to go.
BTW: You can also do this for one target, at least for GNU Make. Each target can have its own variable assignments, like this:
all: a b
a:
#echo "a is $$0"
b: SHELL:=/bin/bash # HERE: this is setting the shell for b only
b:
#echo "b is $$0"
That'll print:
a is /bin/sh
b is /bin/bash
See "Target-specific Variable Values" in the documentation for more details. That line can go anywhere in the Makefile, it doesn't have to be immediately before the target.

You can call bash directly, use the -c flag:
bash -c "diff <(sort file1) <(sort file2) > $#"
Of course, you may not be able to redirect to the variable $#, but when I tried to do this, I got -bash: $#: ambiguous redirect as an error message, so you may want to look into that before you get too into this (though I'm using bash 3.2.something, so maybe yours works differently).

One way that also works is putting it this way in the first line of the your target:
your-target: $(eval SHELL:=/bin/bash)
#echo "here shell is $$0"

If portability is important you may not want to depend on a specific shell in your Makefile. Not all environments have bash available.

You can call bash directly within your Makefile instead of using the default shell:
bash -c "ls -al"
instead of:
ls -al

There is a way to do this without explicitly setting your SHELL variable to point to bash. This can be useful if you have many makefiles since SHELL isn't inherited by subsequent makefiles or taken from the environment. You also need to be sure that anyone who compiles your code configures their system this way.
If you run sudo dpkg-reconfigure dash and answer 'no' to the prompt, your system will not use dash as the default shell. It will then point to bash (at least in Ubuntu). Note that using dash as your system shell is a bit more efficient though.

It's not a direct answer to the question, makeit is limited Makefile replacement with bash syntax and it can be useful in some cases (I'm the author)
rules can be defined as bash-functions
auto-completion feature
Basic idea is to have while loop in the end of the script:
while [ $# != 0 ]; do
if [ "$(type -t $1)" == 'function' ]; then
$1
else
exit 1
fi
shift
done
https://asciinema.org/a/435159

Related

How to trim last four character from the input in bash shell?

I intend to automate compile and run process in C++, I wrote the following code as compile-run.sh
#! /bin/bash
clang++ $1.cpp -o $1.out && ./$1.out
I put this compile-run.sh in /usr/local/bin for global usage,
and when I type the command compile-run.sh XXX.cpp, it intend to compile and run the specified cpp file. But the problem now is I have to manually removed ".cpp" in the command.
Is there any way to trim the last X number of character and assign to a variable in general?
Is there any way to trim the .cpp and apply trimmed $1 in the code?
Is there better way to automate compile and run process?
well, an ugly way could be by using something like:
#! /bin/bash
filename=$1
temp="${filename%%.cpp}"
clang++ $temp.cpp -o $temp.out && ./$temp.out
another way, if you want to trim the last 4 characters whatever the last part is:
#! /bin/bash
filename=$1
temp="${filename::-4}"
clang++ $temp.cpp -o $temp.out && ./$temp.out
but for substrings you could also use cut: ie. https://stackabuse.com/substrings-in-bash/

sclite (SCTK) `make check` faliure, C++/perl/Cygwin, Safe to use Perl4 stuff?

I am currently trying to install NIST's sclite, which is part of SCTK 2.4.0 (github or newer version). I am attempting the install on Cygwin in bash. The installation is done using make.
I have gotten past the make configure and make all parts of the installation. This didn't come without some effort (See the SO posts on the first (file not recognized) and second (template/scoping) problems). When I get to the make check part of the install, a lot of the checks/tests pass, but then I get the following error.
Testing acomp.pl
No tests defined for acomp.pl
make[2]: Leaving directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src/acomp'
(cd def_art; make check)
make[2]: Entering directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src/def_art'
Testing def_art.pl
def_art.pl passed without tests
make[2]: Leaving directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src/def_art'
(cd hubscr; make check)
make[2]: Entering directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src/hubscr'
Testing hubscr.pl
./RunTests.pl
Running test 'test1-sastt', operation 'test', options '-G -f rttm -F rttm -a', directory 'test1-sastt.test'
Executing command
Error: unable to get the version for program def_art.pl with the command 'def_art.pl' at ../hubscr.pl line 419.
Error: Execution failed at ./RunTests.pl line 30.
make[2]: *** [makefile:20: check] Error 2
make[2]: Leaving directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src/hubscr'
make[1]: *** [makefile:68: checkFast] Error 2
make[1]: Leaving directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src'
make: *** [makefile:52: check] Error 2
I've done some research (described below), and I've been able to get past this problem. However, this involved including some outdated perl modules (Perl4).
My first question was how to fix this error or how to skip that part of the test. I've been able to fix the error, and if people think that it's safe, I'll put it as an answer. Note that there is one more problem with make check after this problem is fixed, but I mention how to get past that at the end.
I'm wondering if using the old Perl (Perl4::CoreLibs) is safe and/or good programming practice. Would it be better to change the source code to use Perl5 stuff?
Is there a better way altogether?
One thing I want to be sure of is that there are no critical tests further down the make check line which might fail.
System Details
$ uname -a
CYGWIN_NT-6.1 CAP-D-ENG-INT3 2.10.0(0.325/5/3) 2018-02-02 15:16 x86_64 Cygwin
$ bash --version
GNU bash, version 4.4.12(3)-release (x86_64-unknown-cygwin) ...
$ gcc --version
gcc (GCC) 6.4.0 ...
$ g++ --version
g++ (GCC) 6.4.0 ...
$ make --version
GNU Make 4.2.1
Built for x86_64-unknown-cygwin ...
$ systeminfo | sed -n 's/^OS\ *//p'
Name: Microsoft Windows 7 Enterprise
Version: 6.1.7601 Service Pack 1 Build 7601
Manufacturer: Microsoft Corporation
Configuration: Member Workstation
Build Type: Multiprocessor Free
My Attempts/Research
From the output above, we have def_art.pl passing the check because there are no checks - "def_art.pl passed without tests". However, the next thing checked, hubscr.pl, failed. The error comes from def_art.pl.
The obvious thing to do seemed to be to run def_art.pl, which I did.
$ ./src/def_art/def_art.pl
Can't locate getopts.pl in #INC
(#INC contains: /usr/local/lib/perl5/site_perl/5.26/x86_64-cygwin-threads /usr/local/share/perl5/site_perl/5.26 /usr/lib/perl5/vendor_perl/5.26/x86_64-cygwin-threads /usr/share/perl5/vendor_perl/5.26 /usr/lib/perl5/5.26/x86_64-cygwin-threads /usr/share/perl5/5.26)
at ./src/def_art/def_art.pl line 40.
So it seems to me that this is a deprecated perl file (or module, or whatever).
I dug a little further and found this discussion on a kaldi discussion from 2014. (kaldi is a speech-recognition toolkit that uses the SCTK scoring system). There are 3 sections of the discussion that I think are especially relevant, which I will link (first, second, third). I'll insert parts here:
def_art.pl is looking for getopts.pl which I coudn't find on my machine!
... [T]hese are legacy packages that are no longer supported in recent versions
of Perl 5. I don't think we should accept a dependency on them. They have
been deprecated since the beginning of Perl 5.
Instead of 'require "getopt.pl"', we should be doing
use Getopt::Std
(note: modern perl code should not call "require" for system packages).
There is a similar issue with "flush.pl" in the Perl scripts. I don't know
what the Perl 5 package name is.
... There are several places where this occurs.
I finally found that both getopts.pl and flush.pl are available from Perl4::CoreLibs. The URL that I use for wget was referenced at this site. Apparently, in other *NIX distros, the package manager can be used, e.g.
apt-get install libperl4-corelibs-perl
or
yum install perl-Perl4-CoreLibs
but I could not find an install via apt-cyg. I was able to install them from a tarball, as described in the What I'm Doing section.
One again, I'll state my main question: Is this safe/good programming practice? Is there a better solution?
If there is a better solution (using Perl 5), it seems that this link might lead the way to it.
Some other links that are possibly related: link_{n} and link{n+1} about flush.pl, link_{n+2} & link_{n+3} about getopts.pl and Perl4::CoreLibs.
What I'm Doing
$ mkdir perl_added
$ cd perl_added
$ wget http://search.cpan.org/CPAN/authors/id/Z/ZE/ZEFRAM/Perl4-CoreLibs-0.004.tar.gz
$ tar -xzf Perl4-CoreLibs-0.004.tar.gz
$ cd Perl4-CoreLibs-0.004
Rather than adding this directory's lib subdirectory to the PERLLIB environment variable with a one-time command-line, environment-variable-addition thing, I did the following.
Make a new directory in the /usr/lib directory, move the files there
$ stat /usr/lib/libperl4-corelibs-perl
stat: cannot stat '/usr/lib/libperl4-corelibs-perl': No such file or directory
# Checked that the directory didn't already exist. It didn't exist.
$ mkdir /usr/lib/libperl4-corelibs-perl
# Make each file executable, then move it into the new directory
# I'd like to come back and explain this.
$ find ./lib -type f -name "*.pl" -print0 | xargs -I'{}' -0 \
bash -c 'new_dir=/usr/lib/libperl4-corelibs-perl/; chmod +x {}; \
mv {} ${new_dir}'
Finally, I made it so that this directory will become part of the perl search path every time I use a terminal by adding the following line to my ~/.bashrc
This command adds the path to the PERLLIB environment variable. Different flavors of Linux have different syntax for adding to environment variables, make sure to find out what yours is!
export PERLLIB="/usr/bin/libperl4-corelibs-perl:$PERLLIB"
The commands I ran for this were
$ echo -e "\n\n## Allow Perl to use the files in Perl4::CoreLibs" >> $HOME/.bashrc
$ echo -e "export PERLLIB=\"/usr/lib/libperl4_corelibs_perl:$PERLLIB\"" >> $HOME/.bashrc
$ source $HOME/.bashrc
(Thanks to #melpomene for noting that the current version is 0.004, not 0.003.)
After that, I went back to the base folder of the install and ran make clean, make config, make all, and make check.
That did get me farther in the make check but not by far.
I'm wondering if using the old Perl (Perl4::CoreLibs) is safe and/or good programming practice. Would it be better to change the source code to use Perl5 stuff?
P.S. After all this, you probably want to go back and delete the folder where you untarred everything. In my case:
rm -rf /path/to/where/I/started/perl_added
The Result/Next Steps
A bunch of tests that passed and then
(cd hubscr; make check)
make[2]: Entering directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src/hubscr'
Testing hubscr.pl
./RunTests.pl
Running test 'test1-sastt', operation 'test', options '-G -f rttm -F rttm -a', directory 'test1-sastt.test'
Executing command
Unescaped left brace in regex is illegal here in regex; marked by <-- HERE in m/{_recursive_/_recur_{ <-- HERE _sive_/_si_ve_}_}/ at ../../md-eval/md-eval.pl line 1099, <DATA> line 12.
Error: MDEVAL failed
Command: md-eval.pl -nafcs -c 0.25 -o -r sastt-case1.ref.rttm.filt -s sastt-case1.sys.rttm.filt -M sastt-case1.sys.rttm.filt.mdeval.spkrmap 1> sastt-case1.sys.rttm.filt.mdeval at ../hubscr.pl line 679.
Error: Execution failed at ./RunTests.pl line 30.
make[2]: *** [makefile:20: check] Error 255
make[2]: Leaving directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src/hubscr'
make[1]: *** [makefile:68: checkFast] Error 2
make[1]: Leaving directory '/cygdrive/c/David/programs/sctk2.4.0/sctk/src'
make: *** [makefile:52: check] Error 2
Maybe this will be helpful. I will post a separate question for this issue or, if the solution is quick, I will add the solution on this post.
A Better Way
(Actually, a couple of better ways. See my comment under the question for the kaldi solution.)
In talking with colleagues and friends, it seems that there isn't anything unsafe about the Perl4 stuff. I did find a better way to get them "installed", but I'll leave the notes in the question showing the "long way" with the tarball, PERLPATH, etc.
Check that you have CPAN
$ which cpan
If you see something starting with which: no cpan in (...), you most likely don't have it. Try installing perl. For me, on Cygwin, I used
$ apt-cyg install perl
(Install apt-cyg if necessary, cf. here for instructions.)
You probably won't have to install Perl. You will likely see something like /usr/bin/cpan as the output of which cpan. If so, you're good. Enter cpan at the command prompt.
$ cpan
If it's your first time, it will ask a bunch of questions about the configuration. I just pressed "Enter" to accept the default each time, I finally got a prompt like this:
cpan shell -- CPAN exploration and modules installation (v2.18)
Enter 'h' for help.
cpan[1]>
There, I entered
cpan[1]> install Perl4::CoreLibs
The install will proceed. When it will have finished, you will be able to type exit and press "Enter", which will take you back to the bash command prompt.
cpan[2]> exit
Lockfile removed.
$
At this point, make check will still choke, but the install will complete successfully. If you want the make check to get all the way through, go to the "Getting past make check" section below. At this point, though, you can do the last two steps in the process.
$ make install
At this point I added the install path to my PATH variable. Hopefully, I'll be able to put in a link about that process. Here is a one-time solution.
$ export PATH=/path/to/sctk/bin:$PATH
Here is a lasting solution.
Now, for the last step in the installation process:
$ make doc
After the make doc, I made sure that the man pages were available. I looked on my machine until I found the place where other man files went. (Sorry, I don't have a systematic way of doing it, I just looked in a lot of places.) For me, on Cygwin, the directory was /usr/man/man1
I went into the doc directory
cd doc
and copied all of the files into the directory I had found
cp -r ./* /usr/man/man1/
Note that there are also now html and htm files in the directory that also provide documentation.
Getting past `make check`
So, you really want to see it go through without errors. You need to change the following file: src/hubscr/RunTests.pl
Originally it has the following beginning, which I have used the head command to show.
$ head -n 15 src/hubscr/RunTests.pl
#!/usr/bin/perl -w
use strict;
my $operation = (defined($ARGV[0]) ? $ARGV[0] : "test");
sub runIt{
my ($op, $testId, $options, $glm, $hub, $lang, $ref, $systems) = #_;
my $baseDir = $testId.".base";
my $outDir = $testId.($op eq "setTests" ? ".base" : ".test");
print " Running test '$testId', operation '$op', options '$options',
directory '$outDir'\n";
system ("mkdir -p $outDir");
system ("rm -fr $outDir/test* $outDir/lvc*");
### Copy files
foreach my $file($glm, $ref, split(/\s+/,$systems)){
system("cp $file $outDir");
Change it so that, after the print command, you have new lines as follows. I again use the head command to show the beginning of the file
$ head -n 63 src/hubscr/RunTests.pl
#!/usr/bin/perl -w
use strict;
my $operation = (defined($ARGV[0]) ? $ARGV[0] : "test");
sub runIt{
my ($op, $testId, $options, $glm, $hub, $lang, $ref, $systems) = #_;
my $baseDir = $testId.".base";
my $outDir = $testId.($op eq "setTests" ? ".base" : ".test");
print " Running test '$testId', operation '$op', options '$options', directory '$outDir'\n";
####DWB, 2018-05-21 Getting `make check` to work####
if ( $testId eq "test1-sastt" &&
$operation eq "test" &&
$options eq "-G -f rttm -F rttm -a" &&
$outDir eq "test1-sastt.test" ) # <problem 1>
{
print "\n";
print "\n#### SKIPPING ####";
print "\nJust kidding. That breaks the make.";
print "\nIt said: \n\n";
print "\nUnescaped left brace in regex is illegal here in regex; marked by <-- HERE in m/{_recursive_/_recur_{ <-- HERE _sive_/_si_ve_}_}/ at ../../md-eval/md-eval.pl line 1099, <DATA> line 12.";
print "\nrror: MDEVAL failed";
print "\nCommand: md-eval.pl -nafcs -c 0.25 -o -r sastt-case1.ref.rttm.filt -s sastt-case1.sys.rttm.filt -M sastt-case1.sys.rttm.filt.mdeval.spkrmap 1> sastt-case1.sys.rttm.filt.mdeval at ../hubscr.pl line 679.";
print "\nError: Execution failed at ./RunTests.pl line 30.\n\n";
print "\n"
print "\nThat's a perl legacy problem, see:"
print "\n[https://unix.stackexchange.com/a/375505/291375][1]"
print "\nI'm outta here.";
print "\n Sincerely, bballdave025";
print "\n";
print "\n";
return;
}#endof: if (<problem 1>)
if ( $testId eq "test2-sastt" &&
$operation eq "test" &&
$options eq "-G -f rttm -F rttm -a" &&
$outDir eq "test2-sastt.test" ) # <problem 2>
{
print "\n";
print "\n#### SKIPPING ####";
print "\nJust kidding. That breaks the make.";
print "\nIt said: \n\n";
print "\nError: Test test2-sastt has failed. Diff output is :";
print "\ndiff -i -x CVS -x .DS_Store -x log -x '*lur' -I '[cC]reation[ _]date' -I md-eval -r test2-sastt.test/sastt-case2.sys.rttm.filt.alignments/segmentgroup-116.html test2-sastt.base/sastt-case2.sys.rttm.filt.alignments/segmentgroup-116.html";
print "\n 45c45";
print "\n < jg.drawStringRect(\"SUB48\",0, 47, scale*656, \"left\");";
print "\n ---";
print "\n#### and a whole bunch of other draw stuff! ####";
print "\n1 at ./RunTests.pl line 61.\n\n";
print "\n"
print "\nThat looks like Java drawing code, and I don't"
print "\neven want to mess with it!"
print "\nI'm outta here.";
print "\n Sincerely, bballdave025";
print "\n";
print "\n";
return;
}#endof: if (<problem 2>)
system ("mkdir -p $outDir")
Now you should be able to get through. Try it:
make check

gvim Regular expression in command Line

I want to apply some gvim regular expression to a file in command line. I understand that we have to use command gvim -c '<regexp>' $filename. But I open the file visually that user can see, is there any way to implement in the background, I mean without opening the file visually?
regards
keerthan
Alternatives
Unless you really need special Vim capabilities, you're probably better off using non-interactive tools like sed, awk, or Perl / Python / Ruby / your favorite scripting language here.
That said, you can use Vim (the terminal version, not GUI-only GVIM) non-interactively:
Silent Batch Mode
For very simple text processing (i.e. using Vim like an enhanced 'sed' or 'awk', maybe just benefitting from the enhanced regular expressions in a :substitute command), use Ex-mode.
REM Windows
call vim -N -u NONE -n -i NONE -es -S "commands.ex" "filespec"
Note: silent batch mode (:help -s-ex) messes up the Windows console, so you may have to do a cls to clean up after the Vim run.
# Unix
vim -T dumb --noplugin -n -i NONE -es -S "commands.ex" "filespec"
Attention: Vim will hang waiting for input if the "commands.ex" file doesn't exist; better check beforehand for its existence! Alternatively, Vim can read the commands from stdin. You can also fill a new buffer with text read from stdin, and read commands from stderr if you use the - argument.
Full Automation
For more advanced processing involving multiple windows, and real automation of Vim (where you might interact with the user or leave Vim running to let the user take over), use:
vim -N -u NONE -n -c "set nomore" -S "commands.vim" "filespec"
Here's a summary of the used arguments:
-T dumb Avoids errors in case the terminal detection goes wrong.
-N -u NONE Do not load vimrc and plugins, alternatively:
--noplugin Do not load plugins.
-n No swapfile.
-i NONE Ignore the |viminfo| file (to avoid disturbing the
user's settings).
-es Ex mode + silent batch mode -s-ex
Attention: Must be given in that order!
-S ... Source script.
-c 'set nomore' Suppress the more-prompt when the screen is filled
with messages or output to avoid blocking.
To modify the file using a regular expression, write it and quit immediately, you could use something like vim -c <command> -c x <file>. From a technical point of view, it is possible to suppress output by redirecting the standard streams and run this in the background – if that is what you want.

GNU Make - Set MAKEFILE variable from shell command output within a rule/target

I'm trying to put together some complicated makefile rules to automate building a project against multiple compilers. I have one rule that creates some dynamically generated variables and assigns variables to them, then passes those variables along to a call to build a separate rule.
.PHONY: all
all:
#echo "Detected CPULIST:${CPULIST_DETECTED}"
#echo "CPULIST:${CPULIST}"
#for cpu in $(CPULIST_DETECTED); do \
echo "CPU:$${cpu}:"; \
eval VARIANTLIST_DETECTED=$(shell 2>&1 find ./.build/linux/$$cpu -mindepth 1 -maxdepth 1 | grep -v '\.svn' | grep -v warning ); \
eval echo "Detected Variant List:$${VARIANTLIST_DETECTED}"; \
eval variant_$${cpu}=$${cpu}; \
eval echo "variant_\$${cpu}:\$${variant_$${cpu}}"; \
$(MAKE) build CPULIST=$${cpu}; \
done
.PHONY: build
build: sanity_check $(TARGET)
#true
I'm having two issues. The first is that, despite double-escaping cpu via $$cpu, it translates to null in the line:
eval VARIANTLIST_DETECTED=$(shell 2>&1 find ./.build/linux/$$cpu -mindepth 1 -maxdepth 1 | grep -v '\.svn' | grep -v warning ); \
So, the find command searches against ./.build/linux/ for each iteration of the loop rather than looping through ./.build/linux/arm and ./.build/linux/x86 like I would expect. I've included a post where this is described in the references below. I suspect this might be causing a problem because I'm attempting this within a rule itself rather than in the global assignment portion of the makefile (before the rules).
The other problem is occurring at the exact same line. It seems that the shell command is evaluated, assigned to VARIANTLIST_DETECTED, but then VARIANTLIST_DETECTED is executed as if it were a command, since I get the following error during build:
Detected CPULIST:arm x86
CPULIST:
CPU:arm:
/bin/sh: 3: ./.build/linux/x86: Permission denied
Detected Variant List:
variant_arm:arm
It's attempting to run the first result in my query as if it were a command. That line should also be something like ./.build/linux/x86/so.
How do I go about resolving these two issues? They are the last two thorns impeding completion of my makefiles.
Thanks!
References
Assign a makefile variable value to a bash command result?, Accessed 2014-06-19, <https://stackoverflow.com/questions/2373081/assign-a-makefile-variable-value-to-a-bash-command-result>
You need to step back a little bit and think about what's going on here: you seem to just be throwing things into your makefile and hoping the result is what you want, rather than really understanding what's happening and proceeding with a purpose to solve your problem.
The first thing to understand is how make invokes a recipe: first, make expands the recipe. Expanding means make looks through the text of the recipe and finds anything that begins with a $ and treats it like a make function or variable, and evaluates it. So, $$ is converted into a single $, $(CPU_LIST_DETECTED) is expanded to the value of that variable, and $(shell ....) is a make function which is run and the results are included in the recipe text. Maybe you can already see where your recipe is going wrong...
Then second, AFTER all of the recipe has been expanded, make takes the expanded text of the recipe and passes it to the shell. Make then waits for the shell to complete, and make looks at the exit code of the shell. If it's 0, then the recipe succeeds and make continues. If it's not 0, then the recipe fails and make exits with an error. There is NO INTERACTION between make and the shell except make starts it with a script to run, and gets back the exit code. If you couldn't before, maybe you can now see where your recipe is going wrong.
It's virtually always an error (or at least unnecessary) to use a $(shell ...) function inside a recipe... the recipe is already running in the shell! You don't need make to run another shell first, then give those results to a new shell.
Your shell invocation is trying to use a shell variable, $cpu. But that variable has no value here, when make runs the $(shell ...) function, because make runs that function before it starts your shell script. You can't have things in your make functions that refer to things that won't be defined until the recipe is running. This will run the shell script 2>&1 find ./.build/linux/$cpu -mindepth 1 -maxdepth 1 | grep -v '\.svn' | grep -v warning, but the shell variable cpu is not set so the find will recurse through everything under ./.build/linux/.
Second, you are misusing eval here. The shell function expands to a list of words, say foo bar baz. Then the eval will look like this:
eval VARIANTLIST_DETECTED=foo bar baz
which is the same as writing just this:
VARIANTLIST_DETECTED=foo bar baz
which is the same as running the command bar with an argument baz, after setting the variable VARIANTLIST_DETECTED to the value foo. Which is why it seems to be "running your directory".
You probably wanted this:
VARIANTLIST_DETECTED="foo bar baz"
(note quotes). However, getting quotes through an eval is tricky" you have to escape them. Luckily you don't need eval at all here anyway, because you're not trying to assign a dynamically named variable. You can just write the assignment directly.
So, losing the shell function and the eval, that line should be written:
VARIANTLIST_DETECTED=`2>&1 find ./.build/linux/$$cpu -mindepth 1 -maxdepth 1 | grep -v '\.svn' | grep -v warning`; \
Ditto for the echo: you don't need the eval there.
However, I really don't see the point of setting this variable or the variant_$cpu variable either. You never do anything with them.
In general, try to avoid putting Make syntax inside a command. Try this:
VARIANTLIST_DETECTED=$$(find ./.build/linux/$${cpu} -mindepth 1 -maxdepth 1 | grep -v '.svn' | grep -v warning );

Grep across the file system has no output in a shell script

I'm trying to create a pre-commit hook in Git that will check for any debugging code and prompt the user to fix it. I have a regex that I'm grepping for (ignore the fact that it won't exclude occurrences in multiline comments!):
grep -IiRn --exclude-dir={node_modules,vendor,public,lib,contrib} --include=\*.{module,inc,install,php,js} -P '^\s*(?!\/\/)\s*(dpm\(|dsm\(|console.log\()' /path/to/code/
This works fine when I run it normally in the console, but when I try it in an executable .sh script it does nothing. None of the following has worked for me:
#!/bin/sh
grep ...
MYVAR =`grep ...` # Note the backticks!
echo $MYVAR
MYVAR =$(grep ...)
echo $MYVAR
MYVAR ="`grep ...`"
echo $MYVAR
I tried doing it with Python and os.system() but that did nothing either. It seems to just have no STDOUT. There's possibly something obvious I'm missing but I'm at a loose end.
Any help would be much appreciated! Thanks.
Edit:
This is the exact script, even though it's at the earliest possible stage due to not being able to actually do the first bit. I've hidden the exact folder names because it's probably best to not share my company's code base on SO ;)
#!/bin/bash
echo "Test!"
ONE=`grep -IiRn --exclude-dir={node_modules,vendor,public,lib,contrib} --include=\*.{module,inc,install,php,js} -P '^\s*(?!\/\/)\s*(dpm\(|dsm\(|console.log\()' /company/projects/company/www/sites/all/modules/custom/`
TWO=$(grep -IiRn --exclude-dir={node_modules,vendor,public,lib,contrib} --include=\*.{coffee} -P '^\s*(?!\#)\s*(dpm\(|dsm\(|console.log)' /company/projects/company/www/sites/all/modules/custom/)
echo $ONE
echo "$TWO"
... and running bash -x pre-commit returns:
ubuntu#ip-12-34-56-78:/company/projects/company/scripts$ bash -x pre-commit
+ echo 'Test!'
Test!
++ grep -IiRn --exclude-dir=node_modules --exclude-dir=vendor --exclude-dir=public --exclude-dir=lib --exclude-dir=contrib '--include=*.module' '--include=*.inc' '--include=*.install' '--include=*.php' '--include=*.js' -P '^\s*(?!\/\/)\s*(dpm\(|dsm\(|console.log\()' /company/projects/company/www/sites/all/modules/custom/
+ ONE='/company/projects/company/www/sites/all/modules/custom/some_module/some_module.report.inc:594: dsm('\''test'\'');
/company/projects/company/www/sites/all/modules/custom/goals_app/goals_app.module:170: console.log(e.stack);
/company/projects/company/www/sites/all/modules/custom/company_usage_reports/js/script.js:300: console.log('\''fetch success'\'');
/company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_change_workgroup.js:19: console.log('\''wtf?'\'');
/company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder_table.js:33: console.log(resp);
/company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder_table.js:39: console.log(ui.placeholder);
/company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_goal_form.js:4: console.log($( ".required" ));
/company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder.js:40: console.log(resp);
/company/projects/company/www/sites/all/modules/custom/company_goals/js/views/goal-list.js:87: console.log(data);'
++ grep -IiRn --exclude-dir=node_modules --exclude-dir=vendor --exclude-dir=public --exclude-dir=lib --exclude-dir=contrib '--include=*.{coffee}' -P '^\s*(?!\#)\s*(dpm\(|dsm\(|console.log)' /company/projects/company/www/sites/all/modules/custom/
+ TWO=
+ echo /company/projects/company/www/sites/all/modules/custom/some_module/some_module.report.inc:594: 'dsm('\''test'\'');' /company/projects/company/www/sites/all/modules/custom/goals_app/goals_app.module:170: 'console.log(e.stack);' /company/projects/company/www/sites/all/modules/custom/company_usage_reports/js/script.js:300: 'console.log('\''fetch' 'success'\'');' /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_change_workgroup.js:19: 'console.log('\''wtf?'\'');' /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder_table.js:33: 'console.log(resp);' /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder_table.js:39: 'console.log(ui.placeholder);' /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_goal_form.js:4: 'console.log($(' '".required"' '));' /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder.js:40: 'console.log(resp);' /company/projects/company/www/sites/all/modules/custom/company_goals/js/views/goal-list.js:87: 'console.log(data);'
/company/projects/company/www/sites/all/modules/custom/some_module/some_module.report.inc:594: dsm('test'); /company/projects/company/www/sites/all/modules/custom/goals_app/goals_app.module:170: console.log(e.stack); /company/projects/company/www/sites/all/modules/custom/company_usage_reports/js/script.js:300: console.log('fetch success'); /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_change_workgroup.js:19: console.log('wtf?'); /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder_table.js:33: console.log(resp); /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder_table.js:39: console.log(ui.placeholder); /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_goal_form.js:4: console.log($( ".required" )); /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder.js:40: console.log(resp); /company/projects/company/www/sites/all/modules/custom/company_goals/js/views/goal-list.js:87: console.log(data);
+ echo ''
... but running it without the -x flag STILL doesn't work.
Edit two:
In case anyone is wondering, my env is as follows...
ubuntu#ip-12-34-56-78:~$ uname -a
Linux ip-12-34-56-78 3.2.0-31-virtual #50-Ubuntu SMP Fri Sep 7 16:36:36 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
ubuntu#ip-12-34-56-78:~$ whereis sh && whereis bash
sh: /bin/sh /bin/sh.distrib /usr/share/man/man1/sh.1.gz
bash: /bin/bash /etc/bash.bashrc /usr/share/man/man1/bash.1.gz
I can't say for sure until you post the actual script you're running, but in your current code snippet have
#!/bin/sh
Depending on your OS, this may be a link to /bin/bash, for example, or it may be the actual Bourne shell, which does not support brace expansion (e.g. {a, b, c}). Even if /bin/sh does point to /bin/bash on your machine, you should only use portable constructs if your shebang is #!/bin/sh (i.e. say what you mean). If you want to use brace expansion in your script, change the shebang to #!/bin/bash.
If you put
set -x
at the top of your script, it will print detailed information that can help with debugging. You can also do this by invoking the shell directly instead of modifying your script, for example
sh -x /path/to/script
or
bash -x /path/to/script
EDIT: On Ubuntu, /bin/sh is dash, the Debian Almquist shell. Like the Bourne shell, dash is fairly restrictive, and does not support brace expansion. See this page for a discussion of portability issues and dash.