How to get a global test coverage for a folder - unit-testing

When showing code coverage, go test show code coverage for each package (in percentage).
Is there a way to show a summary for a folder that is taking all subfolder (subpackage) into account?
What I want is a global code coverage percentage for the full project, one number that show code coverage of the folder and all subfolders.

After running:
go test --coverprofile=coverage.out ./...
Run:
go tool cover -func=coverage.out
You will see the total percentage at the end of the result

I found a solution to my problem.
I first run the test on all package and store the test result in a file :
go test --coverprofile=coverage.out ./...
I then run a bash script to calculate my result
#!/usr/bin/env bash
covered=0
total=0
while IFS='' read -r line || [[ -n "$line" ]]; do
IFS=' ' read -r -a array <<< "$line"
total=$(($total+${array[1]}))
if [ "${array[2]}" = "1" ]; then
covered=$(($covered+${array[1]}))
fi
done < "$1"
echo $(awk "BEGIN { pc=100*${covered}/${total}; i=int(pc); print (pc-i<0.5)?i:i+1 }")

You can have the coverage of all sub-packages without any external tools/ with:
go test --coverprofile=coverage.out -coverpkg=./your/package/... ./your/package
-coverpkg pattern1,pattern2,pattern3
Apply coverage analysis in each test to packages matching the patterns.
The default is for each test to analyze only the package being tested.
See 'go help packages' for a description of package patterns.
Sets -cover.
https://golang.org/cmd/go/#hdr-Testing_flags

Related

Bash Regex matches on Ubuntu but not on macOS

I have the following bash script:
#!/bin/bash
git_status="$(git status 2>/dev/null)"
branch_pattern="^(# |)On branch ([^${IFS}]*)"
echo $git_status
echo $branch_pattern
if [[ ${git_status} =~ ${branch_pattern} ]]; then
echo 'hello'
echo $BASH_REMATCH
fi
Here is the output when I run the script on Ubuntu with bash version 4:
On branch master Initial commit Untracked files: (use "git add <file>..." to include in what will be committed) test.sh nothing added to commit but untracked files present (use "git add" to track)
^(# |)On branch ([^ ]*)
hello
On branch master
However, when I run the same script on macOS with bash version 3, the regex does not match, and nothing inside the if block is executed. The rest of the output is identical. What am I missing? Does my regex need to be formatted differently on macOS/in this version of bash? Is there a flag I am missing?
I have seen similar posts about differences in regex behavior across platforms for e.g., the find command, but I have not yet found a post relevant to my issue.
It looks to me like there's a bug in the RE engine in the version of bash that macOS comes with (it's rather old -- 3.2.57). It's something to do with the ^(# |) part -- it doesn't seem to match an empty string at the beginning of the string, as it should. But I found a workaround. Apparently the bug doesn't happen if the ^ is inside the parentheses, like this:
branch_pattern="(^# |^)On branch ([^${IFS}]*)"
BTW, you shouldn't use echo $varname to print the contents of a variable. For one thing, it'll do word splitting (converting all runs of whitespace into single spaces) and wildcard expansion on the value, which can be very confusing/misleading. Try something like printf '<<%q>>\n' "$varname" instead. Its output can be a little cryptic if the variable contains weird characters, but at least it'll make it clear that there are weird things in there.
If you are only trying to get the current git branch name, there is no need for regex. Git already has this built in.
git rev-parse --abbrev-ref HEAD 2>/dev/null
This will print out the current branch name (if any)
If you are in a git repository without any commits, it will only return "HEAD"

How to get all packages' code coverage together in Go?

I have a library consisting of several packages. When running tests, I am using '-cover' flag and its showing the coverage information for each package individually.Like follows:
--- PASS: TestSampleTestSuite (0.00s)
PASS
coverage: 28.7% of statements
ok github.com/path/to/package1 13.021s
? github.com/path/to/package2 [no test files]
=== RUN TestAbc
--- PASS: TestAbc (0.43s)
PASS
coverage: 27.7% of statements
Is there any way to get a full coverage overview easily to get good idea about coverage on the whole project?
Update: Here is the go test command I am using
go test ./... -v -short -p 1 -cover
EDIT: Things have changed since I wrote this answer. See the release notes of Go 1.10: https://golang.org/doc/go1.10#test :
The go test -coverpkg flag now interprets its argument as a
comma-separated list of patterns to match against the dependencies of
each test, not as a list of packages to load anew. For example, go
test -coverpkg=all is now a meaningful way to run a test with coverage
enabled for the test package and all its dependencies. Also, the go
test -coverprofile option is now supported when running multiple
tests.
You can now run
go test -v -coverpkg=./... -coverprofile=profile.cov ./...
go tool cover -func profile.cov
Old answer
Here is a bash script extracted from https://github.com/h12w/gosweep :
#!/bin/bash
set -e
echo 'mode: count' > profile.cov
for dir in $(find . -maxdepth 10 -not -path './.git*' -not -path '*/_*' -type d);
do
if ls $dir/*.go &> /dev/null; then
go test -short -covermode=count -coverprofile=$dir/profile.tmp $dir
if [ -f $dir/profile.tmp ]
then
cat $dir/profile.tmp | tail -n +2 >> profile.cov
rm $dir/profile.tmp
fi
fi
done
go tool cover -func profile.cov

Stop on first test failure with `go test`

How do I have go test several/packages/... stop after the first test failure?
It takes some time to build and execute the rest of the tests, despite already having something to work with.
Go 1.10 add a new flag failfast to go test:
The new go test -failfast flag disables running additional tests after any test fails. Note that tests running in parallel with the failing test are allowed to complete.
https://golang.org/doc/go1.10
However, note this does not work across packages: https://github.com/golang/go/issues/33038
Here's a workaround:
for s in $(go list ./...); do if ! go test -failfast -v -p 1 $s; then break; fi; done
To speed-up the build phase you can run
go test -i several/packages/...
before the tests to build and install packages that are dependencies of the test.
To stop after the first failure you can use something like
go test several/packages/... | grep FAILED | head -n 1

Is it possible to print golang unit test results out to a file?

I run go test in my pkg directory and I get the test results printed to the console as they run, but it would be ideal if I could get them printed to a txt file or even a html file, is it possible to do this? I know you can get coverage reports out from it and generate html files for those which is excellent, but I would have thought it possible to do the same just for the actual results of the tests i.e which tests ran, which passed and which failed. I've been searching the net but even go test help doesn't offer any suggestions for printing results out to a file.
Since I only want to see failed test, I have this script "gt" that I run instead of go test:
go test -coverprofile=coverage.out %*|grep -v -e "^\.\.*$"|grep -v "^$"|grep -v "thus far"
That way, it filters everything but the failed cases.
And you can redirect its content to a file, as mentioned: gt > test.out
It also generates code coverage, which is why I have another script "gc":
grep -v -e " 1$" coverage.out
That way, I don't even wait for a brower to open, I directly see the list of lines which are not yet covered (ie, which don't end with '1' in the coverage.out file)
This will append test results to the test.out file.
go test > test.out
This will overwrite the test results for each test run.
go test |& tee test.out

Grep across the file system has no output in a shell script

I'm trying to create a pre-commit hook in Git that will check for any debugging code and prompt the user to fix it. I have a regex that I'm grepping for (ignore the fact that it won't exclude occurrences in multiline comments!):
grep -IiRn --exclude-dir={node_modules,vendor,public,lib,contrib} --include=\*.{module,inc,install,php,js} -P '^\s*(?!\/\/)\s*(dpm\(|dsm\(|console.log\()' /path/to/code/
This works fine when I run it normally in the console, but when I try it in an executable .sh script it does nothing. None of the following has worked for me:
#!/bin/sh
grep ...
MYVAR =`grep ...` # Note the backticks!
echo $MYVAR
MYVAR =$(grep ...)
echo $MYVAR
MYVAR ="`grep ...`"
echo $MYVAR
I tried doing it with Python and os.system() but that did nothing either. It seems to just have no STDOUT. There's possibly something obvious I'm missing but I'm at a loose end.
Any help would be much appreciated! Thanks.
Edit:
This is the exact script, even though it's at the earliest possible stage due to not being able to actually do the first bit. I've hidden the exact folder names because it's probably best to not share my company's code base on SO ;)
#!/bin/bash
echo "Test!"
ONE=`grep -IiRn --exclude-dir={node_modules,vendor,public,lib,contrib} --include=\*.{module,inc,install,php,js} -P '^\s*(?!\/\/)\s*(dpm\(|dsm\(|console.log\()' /company/projects/company/www/sites/all/modules/custom/`
TWO=$(grep -IiRn --exclude-dir={node_modules,vendor,public,lib,contrib} --include=\*.{coffee} -P '^\s*(?!\#)\s*(dpm\(|dsm\(|console.log)' /company/projects/company/www/sites/all/modules/custom/)
echo $ONE
echo "$TWO"
... and running bash -x pre-commit returns:
ubuntu#ip-12-34-56-78:/company/projects/company/scripts$ bash -x pre-commit
+ echo 'Test!'
Test!
++ grep -IiRn --exclude-dir=node_modules --exclude-dir=vendor --exclude-dir=public --exclude-dir=lib --exclude-dir=contrib '--include=*.module' '--include=*.inc' '--include=*.install' '--include=*.php' '--include=*.js' -P '^\s*(?!\/\/)\s*(dpm\(|dsm\(|console.log\()' /company/projects/company/www/sites/all/modules/custom/
+ ONE='/company/projects/company/www/sites/all/modules/custom/some_module/some_module.report.inc:594: dsm('\''test'\'');
/company/projects/company/www/sites/all/modules/custom/goals_app/goals_app.module:170: console.log(e.stack);
/company/projects/company/www/sites/all/modules/custom/company_usage_reports/js/script.js:300: console.log('\''fetch success'\'');
/company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_change_workgroup.js:19: console.log('\''wtf?'\'');
/company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder_table.js:33: console.log(resp);
/company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder_table.js:39: console.log(ui.placeholder);
/company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_goal_form.js:4: console.log($( ".required" ));
/company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder.js:40: console.log(resp);
/company/projects/company/www/sites/all/modules/custom/company_goals/js/views/goal-list.js:87: console.log(data);'
++ grep -IiRn --exclude-dir=node_modules --exclude-dir=vendor --exclude-dir=public --exclude-dir=lib --exclude-dir=contrib '--include=*.{coffee}' -P '^\s*(?!\#)\s*(dpm\(|dsm\(|console.log)' /company/projects/company/www/sites/all/modules/custom/
+ TWO=
+ echo /company/projects/company/www/sites/all/modules/custom/some_module/some_module.report.inc:594: 'dsm('\''test'\'');' /company/projects/company/www/sites/all/modules/custom/goals_app/goals_app.module:170: 'console.log(e.stack);' /company/projects/company/www/sites/all/modules/custom/company_usage_reports/js/script.js:300: 'console.log('\''fetch' 'success'\'');' /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_change_workgroup.js:19: 'console.log('\''wtf?'\'');' /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder_table.js:33: 'console.log(resp);' /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder_table.js:39: 'console.log(ui.placeholder);' /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_goal_form.js:4: 'console.log($(' '".required"' '));' /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder.js:40: 'console.log(resp);' /company/projects/company/www/sites/all/modules/custom/company_goals/js/views/goal-list.js:87: 'console.log(data);'
/company/projects/company/www/sites/all/modules/custom/some_module/some_module.report.inc:594: dsm('test'); /company/projects/company/www/sites/all/modules/custom/goals_app/goals_app.module:170: console.log(e.stack); /company/projects/company/www/sites/all/modules/custom/company_usage_reports/js/script.js:300: console.log('fetch success'); /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_change_workgroup.js:19: console.log('wtf?'); /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder_table.js:33: console.log(resp); /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder_table.js:39: console.log(ui.placeholder); /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_goal_form.js:4: console.log($( ".required" )); /company/projects/company/www/sites/all/modules/custom/another_module/js/another_module_reorder.js:40: console.log(resp); /company/projects/company/www/sites/all/modules/custom/company_goals/js/views/goal-list.js:87: console.log(data);
+ echo ''
... but running it without the -x flag STILL doesn't work.
Edit two:
In case anyone is wondering, my env is as follows...
ubuntu#ip-12-34-56-78:~$ uname -a
Linux ip-12-34-56-78 3.2.0-31-virtual #50-Ubuntu SMP Fri Sep 7 16:36:36 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
ubuntu#ip-12-34-56-78:~$ whereis sh && whereis bash
sh: /bin/sh /bin/sh.distrib /usr/share/man/man1/sh.1.gz
bash: /bin/bash /etc/bash.bashrc /usr/share/man/man1/bash.1.gz
I can't say for sure until you post the actual script you're running, but in your current code snippet have
#!/bin/sh
Depending on your OS, this may be a link to /bin/bash, for example, or it may be the actual Bourne shell, which does not support brace expansion (e.g. {a, b, c}). Even if /bin/sh does point to /bin/bash on your machine, you should only use portable constructs if your shebang is #!/bin/sh (i.e. say what you mean). If you want to use brace expansion in your script, change the shebang to #!/bin/bash.
If you put
set -x
at the top of your script, it will print detailed information that can help with debugging. You can also do this by invoking the shell directly instead of modifying your script, for example
sh -x /path/to/script
or
bash -x /path/to/script
EDIT: On Ubuntu, /bin/sh is dash, the Debian Almquist shell. Like the Bourne shell, dash is fairly restrictive, and does not support brace expansion. See this page for a discussion of portability issues and dash.