How to measure test coverage in Go - unit-testing

Has anyone succeeded in generating code coverage for Go unit tests? I can't find a tool for that on the web.

Note that Go 1.2 (Q4 2013, rc1 is available) will now display test coverage results:
One major new feature of go test is that it can now compute and, with help from a new, separately installed "go tool cover" program, display test coverage results.
The cover tool is part of the go.tools subrepository. It can be installed by running
$ go get golang.org/x/tools/cmd/cover
The cover tool does two things.
First, when "go test" is given the -cover flag, it is run automatically to rewrite the source for the package and insert instrumentation statements. The test is then compiled and run as usual, and basic coverage statistics are reported:
$ go test -coverprofile fmtcoverage.html fmt
ok fmt 0.060s coverage: 91.4% of statements
$
Second, for more detailed reports, different flags to "go test" can create a coverage profile file, which the cover program, invoked with "go tool cover", can then analyze.
Frank Shearar mentions:
The latest versions of Go (2013/09/19) use:
go test -coverprofile <filename> <package name>
Details on how to generate and analyze coverage statistics can be found by running the commands
$ go help testflag
$ go tool cover -help
Ivan Black mentions in the comments:
go test -coverprofile cover.out and then
go tool cover -html=cover.out opens cover.out in your default browser
I don't even want to wait for the browser to open, so I defined this alias:
alias gc=grep -v -e " 1$" cover.out
That I just type gc, and have a list of all the lines not yet covered (here: with a coverage.out line not ending with " 1").
Update 2022, possibly for Go 1.19
proposal: extend Go's code coverage testing to include applications
While the existing "go test" based coverage workflow will continue to be supported, the proposal is to add coverage as a new build mode for "go build".
In the same way that users can build a race-detector instrumented executable using "go build -race", it will be possible to build a coverage-instrumented executable using "go build -cover".
Merging coverage profiles produced in different GOOS/GOARCH environments will be supported.

Go comes with awesome tool for testing and coverage. Although all Go tools are well documented go tool cover -help I would suggest reading The cover story article on the official Go blog. It has plenty of examples and I strongly recommend it!
I have this function in my ~/.bash_profile. (you can just paste it in the terminal to give it a try).
cover () {
t="/tmp/go-cover.$$.tmp"
go test -coverprofile=$t $# && go tool cover -html=$t && unlink $t
}
Then just cd into a go project/package folder and type cover.
This opens a visual tool in browser which shows you the tested and untested code for each file in the current package. Very useful command! I strongly recommend it for finding what is not 100% tested yet! The shown results are per file. From a drop down in top-left you can see results for all files.
With this command you can also check the coverage of any package for example:
cover fmt
The output in terminal from this command would be:
ok fmt 0.031s coverage: 91.9% of statements
In addition to that in your browser you will see this tool showing in red all lines of code which are not covered with tests:
It is also possible to just save the html coverage file instead of opening it in a browser. This is very useful in cases when your tests + coverage is run by CI tool like Jenkins. That way you can serve the coverage files from a central server and the whole team will be able to see the coverage results for each build.

In addition to the good answers above, I find these three lines to be the simplest way to get it (which includes all packages):
go test -v -coverprofile cover.out ./YOUR_CODE_FOLDER/...
go tool cover -html cover.out -o cover.html
open cover.html
Note that in the HTML file you will find a dropdown button that will direct you to all files.
See go tool cover -help for additional options.

Simply run
go test -cover
or
go test -cover ./...
or
go test -coverprofile=coverage.out ./... ; go tool cover -func=coverage.out
or to check the source code
go test -coverprofile=coverage.out ./... ; go tool cover -html=coverage.out

I can't find a tool for that on the web.
Actually... there is now (2022) such a tool on the web, from Nikolay Dubina's project go-cover-treemap-web:
https://go-cover-treemap.io/
Nothing it uploaded (the processing remains local), but by dragging/dropping your coverprofile, the Web UI (using Go WASM) will run go-cover-treemap and display:
(gocovergage for https://github.com/gohugoio/hugo)

Coverage Report:
a) Run all the tests and enable coverage --> go test ./... -coverprofile coverage.out
b) Get coverage for individual functions as well as overall coverage → go tool cover -func coverage.out
c) See the lines covered and the ones not covered by your tests → go tool cover -html=coverage.out -o coverage.html. Open the coverage.html file hereby generated in the browser and analyze the detailed coverage info.

Already built-in in VSCode
Ctrl+Shift+P to Open Command Palette
Go: Toggle Test Coverage ...
The Green part is tested and Red is not

It's right here, some docs here.
$ go tool
6a
6c
6g
6l
addr2line
api
cgo
cov
dist
ebnflint
fix
gotype
nm
objdump
pack
pprof
prof
vet
yacc
$ go tool cov -h
usage: cov [-lsv] [-g substring] [-m minlines] [6.out args...]
-g specifies pattern of interesting functions or files
go tool cov: exit status 1
$
I haven't used it, this is all I know.

If you like to see the uncovered lines by function directly in a terminal I rewrote the cover tool for this purpose. It's available at https://github.com/gregoryv/uncover.
Usage
go get -u github.com/gregoryv/uncover/...
go test -coverprofile /tmp/c.out
uncover /tmp/c.out
Screenshot

If you are using VSCode this functionality is supported out the box ( But disabled by default )
Just turn on test on save + coverage reporting
https://github.com/microsoft/vscode-go/wiki/On-Save-features
It will even show in your editor which lines are not covered which is super handy.

If you want to find test coverage in Windows, just go to the desired folder in command prompt and type the following command:
go test -coverprofile=coverage.out && go tool cover -html=coverage.out
This is perfectly easy and works reliably.

Inspired by the help menus and other answers to this question, just run:
f=cover.out
if [ -f $f ]; then
rm $f
fi
go test ./... -coverprofile $f && \
go tool cover -html $f && \
rm $f

A quick and easy way is to use the coverage tool that comes with built-in go :
$ go test -coverprofile cp.out
// Emits the coverage in one liner percentage wise
After you execute the above command, if you wish to visually see the code coverage (like covered statements and missed etc)
$ go tool cover -html=cp.out
Note : You need to execute the above commands in the folder where you wish to see coverage

Try using gaia-docker/base-go-build Docker Image.
This is a Docker image that contains all you need in order to build and test coverage.
Running test coverage inside a Docker container creates .cover folder with test coverage results of your project.
docker run --rm -v "$PWD":$PROJECT_PATH -w $PROJECT_PATH $BUILDER_IMAGE_NAME /go/script/coverage.sh
The test coverage script running on all projects' folders and generates, inside .cover folder junit and coverage reports for each folder, and a combine coverage report of all projects' tests.
Codecov also suggests a script that collect coverage results: multiple files

Test Coverage for Golang
go get github.com/axw/gocov/gocov
go get -u gopkg.in/matm/v1/gocov-html
Check It is Installed Correctly And you have access from your Terminal
Run the Test Case
If you run the test case it will Reder the .json File Based on the file you will get the Code Coverage Report in .html file
gocov test >your_Coverage_report.json
Once Your Test case is done Generate a Report in .html File using .json
gocov-html your_Coverage_report.json >your_Coverage_report.html
Reference
GoTest Coverage Tool for go lang
Go Test Report Tool
Alternate Method
Go Native Test coverage
go test -coverprofile=coverage.out
go tool cover -html=coverage.out

Related

Testing generated Go code without co-locating tests

Have some auto-generated golang code for protobuf messages and I'm looking to add some additional testing, without locating the file under the same directory path. This is to allow easy removal of the existing generated code to be sure that if a file is dropped from being generated, it's not left included in the codebase by accident.
The current layout of these files are controlled by prototool so I have something like the following:
/pkg/<other1>
/pkg/<other2>
/pkg/<name-generated>/v1/component_api.pb.go
/pkg/<name-generated>/v1/component_api.pb.gw.go
/pkg/<name-generated>/v1/component_api.pb.validate.go
The *.validate.go comes from envoyproxy/protoc-gen-validate, and *.pb.go & *.pb.gw.go are coming from protobuf and grpc libraries. The other1 and other2 are two helper libraries that we have included along with the generated code to make it easier for client side apps. The server side is in a separate repo and imports as needed.
Because it's useful to be able to delete /pkg/<name> before re-running prototool I've placed some tests of component_api (mostly to exercise the validate rules generated automatically) under the path:
/internal/pkg/<name>/v1/component_api_test.go
While this works for go test -v ./..., it appears not to work to well when generating coverage with -coverpkg.
go test -coverpkg=./... -coverprofile=coverage/go/coverage.out -v ./...
go build <pkgname>/internal/pkg/<name>/v1: no non-test Go files in ....
<output from the tests in /internal/pkg/<name>/v1/component_api_test.go>
....
....
coverage: 10.5% of statements in ./...
ok <pkgname>/internal/pkg/<name>/v1 0.014s coverage: 10.5% of statements in ./...
FAIL <pkgname>/pkg/other1 [build failed]
FAIL <pkgname>/pkg/other2 [build failed]
? <pkgname>/pkg/<name>/v1 [no test files]
FAIL
Coverage tests failed
Generated coverage/go/html/main.html
The reason for use of -coverpkg is that without it there doesn't seem to be anything spotting that any of the code under <pkgname>/pkg/<name>/v1 is covered, and we've see issues with what it reports previously not showing the real level of coverage, which are solved by use of -coverpkg:
go test -cover -coverprofile=coverage/go/coverage.out ./...
ok <pkgname>internal/pkg/<name>/v1 0.007s coverage: [no statements]
ok <pkgname>/pkg/other1 0.005s coverage: 100.0% of statements
ok <pkgname>/pkg/other2 0.177s coverage: 100.0% of statements
? <pkgname>/pkg/<name>/v1 [no test files]
Looking at the resulting coverage/go/coverage.out includes no mention of anything under <pkgname>/pkg/<name>/v1 being exercised.
I'm not attached to the current layout beyond being limited on <pkgname>/pkg/<name>/v1 being automatically managed by prototool and it's rules around naming for the generated files. Would like to ensure the other modules we have can remain exported to be used as helper libraries and I would like to be able to add tests for <pkgname>/pkg/<name>/v1 without needing to locate them in the same directory to allow for easy delete + recreate of generated files, while still getting sensible coverage reports.
I've tried fiddling with the packages passed to -coverpkg and replacing ./... on the command-line and haven't been able to come up with something that works. Perhaps I'm just not familiar with the right invocation?
Other than that is there a different layout that will take care of this for me?
To handle this scenario, simply create a doc.go file in the same directory as the dis-located tests with just the package and comment. This will allow the standard arguments to work and golang appears to be reasonably happy with an empty file.
Once in place the following will work as expected.
go test -coverpkg=./... -coverprofile=coverage/go/coverage.out -v ./...
Idea based on suggestion in https://stackoverflow.com/a/47025370/1597808

Temporarily get jest coverage to show only files in a specific folder

For my reactjs app, I'm adding integration tests to a view and want to keep an eye on coverage along the way. I'm currently getting all files on every run.
Question
While adding tests to increase coverage, how can I get jest coverage to show only files in a specific folder?
Examples tried
$ yarn test --collectCoverageFrom=src/app/components/Tools
Test run, bu no coverage is showing here.
$ yarn test Tools --coverage --collectCoverageFrom=src/app/components/Tools
I get Ran all test suites matching "Tools".
$ yarn test src/app/components/Tools --coverage
Here I see the coverage percentage is smaller but still lists all files.
$ yarn test -o --coverage
Again as the previous, I see the coverage percentage is smaller but still lists all files.
It would be best to have an argument in the CLI, but you can add a temporary script in your package.json:
"tools-coverage": "react-app-rewired test --env=jsdom src/app/components/Tools --coverage --collectCoverageFrom=src/app/components/Tools/**/*.js"
Running the full command in the terminal just doesn't work.
Now you can instead run $ yarn tools-coverage to just test your Tools folder you targeted in your package.json.
An example shown here in the create-react-app issue:
https://github.com/facebook/create-react-app/issues/1455#issuecomment-277010725

WebStorm run all dart unit tests

In WebStorm 11 I want to create a run configuration which runs all dart tests in my project.
However there is no option to do this in the "Dart Test" configuration template. The only options are:
Test Kind: All in file, Test group, single test
Test file: must point to a .dart file, otherwise I get "Dart file is not found"
VM Options (text input)
If I point WebStorm to a single test file this command gets executed in the test window:
C:\path\to\dart\bin\dart.exe --ignore-unrecognized-flags --checked --trace_service_pause_events file:\\\C:\path\to\dart\bin\snapshots\pub.dart.snapshot run test:test -r json C:/path/to/project/test/someclass_test.dart
I don't want to create a run configuration for every unit test class I write, there must be a better way.
Currently I prefer to navigate to the project directory and just run
pub run test:test
This runs all tests which live in files ending with _test.dart which is perfectly what I want. More info here: https://github.com/dart-lang/test#running-tests
Is there no such option in WebStorm for dart developers?
Accordingly to WEB-14747 ticket this functionality is already implemented for the next major version.
You can try latest EAP build of WebStorm v12 here.
I guess that's currently not supported.
The feature to run tests this way is quite new anyway.
If you think this feature is important, lease create a feature request in https://youtrack.jetbrains.com/issues/WEB

Publishing unit test results from TFS2013 Build to SonarQube

I have created a TFS2013 Build Definition using the template TfvcTemplate.12.xaml
I have specified a test run using VSTestRunner and enabled code coverage.
I am integrating this build with sonar analysis by specifying pre-build and post-test execution script.
Prebuild script arguments: begin /name:PrjName /key:PrjKey /version:1.0 /d:sonar.cs.vstest.reportsPaths="tst*.trx"
I have the "Unit Test Coverage" widget on my sonar dashboard.
It shows Unit Test Coverage %
However, it does not show the unit tests (ie how many tests were run, how many failed ,etc).
I looked in the build output. There is a "tst" folder, however it is empty.
I cannot find the trx files.
I believe that either the trx files are not properly generated or
I am not setting the "sonar.cs.vstest.reportsPaths" correctly.
Please help !!
Relative paths are not well supported: Specify an absolute path wildcard to your *.trx reports. See https://jira.sonarsource.com/browse/SONARMSBRU-100 for details on the bug.
Note that you probably can use the TFS 2013 environment variables to construct this absolute path wildcard: https://msdn.microsoft.com/en-us/library/hh850448.aspx#env_vars

`go test` file modification monitoring for automated testing

Is there a way to have go test run whenever a project's files are modified?
Perhaps there is a good general solution to run a command when files in a path are modified that could be used for this use?
You can use inotifywait for that. Example watching some dir and executing go test on close while having data written:
inotifywait -e close_write <dir> | while read file; do go test; done
Or you write your own tool in go utilizing the howeyc/fsnotify package: Similar example application.
There's also rerun, written in ruby:
rerun go test
Above command will watch the current directory for changes and run go test when a change occurs.