How do I have go test several/packages/... stop after the first test failure?
It takes some time to build and execute the rest of the tests, despite already having something to work with.
Go 1.10 add a new flag failfast to go test:
The new go test -failfast flag disables running additional tests after any test fails. Note that tests running in parallel with the failing test are allowed to complete.
https://golang.org/doc/go1.10
However, note this does not work across packages: https://github.com/golang/go/issues/33038
Here's a workaround:
for s in $(go list ./...); do if ! go test -failfast -v -p 1 $s; then break; fi; done
To speed-up the build phase you can run
go test -i several/packages/...
before the tests to build and install packages that are dependencies of the test.
To stop after the first failure you can use something like
go test several/packages/... | grep FAILED | head -n 1
Related
I'm working in a large legacy codebase with lots of broken tests making running go test ./... returning a lot of failed tests that I don't want to focus on right now.
I tried using +build which is good enough for targeting certain sub-package but it runs all of the test not configured as well. Going through all the tests right now is not an option for me.
Currently I have to do it the long way with:
go test \
-coverprofile=coverage.out \
-tags=moduleB_feature1,moduleA_feature1,moduleA_feature3 \
code_dir/projectX/moduleA/... \
code_dir/projectY/moduleB/feature1
Is there a command for me to ignore all the test without +build configured so I can use ./...?
You can run test for specific sub-package. E.g., if you have package example.org/your/module/some/package then the command
go test example.org/your/module/some/package
executes tests for some/package only.
It could be combined with selecting tests by regular expression: -run switch
go test example.org/your/module/some/package -run `Foo`
executes all test functions from the package some/package that match the regular expression Foo, such as TestFooBar(t *testing.T)
I'm writing a project to learn how to use Rust and I'm calling my project future-finance-labs. After writing some basic functions and verifying the app can be built I wanted to include some tests, located in aggregates/mod.rs. [The tests are in the same file as the actual code as per the documentation.] I'm unable to get the tests to run despite following the documentation to the best of my ability. I have tried to build the project using PowerShell as well as Bash. [It fails to run on Fedora Linux as well]
Here is my output on Bash:
~/future-finance-labs$ cargo test -- src/formatters/mod.rs
Finished test [unoptimized + debuginfo] target(s) in 5.98s
Running target/debug/deps/future_finance_labs-16ed066e1ea3b9a1
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
Using PowerShell I get the same output with some errors like the following:
error: failed to remove C:\Users\jhale\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_79rhkp1fndgsc\LocalState\rootfs\home\jhale\future-finance-labs\target\debug\build\mime_guess-890328c8763afc22\build_script_build-890328c8763afc22.build_script_build.c22di3i8-cgu.0.rcgu.o: The system cannot find the path specified. (os error 3)
After my initial excitement at the prospect of writing a few tests that passed on the first attempt, I quickly realized all the green was indicative; rather, of a failure to even run the tests. I just want to run the unit tests. Running cargo test alone without a separate and file fails as well. Why can't I run any test in this project with my current setup?
It can't find your test because the rust compiler doesn't know about it. You need to add mod aggregates to main.
mod aggregates;
fn main() {
println!("Hello, world!");
}
After you do that, you'll see that your aggregates/mod.rs doesn't compile for many reasons.
And as Mihir was trying to say, you need to use the name of the test, not the name of the file to run a specific test:
cargo test min_works
cargo test aggregates
See also:
How do I “use” or import a local Rust file?
Rust Book: Controlling How Tests Are Run
We are currently setting up our build-process within an automated continious integration environment and facing the fundamental question, when to run unit-tests?
One way would be to run the unit test with every build task. So as soon as one unit-test fails, the whole build fails. This has the advantage, that the developer is always forced to keep the unit-tests green, as s/he is otherwise not able to run the application. On the other hand, you are always distracted by fixing the tests during a development process - which might force you to work in very small iterations. Besides that the time to run your application always increases, as you have to wait for the tests every time.
The other way would be, to let the CI-Server run the tests after each new commit and let the developer simply know, that something went wrong. In this way the developer is pretty free, at what time to care for the unit-tests, but also other developers on the same branch might suffer, because they cannot be sure, that all parts of the software work as expected and have to check theirselves, if the failing tests might also influence their work.
So do you have any best-practices or recommendations, which would be a good time to run the tests?
BTW: of course we also run bigger integration-tests, which are handled in a seperate CI-process.
Short answer: run all unit tests on the build server for every commit, on every branch. Assuming your unit tests don't take a really long time to run, there really is no downside to this. As for running all unit tests on every build task locally, that would be a overkill. Developers should have the discipline to decide when to run the tests and when not to.
You want to know as soon as possible when something is wrong so you can fix it promptly. You also want to know all of the tests that fail rather than just the first test that fails. When there are multiple issues it would be a pretty annoying workflow to only fix the one issue and then have to commit, push, and wait for the build to run again to see if there are more issues.
Your build process should have two targets: build and test. test should be the default target when not specifying anything else. The test can't run until the project was build, so the build target is a dependancy of test. So (suppose use use make): make or make test will build and test. make build will just build the project.
Now, if you're using some IDE, you could consider doing the test in some separate way "outside" of the IDE. So, maybe add a third target ide and let the ide build that one. It could then have the build target as normal dependency and as last step spawn a new job in background to do the testing in it's own terminal window, something like (under linux): ( xterm -e ./run-tests & ).
And if you're developing outside of an ide (like I do), then just have a separate terminal run the build & test. As soon as testing starts, you know the build process finished, so you can run you application already, even thou the tests are still running.
Just to demonstrate this (and as a proof of concept for having the test run in background) I just created some trivial test case:
bodo.c:
#include <stdio.h>
int main(int argc, char * argv[]) {
printf("Hallo %s", argc > 1 ? argv[1] : "Welt");
return 0;
}
Makefile:
test: build run-tests
ide: build run-tests-background
run-tests-background:
( xterm -e ./run-tests --wait & )
run-tests:
./run-tests
build: bodo
bodo: bodo.o
bodo.o: bodo.c
.PHONY: run-tests run-tests-background
run-tests:
#! /bin/sh
retval=true
if test "$(./bodo)" != "Hallo Welt"
then
echo "Test failed []"
retval=false
fi
if test "$(./bodo Bodo)" != "Hallo Bodo"
then
echo "Test failed [Bodo]"
retval=false
fi
if test "$(./bodo Fail)" != "Hallo Bodo"
then
echo "Test failed [Fail]"
retval=false
fi
sleep 5 # Simulate some more tests
if $retval
then
echo "All tests suceeded ;)"
else
echo "Some tests failed :("
fi
if test "$1" == "--wait"
then
read -p "Press ENTER to close" enter
fi
if $retval
then
exit 0
else
exit 2
fi
Usage:
Build the project but do not run the tests
make build
Build the project and do run the test in current terminal
make
Build the project and do run the test in separate terminal. Make will return once the build process completed and the test got started
make ide
And two helpers, which are not supposed to be run by hand:
Only run the tests in current terminal (this will fail, if the project wasn't built yet)
make run-tests
Only run the tests in separate terminal (this will fail, if the project wasn't built yet). Make will return immediatelly
make run-tests-background
When I execute go test for a whole package the tests fail with:
$ go test github.com/dm03514/go-edu-db/...
# github.com/dm03514/go-edu-db/backends
go1: internal compiler error: in read_type, at go/gofrontend/import.cc:669
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gccgo-4.9/README.Bugs> for instructions.
FAIL github.com/dm03514/go-edu-db/backends [build failed]
? github.com/dm03514/go-edu-db/cmd [no test files]
# github.com/dm03514/go-edu-db/httpd
go1: internal compiler error: in read_type, at go/gofrontend/import.cc:669
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gccgo-4.9/README.Bugs> for instructions.
FAIL github.com/dm03514/go-edu-db/httpd [build failed]
? github.com/dm03514/go-edu-db/logging [no test files]
While the above tests fail go install builds correctly and I can run each of my individual tests correctly:
$ go test github.com/dm03514/go-edu-db/backends/backends_test.go
ok command-line-arguments 0.025s
go test github.com/dm03514/go-edu-db/httpd/handlers_test.go
ok command-line-arguments 0.021s
Has anyone ran into this before? I am brand new to Go, and to get around this I have just been executing each one of my test files individually.
The output of go build is nothing
$ go build github.com/dm03514/go-edu-db/...
$
go version is
$ go version
go version xgcc (Ubuntu 4.9-20140406-0ubuntu1) 4.9.0 20140405 (experimental) [trunk revision 209157] linux/amd64
This happened to me as well. I ended up just commenting out different tests until I was able to see useful output and to see when it would start passing. The root cause was one of my concurrently running test goroutines was calling t.Errorf (specifically, I was using the testify/assert package, but this eventually calls t.Errorf) after the test was completed. The output using go test -v eventually had this error message:
Fail in goroutine after TestTradeReader_Subscribe has completed
For me, this happened because I was using an httptest.Server (which runs concurrently during my test) and was checking input on a test case that exited quickly and didn't require this check.
The thing that helped me.. If you use plenty of tests in a loop and you create some of the mocked services OUTSIDE the loop, it may cause some problem.
TO SOLVE THIS: just move your mocked objects creation for your complex tests inside the loop and it will be done!
There is probably a routine leak. You maybe be modifying/updating a global variable in the test and not reverting for the second test.
Second reason for this error could be your test in not running in a closed env. and effecting other test after.
you can re-structure your test so that the test giving error runs at first so that it succeeds
I run go test in my pkg directory and I get the test results printed to the console as they run, but it would be ideal if I could get them printed to a txt file or even a html file, is it possible to do this? I know you can get coverage reports out from it and generate html files for those which is excellent, but I would have thought it possible to do the same just for the actual results of the tests i.e which tests ran, which passed and which failed. I've been searching the net but even go test help doesn't offer any suggestions for printing results out to a file.
Since I only want to see failed test, I have this script "gt" that I run instead of go test:
go test -coverprofile=coverage.out %*|grep -v -e "^\.\.*$"|grep -v "^$"|grep -v "thus far"
That way, it filters everything but the failed cases.
And you can redirect its content to a file, as mentioned: gt > test.out
It also generates code coverage, which is why I have another script "gc":
grep -v -e " 1$" coverage.out
That way, I don't even wait for a brower to open, I directly see the list of lines which are not yet covered (ie, which don't end with '1' in the coverage.out file)
This will append test results to the test.out file.
go test > test.out
This will overwrite the test results for each test run.
go test |& tee test.out