Have a moduled application. Have a bunch of tests that use a set of application modules, each test requires different set. Some modules are tuned through the command-line, e.g:
func init() {
flag.StringVar(&this.customPath, "gamedir.custom", "", "Custom game resources directory")
}
But I cannot test this functionality. If I run
go test -test.v ./... -gamedir.custom=c:/resources
the runtime answers with
flag provided but not defined: -gamedir.custom
and fails the test.
What am I doing wrong with testing command-line args?
I think I got it what is wrong with flags in my case.
With the following command
go test -test.v ./... -gamedir.custom=c:/resources
the compiler runs one or several tests on a workspace. In my particular case there are several tests, because ./... means find and create test executable for every _test.go file found. The test executable applies all the additional params unless one or some of them is ignored within it.
Thus the test executables that do use param pass the test, all others fail. This may be overridden by running go test for each test.go separately, with appropriate set of params respectively.
You'll also get this message if you put your flag declarations inside of a test. Don't do this:
func TestThirdParty(t *testing.T) {
foo := flag.String("foo", "", "the foobar bang")
flag.Parse()
}
Instead use the init function:
var foo string
func init() {
flag.StringVar(&foo, "foo", "", "the foo bar bang")
flag.Parse()
}
func TestFoo() {
// use foo as you see fit...
}
The accepted answer, I found wasn't completely clear. In order to pass a parameter to a test (without an error) you must first consume that parameter using the flag. For the above example where gamedir.custom is a passed flag you must have this in your test file
var gamedir *string = flag.String("gamedir.custom", "", "Custom gamedir.")
Or add it to the TestMain
Note that from Go 1.13, you'll get the following error if you use flag.Parse() in init()
flag provided but not defined: -test.timeout
To fix this, you have to use TestMain
func TestMain(m *testing.M) {
flag.Parse()
os.Exit(m.Run())
}
TestFoo(t *testing.T) {}
Related
I have a Rust test which delegates to a C++ test suite using doctest and wants to pass command-line parameters to it. My first attempt was
// in mod ffi
pub fn run_tests(cli_args: &mut [String]) -> bool;
#[test]
fn run_cpp_test_suite() {
let mut cli_args: Vec<String> = env::args().collect();
if !ffi::run_tests(
cli_args.as_mut_slice(),
) {
panic!("C++ test suite reported errors");
}
}
Because cargo test help shows
USAGE:
cargo.exe test [OPTIONS] [TESTNAME] [-- <args>...]
I expected
cargo test -- --test-case="X"
to let run_cpp_test_suite access and pass on the --test-case="X" parameter. But it doesn't; I get error: Unrecognized option: 'test-case' and cargo test -- --help shows it has a fixed set of options
Usage: --help [OPTIONS] [FILTER]
Options:
--include-ignored
Run ignored and not ignored tests
--ignored Run only ignored tests
...
My other idea was to pass the arguments in an environment variable, that is
DOCTEST_ARGS="--test-case='X'" cargo test
but then I need to somehow split that string into arguments (handling at least spaces and quotes correctly) either in Rust or in C++.
There are two pieces of Rust toolchain involved when you run cargo test.
cargo test itself looks for all testable targets in your package or workspace, builds them with cfg(test), and runs those binaries. cargo test processes the arguments to the left of the --, and the arguments to the right are passed to the binary.
Then,
Tests are built with the --test option to rustc which creates an executable with a main function that automatically runs all functions annotated with the #[test] attribute in multiple threads. #[bench] annotated functions will also be run with one iteration to verify that they are functional.
The libtest harness may be disabled by setting harness = false in the target manifest settings, in which case your code will need to provide its own main function to handle running tests.
The “libtest harness” is what rejects your extra arguments. In your case, since you're intending to run an entire other test suite, I believe it would be appropriate to disable the harness.
Move your delegation code to its own file, conventionally located in tests/ in your package directory:
Cargo.toml
src/
lib.rs
...
tests/
cpp_test.rs
Write an explicit target section in your Cargo.toml for it, with harness disabled:
[[test]]
name = "cpp_test"
# path = "tests/cpp_test.rs" # This is automatic; you can use a different path if you really want to.
harness = false
In cpp_test.rs, instead of writing a function with the #[test] attribute, write a normal main function which reads env::args() and calls the C++ tests.
[Disclaimer: I'm familiar with these mechanisms because I've used Criterion benchmarking (which similarly requires disabling the default harness) but I haven't actually written a test with custom arguments the way you're looking for. So, some details might be wrong. Please let me know if anything needs correcting.]
In addition to Kevin Reid's answer, if you don't want to write your own test harness, you can use the shell-words crate to split an environment variable into individual arguments following shell rules:
let args = var ("DOCTEST_ARGS").unwrap_or_else (|_| String::new());
let args = shell_words::split (&args).expect ("failed to parse DOCTEST_ARGS");
Command::new ("cpptest")
.args (args)
.spawn()
.expect ("failed to start subprocess")
.wait()
.expect ("failed to wait for subprocess");
I have the following method to test, which uses a function imported from a package.
import x.y.z
func abc() {
...
v := z.SomeFunc()
...
}
Is it possible to mock SomeFunc() in Go?
Yes, with a simple refactoring. Create a zSomeFunc variable of function type, initialized with z.SomeFunc, and have your package call that instead of z.SomeFunc():
var zSomeFunc = z.SomeFunc
func abc() {
// ...
v := zSomeFunc()
// ...
}
In tests you may assign another function to zSomeFunc, one that is defined in tests, and does whatever the test wants it to.
For example:
func TestAbc(t *testing.T) {
// Save current function and restore at the end:
old := zSomeFunc
defer func() { zSomeFunc = old }()
zSomeFunc = func() int {
// This will be called, do whatever you want to,
// return whatever you want to
return 1
}
// Call the tested function
abc()
// Check expected behavior
}
See related / possible duplicate:
Testing os.Exit scenarios in Go with coverage information (coveralls.io/Goveralls)
One thing you can do is this:
import "x/y/z"
var someFunc = z.SomeFunc
func abc() {
...
v := someFunc()
...
}
And in your test file you would do this.
func Test_abc() {
someFunc = mockFunc
abc()
}
But make sure that you do this in a concurrent manner, if you have multiple TestXxx functions calling abc or setting someFunc you may be better of using a struct with a someFunc field.
Having function pointer and monkey patching it is the one of doing it. But then when you multiple functions to mock, you will have a number function pointers and unnecessarily you will have to call function using the function pointer only.
The better and the recommended idea to have an interface and make your function part of the struct implementing the interface. Once done that, you can generate mocks using some nice tools available for go.
I have been using this:
mockgen -source myModule.go -package myPackage -destination myModuleMock.go
You can install it by:
go get github.com/golang/mock
While creating a package level variable is a viable option, it comes with some restrictions. to name a few:
It discourages running parallel tests using t.Parallel() as there can be race conditions on different behaviors of mocked function.
it is dangerous as a future developer in the same package can accidentally update the implementation of this global variable.
another way is to pass along the methods you want to mock as arguments to the function to enable testability. In my case, I already had numerous clients calling this method and thus, I wanted to avoid violating the existing contracts. so, I ended up creating a wrapped function.
eg:
import (
z "x.y.z"
)
//this should replicate the type of function z from x.y.z
type operation func() resp
func wrappedAbc(op operation) {
....
resp := op()
....
}
func Abc() {
wrappedAbc(z)
}
now for testing the actual logic, you will test calls to wrappedAbc instead of abc and you would pass it a mocked operation. this will allow you to test all the business logic while not violating API contract with current clients of method Abc.
mockcompose uses an approach that allows you to generate a mocking class, you can direct mockcompose to include your selected dependency closure (any imported functions from other packages). In the mean time, it generates a cloned copy of your subject function with local overrides so that you can test against. You can embed code generation process with go generate, therefore ensure your cloned copy is always in-sync with your code changes.
Say you have a function functionThatUsesGlobalFunction that imports
Sprintf in package fmt.
func functionThatUsesGlobalFunction(
format string,
args ...interface{},
) string {
//
// skip fansy logic...
//
// call out to a global function in fmt package
return fmt.Sprintf(format, args...)
}
Your goal is to test functionThatUsesGlobalFunction with Sprintf in package fmt being mocked.
To do that, you can configure go generate with mockcompose as following:
mocks.go
//go:generate mockcompose -n mockFmt -p fmt -mock Sprintf
//go:generate mockcompose -n mockJson -p encoding/json -mock Marshal
//go:generate mockcompose -n clonedFuncs -real "functionThatUsesGlobalFunction,fmt=fmtMock"
package clonefn
go generate mockcompose will then generate plumbing classes for you, enable you to write test code as following:
package clonefn
import (
"testing"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
)
var fmtMock *mockFmt = &mockFmt{}
func TestClonedFuncs(t *testing.T) {
assert := require.New(t)
// setup function mocks
fmtMock.On("Sprintf", mock.Anything, mock.Anything).Return("mocked Sprintf")
// inside functionThatUsesMultileGlobalFunctions: fmt.Sprintf is mocked
assert.True(functionThatUsesGlobalFunction_clone("format", "value") == "mocked Sprintf")
}
Please checkout this for details.
I want to test a few functions that are included in my main package, but my tests don't appear to be able to access those functions.
My sample main.go file looks like:
package main
import (
"log"
)
func main() {
log.Printf(foo())
}
func foo() string {
return "Foo"
}
and my main_test.go file looks like:
package main
import (
"testing"
)
func Foo(t testing.T) {
t.Error(foo())
}
when I run go test main_test.go I get
# command-line-arguments
.\main_test.go:8: undefined: foo
FAIL command-line-arguments [build failed]
As I understand, even if I moved the test file elsewhere and tried importing from the main.go file, I couldn't import it, since it's package main.
What is the correct way of structuring such tests? Should I just remove everything from the main package asides a simple main function to run everything and then test the functions in their own package, or is there a way for me to call those functions from the main file during testing?
when you specify files on the command line, you have to specify all of them
Here's my run:
$ ls
main.go main_test.go
$ go test *.go
ok command-line-arguments 0.003s
note, in my version, I ran with both main.go and main_test.go on the command line
Also, your _test file is not quite right, you need your test function to be called TestXXX and take a pointer to testing.T
Here's the modified verison:
package main
import (
"testing"
)
func TestFoo(t *testing.T) {
t.Error(foo())
}
and the modified output:
$ go test *.go
--- FAIL: TestFoo (0.00s)
main_test.go:8: Foo
FAIL
FAIL command-line-arguments 0.003s
Unit tests only go so far. At some point you have to actually run the program. Then you test that it works with real input, from real sources, producing real output to real destinations. For real.
If you want to unit test a thing move it out of main().
This is not a direct answer to the OP's question and I'm in general agreement with prior answers and comments urging that main should be mostly a caller of packaged functions. That being said, here's an approach I'm finding useful for testing executables. It makes use of log.Fataln and exec.Command.
Write main.go with a deferred function that calls log.Fatalln() to write a message to stderr before returning.
In main_test.go, use exec.Command(...) and cmd.CombinedOutput() to run your program with arguments chosen to test for some expected outcome.
For example:
func main() {
// Ensure we exit with an error code and log message
// when needed after deferred cleanups have run.
// Credit: https://medium.com/#matryer/golang-advent-calendar-day-three-fatally-exiting-a-command-line-tool-with-grace-874befeb64a4
var err error
defer func() {
if err != nil {
log.Fatalln(err)
}
}()
// Initialize and do stuff
// check for errors in the usual way
err = somefunc()
if err != nil {
err = fmt.Errorf("somefunc failed : %v", err)
return
}
// do more stuff ...
}
In main_test.go,a test for, say, bad arguments that should cause somefunc to fail could look like:
func TestBadArgs(t *testing.T) {
var err error
cmd := exec.Command(yourprogname, "some", "bad", "args")
out, err := cmd.CombinedOutput()
sout := string(out) // because out is []byte
if err != nil && !strings.Contains(sout, "somefunc failed") {
fmt.Println(sout) // so we can see the full output
t.Errorf("%v", err)
}
}
Note that err from CombinedOutput() is the non-zero exit code from log.Fatalln's under-the-hood call to os.Exit(1). That's why we need to use out to extract the error message from somefunc.
The exec package also provides cmd.Run and cmd.Output. These may be more appropriate than cmd.CombinedOutput for some tests. I also find it useful to have a TestMain(m *testing.M) function that does setup and cleanup before and after running the tests.
func TestMain(m *testing.M) {
// call flag.Parse() here if TestMain uses flags
os.Mkdir("test", 0777) // set up a temporary dir for generate files
// Create whatever testfiles are needed in test/
// Run all tests and clean up
exitcode := m.Run()
os.RemoveAll("test") // remove the directory and its contents.
os.Exit(exitcode)
How to test main with flags and assert the exit codes
#MikeElis's answer got me half way there, but there was a major part missing which Go's own flag_test.go help me figure out.
Disclaimer
You essentially want to run your app and test correctness. So please label this test anyway you want and file it in that category. But its worth trying this type of test out and seeing the benefits. Especially if your a writing a CLI app.
The idea is to run go test as usual, and
Have a unit test run "itself" in a sub-process using the test build of the app that go test makes (see line 86)
We also pass environment variables (see line 88) to the sub-process that will execute the section of code that will run main and cause the test to exit with main's exit code:
if os.Getenv(SubCmdFlags) != "" {
// We're in the test binary, so test flags are set, lets reset it so
// so that only the program is set
// and whatever flags we want.
args := strings.Split(os.Getenv(SubCmdFlags), " ")
os.Args = append([]string{os.Args[0]}, args...)
// Anything you print here will be passed back to the cmd.Stderr and
// cmd.Stdout below, for example:
fmt.Printf("os args = %v\n", os.Args)
// Strange, I was expecting a need to manually call the code in
// `init()`,but that seem to happen automatically. So yet more I have learn.
main()
}
NOTE: If main function does not exit the test will hang/loop.
Then assert on the exit code returned from the sub-process.
// get exit code.
got := cmd.ProcessState.ExitCode()
if got != test.want {
t.Errorf("got %q, want %q", got, test.want)
}
NOTE: In this example, if anything other than the expected exit code is returned, the test outputs the STDOUT and or STDERR from the sub-process, for help with debugging.
See full example here: go-gitter: Testing the CLI
Because you set only one file for the test, it will not use other go files.
Run go test instead of go test main_test.go.
Also change the test function signature Foo(t testing.T) to TestFoo(t *testing.T).
Change package name from main to foobar in both sources.
Move source files under src/foobar.
mkdir -p src/foobar
mv main.go main_test.go src/foobar/
Make sure to set GOPATH to the folder where src/foobar resides.
export GOPATH=`pwd -P`
Test it with
go test foobar
I have unit tests for most of our code. But I cannot figure out how to generate unit tests coverage for certain code in main() in main package.
The main function is pretty simple. It is basically a select block. It reads flags, then either call another function/execute something, or simply print help on screen. However, if commandline options are not set correctly, it will exit with various error codes. Hence, the need for sub-process testing.
I tried sub-process testing technique but modified code so that it include flag for coverage:
cmd := exec.Command(os.Args[0], "-test.run=TestMain -test.coverprofile=/vagrant/ucover/coverage2.out")
Here is original code: https://talks.golang.org/2014/testing.slide#23
Explanation of above slide: http://youtu.be/ndmB0bj7eyw?t=47m16s
But it doesn't generate cover profile. I haven't been able to figure out why not. It does generate cover profile for main process executing tests, but any code executed in sub-process, of course, is not marked as executed.
I try to achieve as much code coverage as possible. I am not sure if I am missing something or if there is an easier way to do this. Or if it is just not possible.
Any help is appreciated.
Thanks
Amer
I went with another approach which didn't involve refactoring main(): see this commit:
I use a global (unexported) variable:
var args []string
And then in main(), I use os.Args unless the private var args was set:
a := os.Args[1:]
if args != nil {
a = args
}
flag.CommandLine.Parse(a)
In my test, I can set the parameters I want:
args = []string{"-v", "-audit", "_tests/p1/conf/gitolite.conf"}
main()
And I still achieve a 100% code coverage, even over main().
I would factor the logic that needs to be tested out of main():
func main() {
start(os.Args)
}
func start(args []string) {
// old main() logic
}
This way you can unit-test start() without mutating os.Args.
Using #VonC solution with Go 1.11, I found I had to reset flag.CommandLine on each test redefining the flags, to avoid a "flag redefined" panic.:
for _, check := range checks {
t.Run("flagging " + check.arg, func(t *testing.T) {
flag.CommandLine = flag.NewFlagSet(cmd, flag.ContinueOnError)
args = []string{check.arg}
main()
})
}
I was trying to use gocheck to test my go code. I was guiding myself with the following example (similar to the one provided by their website):
package hello_test
import (
"testing"
gocheck "gopkg.in/check.v1"
)
// Hook up gocheck into the "go test" runner.
func Test(t *testing.T) {
gocheck.TestingT(t)
}
type MySuite struct{} //<==== Does the struct have to be named that way, what if we have multiple of these and register them, is it a problem?
var _ = gocheck.Suite(&MySuite{}) // <==== What does this line do?
func (s *MySuite) TestHelloWorld(c *gocheck.C) {
c.Assert(42, gocheck.Equals, "42")
c.Check(42, gocheck.Equals, 42)
}
However, there are some lines I am not sure I understand even after reading the documentation. Why is the line type MySuite struct{} needed and even more of an interesting line, why is var _ = gocheck.Suite(&MySuite{}) needed? The first one, its easy to infer that one probably has to declare that struct first and create functions that will run the tests if implemented with the signature as shown. However, the second line beats me. I have literally no idea why its needed. The documentation says:
Suite registers the given value as a test suite to be run. Any methods starting with the Test prefix in the given value will be considered as a test method.
However, I am not sure about a lot of things. For instance, is there a problem if I run this function with multiple MySuite structs in the same file? Is there anything special about the type MySuite struct? Could the gocheck testing suite work even with some different struct being registered? Basically, how many times can we register a struct in one file and will it still work?
The gocheck.Suite function has the side effect of registering the given suite value with the gocheck package. Internally, it just adds the the suite to a slice of registered test suites. You could get the same effect with:
func init() {
gocheck.Suite(&MySuite{})
}
Either form should work, so it is just a matter of style.
The tests in the registered test suites are run when you call gocheck.TestingT. You do this in your test called Test, which will be picked up by Go's testing framework. This is how gocheck tests are integrated into the testing framework. Note that you only need a single invocation of TestingT to run all test suites: not one for each test suite.