How to test the main package functions - unit-testing

I want to test a few functions that are included in my main package, but my tests don't appear to be able to access those functions.
My sample main.go file looks like:
package main
import (
"log"
)
func main() {
log.Printf(foo())
}
func foo() string {
return "Foo"
}
and my main_test.go file looks like:
package main
import (
"testing"
)
func Foo(t testing.T) {
t.Error(foo())
}
when I run go test main_test.go I get
# command-line-arguments
.\main_test.go:8: undefined: foo
FAIL command-line-arguments [build failed]
As I understand, even if I moved the test file elsewhere and tried importing from the main.go file, I couldn't import it, since it's package main.
What is the correct way of structuring such tests? Should I just remove everything from the main package asides a simple main function to run everything and then test the functions in their own package, or is there a way for me to call those functions from the main file during testing?

when you specify files on the command line, you have to specify all of them
Here's my run:
$ ls
main.go main_test.go
$ go test *.go
ok command-line-arguments 0.003s
note, in my version, I ran with both main.go and main_test.go on the command line
Also, your _test file is not quite right, you need your test function to be called TestXXX and take a pointer to testing.T
Here's the modified verison:
package main
import (
"testing"
)
func TestFoo(t *testing.T) {
t.Error(foo())
}
and the modified output:
$ go test *.go
--- FAIL: TestFoo (0.00s)
main_test.go:8: Foo
FAIL
FAIL command-line-arguments 0.003s

Unit tests only go so far. At some point you have to actually run the program. Then you test that it works with real input, from real sources, producing real output to real destinations. For real.
If you want to unit test a thing move it out of main().

This is not a direct answer to the OP's question and I'm in general agreement with prior answers and comments urging that main should be mostly a caller of packaged functions. That being said, here's an approach I'm finding useful for testing executables. It makes use of log.Fataln and exec.Command.
Write main.go with a deferred function that calls log.Fatalln() to write a message to stderr before returning.
In main_test.go, use exec.Command(...) and cmd.CombinedOutput() to run your program with arguments chosen to test for some expected outcome.
For example:
func main() {
// Ensure we exit with an error code and log message
// when needed after deferred cleanups have run.
// Credit: https://medium.com/#matryer/golang-advent-calendar-day-three-fatally-exiting-a-command-line-tool-with-grace-874befeb64a4
var err error
defer func() {
if err != nil {
log.Fatalln(err)
}
}()
// Initialize and do stuff
// check for errors in the usual way
err = somefunc()
if err != nil {
err = fmt.Errorf("somefunc failed : %v", err)
return
}
// do more stuff ...
}
In main_test.go,a test for, say, bad arguments that should cause somefunc to fail could look like:
func TestBadArgs(t *testing.T) {
var err error
cmd := exec.Command(yourprogname, "some", "bad", "args")
out, err := cmd.CombinedOutput()
sout := string(out) // because out is []byte
if err != nil && !strings.Contains(sout, "somefunc failed") {
fmt.Println(sout) // so we can see the full output
t.Errorf("%v", err)
}
}
Note that err from CombinedOutput() is the non-zero exit code from log.Fatalln's under-the-hood call to os.Exit(1). That's why we need to use out to extract the error message from somefunc.
The exec package also provides cmd.Run and cmd.Output. These may be more appropriate than cmd.CombinedOutput for some tests. I also find it useful to have a TestMain(m *testing.M) function that does setup and cleanup before and after running the tests.
func TestMain(m *testing.M) {
// call flag.Parse() here if TestMain uses flags
os.Mkdir("test", 0777) // set up a temporary dir for generate files
// Create whatever testfiles are needed in test/
// Run all tests and clean up
exitcode := m.Run()
os.RemoveAll("test") // remove the directory and its contents.
os.Exit(exitcode)

How to test main with flags and assert the exit codes
#MikeElis's answer got me half way there, but there was a major part missing which Go's own flag_test.go help me figure out.
Disclaimer
You essentially want to run your app and test correctness. So please label this test anyway you want and file it in that category. But its worth trying this type of test out and seeing the benefits. Especially if your a writing a CLI app.
The idea is to run go test as usual, and
Have a unit test run "itself" in a sub-process using the test build of the app that go test makes (see line 86)
We also pass environment variables (see line 88) to the sub-process that will execute the section of code that will run main and cause the test to exit with main's exit code:
if os.Getenv(SubCmdFlags) != "" {
// We're in the test binary, so test flags are set, lets reset it so
// so that only the program is set
// and whatever flags we want.
args := strings.Split(os.Getenv(SubCmdFlags), " ")
os.Args = append([]string{os.Args[0]}, args...)
// Anything you print here will be passed back to the cmd.Stderr and
// cmd.Stdout below, for example:
fmt.Printf("os args = %v\n", os.Args)
// Strange, I was expecting a need to manually call the code in
// `init()`,but that seem to happen automatically. So yet more I have learn.
main()
}
NOTE: If main function does not exit the test will hang/loop.
Then assert on the exit code returned from the sub-process.
// get exit code.
got := cmd.ProcessState.ExitCode()
if got != test.want {
t.Errorf("got %q, want %q", got, test.want)
}
NOTE: In this example, if anything other than the expected exit code is returned, the test outputs the STDOUT and or STDERR from the sub-process, for help with debugging.
See full example here: go-gitter: Testing the CLI

Because you set only one file for the test, it will not use other go files.
Run go test instead of go test main_test.go.
Also change the test function signature Foo(t testing.T) to TestFoo(t *testing.T).

Change package name from main to foobar in both sources.
Move source files under src/foobar.
mkdir -p src/foobar
mv main.go main_test.go src/foobar/
Make sure to set GOPATH to the folder where src/foobar resides.
export GOPATH=`pwd -P`
Test it with
go test foobar

Related

Go not running some tests

I'm new to Golang and I'm trying to run some tests from a golang app.
Tests are running at a docker container with a Golang 1.12.
My problem is that some tests appear to be running correctly, but others do not.
Example:
I have a function that I wanted to fail on purpose.
func TestLol(t *testing.T) {
assert.EqualValues(t, 1, 2)
t.Fail()
}
When I execute the docker container with a "docker run ... go test -v ./..." it should run all tests and about this function on specific it should fail, but what happens that it doesn't fail. Golang just log a "ok" beside the test.
Then I tried to run only the folder with test file that should fail.
Log:
ok github.com/eventials/csw-notifications/services 0.016s
2021/09/25 21:08:44 Command finished successfully.
Tests exited with status code: 0
Stopping csw-notifications_db_1 ... done
Stopping csw-notifications_broker_1 ... done
Going to remove csw-notifications_app_run_ed70597b5c20, csw-notifications_db_1, csw-notifications_broker_1
Removing csw-notifications_app_run_ed70597b5c20 ... done
Removing csw-notifications_db_1 ... done
Removing csw-notifications_broker_1 ... done
My question is why golang dosn't output any log with a FAIL message for this file in specific?
I think it's somewhat related to this question, but as it didn't received any answear I'm reposting it.
Why the tests are not running ? ( Golang ) - goapp test - bug?
EDIT: I'm editting to this question be more clear.
You can try add -timeout
If your test files contain FuncTest with samenames rename.
You can try change output to go test -v -json ./...
-timeout d
If a test binary runs longer than duration d, panic.
If d is 0, the timeout is disabled.
The default is 10 minutes (10m).
main.go
func Start(s ...string) {
fmt.Println(s)
}
main_test.go
func TestStart(t *testing.T) {
Start("rocket")
if 0 == 1 {
t.Log("Houston, we have a problem")
t.Fail()
}
}
func ExampleStart() {
fmt.Println("Ground Control to Major Tom")
// Output:
// Ground Control to Major Tom
}
Change if condition to 0 == 0 to see Fail with somelogs.
I found out that a import was causing this issue.
Every test file importing porthos-go or porthos-go/mock isn't running the test files. Removing those imports fixed my problem.
lib: https://github.com/porthos-rpc/porthos-go
I still don't know why, but when I do I'll update this answear.

Coverage test for standard library errors

I've been trying to learn about Go's built-in testing framework and getting proper test coverage.
In one of the files I'm testing I'm only getting ~87% coverage:
coverage: 87.5% of statements
Here's a section of the code covered in the test:
// Check that the working directory is set
if *strctFlags.PtrWorkDir == "" {
// if the working directory is empty, set it to the current directory
strTemp, err := os.Getwd()
if err != nil {
return "", errors.New("Could not get current working directory")
}
*strctFlags.PtrWorkDir = strTemp
} else if stat, err := os.Stat(*strctFlags.PtrWorkDir); err != nil ||
!stat.IsDir() {
// Existence check of the work dir
return "", errors.New("Specified directory \"" + *strctFlags.PtrWorkDir + "\" could not be found or was not a directory")
}
*strctFlags.PtrWorkDir, err = filepath.Abs(*strctFlags.PtrWorkDir)
if err != nil {
return "", errors.New("Could not determine absolute filepath: " + err.Error())
}
The parts not covered in the test according to the .out file are the "if err != nil {}" blocks, which would be errors returned from standard library calls.
While I think the likelihood of the standard library passing errors would be slim unless there would be hardware failure, I think it would be good to know that the error is handled in the application properly. Also, checking returned errors is, to my understanding, idiomatic Go so I would think it would be good to be able to test error handling properly.
How do people handle testing errors like the situations above? Is it possible to get 100% coverage, or am I doing or structuring something incorrectly? Or do people skip testing those conditions?
As #flimzy explained in his answer, it is not good to aim for 100% coverage instead aim for useful test coverage.
Though you can test the system calls with slight modification to the code like this
package foo
// Get Working directory function
var osGetWd = os.Getwd
func GetWorkingDirectory() (string,error){
strTemp, err := osGetWd() // Using the function defined above
if err != nil {
return "", errors.New("Could not get current working directory")
return strTemp,nil
}
And while testing
package foo
func TestGetWdError() {
// Mocked function for os.Getwd
myGetWd := func() (string,error) {
myErr := errors.New("Simulated error")
return "",myErr
}
// Update the var to this mocked function
osGetWd = myGetWd
// This will return error
val,err := GetWorkingDirectory()
}
This will help you to achieve 100% coverage
There are many non-hardware failure scenarios where most standard library functoins might fail. Whether you care to test those is another question. For os.Getwd(), for instance, I might expect that call to fail if the working directory doesn't exist, (and you could go to the effort of testing this scenario).
What's probably more useful (and a better testing approach in general), would be to mock those calls, so that you can trigger errors during testing, just so that you can test your error-case code.
But please, for the love of code, don't aim for 100% test coverage. Aim for useful test coverage. It is possible to make a tool report 100% coverage without covering useful cases, and it is possible to cover useful cases without making the tool report 100%.
But true 100% coverage is a literal impossibility in most programs (even the simple "Hello World!"). So don't aim for it.

How To Write Pending Tests In Go

Is there legitimate way to write down a test case for which I intent to write full test function later on? As like pending tests of mochajs?
The package docs describe such example with testing.(*T).Skip:
Tests and benchmarks may be skipped if not applicable with a call to the Skip method of *T and *B:
func TestTimeConsuming(t *testing.T) {
if testing.Short() {
t.Skip("skipping test in short mode.")
}
...
}
The message you provided for Skip will be printed if you launch go test with a -v flag (in this example you'll also need to provide -short flag to see the skip message).

Custom command line flags in Go's unit tests

Have a moduled application. Have a bunch of tests that use a set of application modules, each test requires different set. Some modules are tuned through the command-line, e.g:
func init() {
flag.StringVar(&this.customPath, "gamedir.custom", "", "Custom game resources directory")
}
But I cannot test this functionality. If I run
go test -test.v ./... -gamedir.custom=c:/resources
the runtime answers with
flag provided but not defined: -gamedir.custom
and fails the test.
What am I doing wrong with testing command-line args?
I think I got it what is wrong with flags in my case.
With the following command
go test -test.v ./... -gamedir.custom=c:/resources
the compiler runs one or several tests on a workspace. In my particular case there are several tests, because ./... means find and create test executable for every _test.go file found. The test executable applies all the additional params unless one or some of them is ignored within it.
Thus the test executables that do use param pass the test, all others fail. This may be overridden by running go test for each test.go separately, with appropriate set of params respectively.
You'll also get this message if you put your flag declarations inside of a test. Don't do this:
func TestThirdParty(t *testing.T) {
foo := flag.String("foo", "", "the foobar bang")
flag.Parse()
}
Instead use the init function:
var foo string
func init() {
flag.StringVar(&foo, "foo", "", "the foo bar bang")
flag.Parse()
}
func TestFoo() {
// use foo as you see fit...
}
The accepted answer, I found wasn't completely clear. In order to pass a parameter to a test (without an error) you must first consume that parameter using the flag. For the above example where gamedir.custom is a passed flag you must have this in your test file
var gamedir *string = flag.String("gamedir.custom", "", "Custom gamedir.")
Or add it to the TestMain
Note that from Go 1.13, you'll get the following error if you use flag.Parse() in init()
flag provided but not defined: -test.timeout
To fix this, you have to use TestMain
func TestMain(m *testing.M) {
flag.Parse()
os.Exit(m.Run())
}
TestFoo(t *testing.T) {}

How to generate unit test coverage when using command line flags in Golang subprocess testing?

I have unit tests for most of our code. But I cannot figure out how to generate unit tests coverage for certain code in main() in main package.
The main function is pretty simple. It is basically a select block. It reads flags, then either call another function/execute something, or simply print help on screen. However, if commandline options are not set correctly, it will exit with various error codes. Hence, the need for sub-process testing.
I tried sub-process testing technique but modified code so that it include flag for coverage:
cmd := exec.Command(os.Args[0], "-test.run=TestMain -test.coverprofile=/vagrant/ucover/coverage2.out")
Here is original code: https://talks.golang.org/2014/testing.slide#23
Explanation of above slide: http://youtu.be/ndmB0bj7eyw?t=47m16s
But it doesn't generate cover profile. I haven't been able to figure out why not. It does generate cover profile for main process executing tests, but any code executed in sub-process, of course, is not marked as executed.
I try to achieve as much code coverage as possible. I am not sure if I am missing something or if there is an easier way to do this. Or if it is just not possible.
Any help is appreciated.
Thanks
Amer
I went with another approach which didn't involve refactoring main(): see this commit:
I use a global (unexported) variable:
var args []string
And then in main(), I use os.Args unless the private var args was set:
a := os.Args[1:]
if args != nil {
a = args
}
flag.CommandLine.Parse(a)
In my test, I can set the parameters I want:
args = []string{"-v", "-audit", "_tests/p1/conf/gitolite.conf"}
main()
And I still achieve a 100% code coverage, even over main().
I would factor the logic that needs to be tested out of main():
func main() {
start(os.Args)
}
func start(args []string) {
// old main() logic
}
This way you can unit-test start() without mutating os.Args.
Using #VonC solution with Go 1.11, I found I had to reset flag.CommandLine on each test redefining the flags, to avoid a "flag redefined" panic.:
for _, check := range checks {
t.Run("flagging " + check.arg, func(t *testing.T) {
flag.CommandLine = flag.NewFlagSet(cmd, flag.ContinueOnError)
args = []string{check.arg}
main()
})
}