Unit testing go code with fatal flows in a loop - unit-testing

Question

I am trying to add tests to a go cli code.
The code has a lot of log.Fatal flows.
A bit of googling led me here, so I followed it and got the tests working.
But, the way my tests are, they are setup to run the function being tested in a loop with different arguments.
Here is the test code
func TestGetXXX_FatalFlow(t *testing.T) {
type args struct {
varA string
varB string
}
tests := []struct {
name string
args args
expected string
}{
{
name: "Scenario 1: varA and varB both blank",
args: args{},
expected: "message1",
},
{
name: "Scenario 2: varA and varB not blank but invalid",
args: args{
varA: "somevalueA",
varB: "somevalueB",
},
expected: "message2",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Only run the failing part when a specific env variable is set
if os.Getenv("BE_CRASHER") == "1" {
GetXXX(tt.args.serverName, tt.args.address)
return
}
// Start the actual test in a different subprocess
cmd := exec.Command(os.Args[0], "-test.run=TestGetXXX_FatalFlow")
cmd.Env = append(os.Environ(), "BE_CRASHER=1")
stdout, _ := cmd.StderrPipe()
if err := cmd.Start(); err != nil {
t.Fatal(err)
}
// Check that the log fatal message is what we expected
gotBytes, _ := ioutil.ReadAll(stdout)
if !strings.Contains(string(gotBytes), tt.expected) {
t.Fatalf("Unexpected log message. Got %s but should contain %s", strippedMsg, tt.expected)
}
// Check that the program exited
cmd.Env = append(os.Environ(), "BE_CRASHER=0")
err = cmd.Wait()
if e, ok := err.(*exec.ExitError); !ok || e.Success() {
t.Fatalf("Process ran with err %v, want exit status 1", err)
}
})
}
}
The problem I am running into is , I feel my method GetXXX is never called with the second pair of input variables, somehow the method GetXXX keeps getting called with just the very first pair of argument in tests array.
I am not so sure if its because this spawns a child process.
Any help will be greatly appreciated.
Thanks

Answer 1

If you walk through the code you'll see it's doing what you would expect:
Outer test run starts, loops over the test cases, env var isn't set, so for each iteration, it forks out a new go test process.
New process starts running the same test function, which loops over the test cases, env var is set, so for each iteration, it calls GetXXX, which crashes.
You'll see that at step 2 here, for each iteration, the child process runs the first test case, which crashes, and it never gets to the second case. The loop iteration in the parent is irrelevant - it never passes any parameters of the test case to the child process, so the child has no idea which test case the parent thinks it's testing. It iterates the cases itself again, but only manages to execute the first before crashing.
Generally I would advise against this sort of structure (go test forking out a new go test), and I would also advise against having fatals anywhere in your code. For 99% of cases, your functions should be returning an error when something goes wrong. For truly unrecoverable fatal errors, you should be using panic, which you can then test for using recover. log.Fatal (which I'm guessing is what you're using) just prints a log message and then calls os.Exit, which makes it nearly impossible to test for, as you've found.
If structuring the program correctly really, truly isn't an option, then as a last resort, you could do something like this (untested but I hope it gets the point across):
crasher := os.Getenv("BE_CRASHER")
if crasher == "" { // Parent process
for idx := range tests {
t.Run(tt.name, func(t *testing.T) {
// Start the actual test in a different subprocess
cmd := exec.Command(os.Args[0], "-test.run=TestGetXXX_FatalFlow")
cmd.Env = append(os.Environ(), fmt.Sprintf("BE_CRASHER=%d", idx))
stdout, _ := cmd.StderrPipe()
if err := cmd.Start(); err != nil {
t.Fatal(err)
}
// Validate child process did as expected yadda yadda
})
}
} else { // Child process
idx, err := strconv.Atoi(crasher)
if err != nil {
panic(err)
}
tt := tests[idx]
GetXXX(tt.args.serverName, tt.args.address)
return
}
This changes it up by having the parent iterate the test cases and when it forks the child it uses the env var to tell the child which case to run. The child then runs only the specified case.