How to test multi-parameter formula - unit-testing

Question

I'm refactoring some code that implements a formula and I want to do it test-first, to improve my testing skills, and leave the code covered.
This particular piece of code is a formula that takes 3 parameters and returns a value. I even have some data tables with expected results for different inputs, so in theory, I could jusst type a zillion tests, just changing the input parameters and checking the results against the corresponding expected value.
But I thought there should be a better way to do it, and looking at the docs I've found Value Parameterized Tests.
So, with that I now know how to automatically create the tests for the different inputs.
But how do I get the corresponding expected result to compare it with my calculated one?
The only thing I've been able to come up with is a static lookup table and a static member in the text fixture which is an index to the lookup table and is incremented in each run. Something like this:
#include "gtest/gtest.h"
double MyFormula(double A, double B, double C)
{
return A*B - C*C; // Example. The real one is much more complex
}
class MyTest:public ::testing::TestWithParam<std::tr1::tuple<double, double, double>>
{
protected:
MyTest(){ Index++; }
virtual void SetUp()
{
m_C = std::tr1::get<0>(GetParam());
m_A = std::tr1::get<1>(GetParam());
m_B = std::tr1::get<2>(GetParam());
}
double m_A;
double m_B;
double m_C;
static double ExpectedRes[];
static int Index;
};
int MyTest::Index = -1;
double MyTest::ExpectedRes[] =
{
// C = 1
// B: 1 2 3 4 5 6 7 8 9 10
/*A = 1*/ 0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0,
/*A = 2*/ 1.0, 3.0, 5.0, 7.0, 9.0, 11.0, 13.0, 15.0, 17.0, 19.0,
/*A = 3*/ 2.0, 5.0, 8.0, 11.0, 14.0, 17.0, 20.0, 23.0, 26.0, 29.0,
// C = 2
// B: 1 2 3 4 5 6 7 8 9 10
/*A = 1*/ -3.0, -2.0, -1.0, 0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0,
/*A = 2*/ -2.0, 0.0, 2.0, 4.0, 6.0, 8.0, 10.0, 12.0, 14.0, 16.0,
/*A = 3*/ -1.0, 2.0, 5.0, 8.0, 11.0, 14.0, 17.0, 20.0, 23.0, 26.0,
};
TEST_P(MyTest, TestFormula)
{
double res = MyFormula(m_A, m_B, m_C);
ASSERT_EQ(ExpectedRes[Index], res);
}
INSTANTIATE_TEST_CASE_P(TestWithParameters,
MyTest,
testing::Combine( testing::Range(1.0, 3.0), // C
testing::Range(1.0, 4.0), // A
testing::Range(1.0, 11.0) // B
));
Is this a good approach or is there any better way to get the right expected result for each run?

Answer 1

Include the expected result along with the inputs. Instead of a triple of input values, make your test parameter be a 4-tuple.
class MyTest: public ::testing::TestWithParam<
std::tr1::tuple<double, double, double, double>>
{ };
TEST_P(MyTest, TestFormula)
{
double const C = std::tr1::get<0>(GetParam());
double const A = std::tr1::get<1>(GetParam());
double const B = std::tr1::get<2>(GetParam());
double const result = std::tr1::get<3>(GetParam());
ASSERT_EQ(result, MyFormula(A, B, C));
}
The downside is that you won't be able to keep your test parameters concise with testing::Combine. Instead, you can use testing::Values to define each distinct 4-tuple you wish to test. You might hit the argument-count limit for Values, so you can split your instantiations, such as by putting all the C = 1 cases in one and all the C = 2 cases in another.
INSTANTIATE_TEST_CASE_P(
TestWithParametersC1, MyTest, testing::Values(
// C A B
make_tuple( 1.0, 1.0, 1.0, 0.0),
make_tuple( 1.0, 1.0, 2.0, 1.0),
make_tuple( 1.0, 1.0, 3.0, 2.0),
// ...
));
INSTANTIATE_TEST_CASE_P(
TestWithParametersC2, MyTest, testing::Values(
// C A B
make_tuple( 2.0, 1.0, 1.0, -3.0),
make_tuple( 2.0, 1.0, 2.0, -2.0),
make_tuple( 2.0, 1.0, 3.0, -1.0),
// ...
));
Or you can put all the values in an array separate from your instantiation and then use testing::ValuesIn:
std::tr1::tuple<double, double, double, double> const FormulaTable[] = {
// C A B
make_tuple( 1.0, 1.0, 1.0, 0.0),
make_tuple( 1.0, 1.0, 2.0, 1.0),
make_tuple( 1.0, 1.0, 3.0, 2.0),
// ...
make_tuple( 2.0, 1.0, 1.0, -3.0),
make_tuple( 2.0, 1.0, 2.0, -2.0),
make_tuple( 2.0, 1.0, 3.0, -1.0),
// ...
};
INSTANTIATE_TEST_CASE_P(
TestWithParameters, MyTest, ::testing::ValuesIn(FormulaTable));

Answer 2

See hard coding the expected result is like you are limiting again the no of test cases. If you want to get a complete data driven model, I would rather suggest you to read inputs, expected result from a flat file/xml/xls file.

Answer 3

I don't have much experience with unit testing, but as a mathematician, I think there is not a lot more you could do.
If you would know some invariants of your formula, you could test for them, but i think that does only make sense in very few scenarios.
As an example, if you would want to test, if you have correctly implemented the natural exponential function, you could make use of the knowledge, that it's derivative should have the same value as the function itself. You could then calculate a numerical approximation to the derivative for a million points and see if they are close to the actual function value.