Assume that I want to test following function:
template<typename Pack>
bool is_valid(const Pack& pack) {
if (pack.a() > 0)
return pack.b();
return false;
};
I will write a small mocker for Pack as follows:
class PackMock {
public:
MOCK_METHOD(int, a, (), (const));
MOCK_METHOD(bool, b, (), (const));
};
and now I try to write test this method with in test function:
TEST(MyTests, IsValidTest) {
PackMock p;
EXPECT_CALL(p, a())
.WillOnce(Return(0))
.WillOnce(Return(1))
.WillOnce(Return(20));
EXPECT_CALL(p, b())
.WillOnce(Return(true))
.WillOnce(Return(false))
.WillOnce(Return(true));
EXPECT_FALSE(is_valid(p));
EXPECT_FALSE(is_valid(p));
EXPECT_TRUE(is_valid(p));
}
At the first sight, this code should be OK. I have 3 expectation and my mocker returns 3 different set of values and this is what happens in real application running (data-wise).
But if you run this code, you will get error that mocker expectations is not satisfied (it says that b() is expected to call 3 times while it is called twice). The reason is that when first test happens, a() returns 0, and because of if condition, b() is not called at all. So at the end, everything is messed up.
Now I wonder if there is a way to for example connect these expectations together or I should always set my expectations based on how code works rather than how my data works? I personally think that the later should be correct. For example, if a() in mocker called we make sure that at we always satisfy 1 expectation in b()? or for example we chain these expectations together in a way that b() is expected only if a() > 0?
Firstly, you can use a testing::Sequence instance to keep track of mock calls in sequence. You can also directly define a InSequence for the test as in the docs.
Secondly, your first EXPECT_CALL does not call the b() mock call, because a() returns 0 which then never evaluates the return pack.b().
As an example found here, I've indicated how you can use a Sequence class in your example as follows:
TEST(MyTests, IsValidTest) {
// Setup the mock object
PackMock p;
// Keep track of the sequence of events
// using a sequence class
testing::Sequence sequence;
// # First is_valid() - a
EXPECT_CALL(p, a())
.InSequence(sequence)
.WillOnce(Return(0));
// # First is_valid() - b - not called
// EXPECT_CALL(p, b())
// .InSequence(sequence)
// .WillOnce(Return(true));
// # Second is_valid() - a
EXPECT_CALL(p, a())
.InSequence(sequence)
.WillOnce(Return(1));
// # Second is_valid() - b
EXPECT_CALL(p, b())
.InSequence(sequence)
.WillOnce(Return(false));
// # Third is_valid() - a
EXPECT_CALL(p, a())
.InSequence(sequence)
.WillOnce(Return(20));
// # Third is_valid() - b
EXPECT_CALL(p, b())
.InSequence(sequence)
.WillOnce(Return(true));
// Act - Initiate the calls and inject the
// mock object as a dependency
// # First call - a() returns 0,
// thus returns false (but() b is not called)
EXPECT_FALSE(is_valid(p));
// # Second call - a() returns 1,
// thus returns false, because b returns false
EXPECT_FALSE(is_valid(p));
// # Third call - a() returns 20,
// thus returns true, because b returns true
EXPECT_TRUE(is_valid(p));
}
This will provide the following output:
Running main() from /opt/compiler-explorer/libs/googletest/release-1.10.0/googletest/src/gtest_main.cc
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from MyTests
[ RUN ] MyTests.IsValidTest
[ OK ] MyTests.IsValidTest (0 ms)
[----------] 1 test from MyTests (0 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[ PASSED ] 1 test.
Thus, to answer your question:
Is it possible to depend expected calls in gMock together?
Yes, using a testing::Sequence class with or just using testing::InSequence()
Thus, if you have calls depending on each other in your test, use a sequence class instance(s) to link them.
Related
I created a test case with Catch2 and I am trying to use TrompeLoeil for finer testing.
However the mock calls either don't happen at all, or happen but don't seem to be detecting by the framework.
Here is a minimal reproductible example :
class MyRealClass
{
public:
virtual int returnSumOfValues(int a, int b)
{
return a + b;
}
};
class MyMockClass : public MyRealClass
{
public:
MAKE_MOCK2(returnSumOfValues, int(int,int));
};
class MyTestedClass
{
public:
MyTestedClass(MyRealClass* _computeObj)
: computeObj(_computeObj)
{
}
int compute()
{
return computeObj->returnSumOfValues(2,3);
}
MyRealClass* computeObj;
};
TEST_CASE("Testing Trompe L'oeil")
{
auto mock = new MyMockClass();
MyTestedClass testedClass(mock);
int val = testedClass.compute();
CHECK(val == 5);
REQUIRE_CALL(*mock, returnSumOfValues(2,3)).RETURN(5);
}
And here is the error I get from running this test :
------------------------------------------------------------------------------- Testing Trompe L'oeil
------------------------------------------------------------------------------- ../../../src/MEGAAutoTests/UnitTests/control/TransferBatchTests.cpp:164
...............................................................................
../../../src/MEGAAutoTests/UnitTests/main.cpp:38: FAILED: explicitly
with message: No match for call of returnSumOfValues with signature
int(int,int) with.
param _1 == 2
param _2 == 3
I debugged this step by step and the mocked returnSumOfValues() is the one being executed.
However, if I make MyRealClass::returnSumOfValues() not virtual, the mock class is not used at all.
And the error is :
------------------------------------------------------------------------------- Testing Trompe L'oeil
------------------------------------------------------------------------------- ../../../src/MEGAAutoTests/UnitTests/control/TransferBatchTests.cpp:164
...............................................................................
../../../src/MEGAAutoTests/UnitTests/main.cpp:43: FAILED: CHECK(
failure.empty() ) with expansion: false with message: failure :=
"../../../src/MEGAAutoTests/UnitTests/control/TransferBatchTests.
cpp:172 Unfulfilled expectation: Expected
*mock.returnSumOfValues(2,3) to be called once, actually never called
param _1 == 2
param _2 == 3 "
This seems to not be consistent with the official documentation, which states that :
The line MAKE_MOCK2(log, void(int severity, const std::string& msg))
creates a mock function void Logger::log(int, const std::string&). If
MAKE_MOCKn(...) or MAKE_CONST_MOCKn(...) are used to implement a
virtual function from a base class, it is always recommended to add a
third macro parameter override since it gives the compiler an ability
to complain about mistakes.
Recommended, not required. And to give the compiler more information, not to make the test work.
TL, DR
Why is my code sample not working?
Why do I need to make mocked functions virtual when the documentation suggests it is not mandatory?
The issue was that expectations need to be set beforehand.
I though they worked like assertions, checking a state after the code was executed, but this is not the case.
In this case, moving the call solved the problem. Here is the fixed code sample :
TEST_CASE("Testing Trompe L'oeil")
{
auto mock = new MyMockClass();
MyTestedClass testedClass(mock);
REQUIRE_CALL(*mock, returnSumOfValues(2,3)).RETURN(5);
int val = testedClass.compute();
CHECK(val == 5);
}
This is the solution to the first problem, the second one (only virtual functions can be mocked) remains unanswered.
There is a method with one string parameter virtual void method(const std::string& str) = 0;
In some cases, I need to be sure that this method is not called with a specific argument, for example, "321".
If the method is called somewhere as ifce->method("123"); and in tests I do EXPECT_CALL(mock, method("321")).Times(0); then the test fails:
Expected arg #0: is equal to "321"
Actual: "123"
Expected: to be never called
Actual: never called - saturated and active
[ FAILED ] test.invokeMethodWithDiferentParameters (0 ms)
How to do it correctly?
Either use testing::NiceMock, which ignores all uninteresting calls
NiceMock<MockFoo> mock;
EXPECT_CALL(mock, method("321")).Times(0);
Or add several expectations
EXPECT_CALL(mock, method(_)).Times(AtLeast(1));
EXPECT_CALL(mock, method("321")).Times(0);
namespace {
using namespace std;
// Declare the google test case
class InjectorTest : public ::testing::Test {
std::shared_ptr<MyData>> m_reader;
public:
static void SetUpTestCase() {
}
static void TearDownTestCase() {
}
InjectorTest()
{
//Create Reader code here !!!!
m_reader = CreateReader();
}
std::shared_ptr<DDSTopicReader<MyData>> getXMLReader() {
return m_reader;
}
};
TEST_F(InjectorTest, CreateReaderAndGetTopic) {
auto reader = getXMLReader();
std::string topic_name = reader->getTopicName();
EXPECT_EQ(topic_name, "test_topic");
}
}; // anonymous namespace
My questions is
1) When I run the test case CreateReaderAndGetTopic does the Constructor of InjectorTest gets called before executing the test case? Or should it be called explictly ?
2) Should my class InjectorTest have a constructor or should the code inside the constructor be moved to SetUpTestCase.
Let's mention first that the static class members SetUpTestCase and
TearDownTestCase have been renamed respectively to SetUpTestSuite and TearDownTestSuite
since googletest v1.8. See Beware of the nomenclature
They did that because the names SetUpTestCase and TearDownTestCase were grossly
misleading. They suggest functions that will be run in every case of a test that
is derived from the fixture class in which they are defined, which they weren't.
SetUpTestCase was in fact run before the first of all tests derived from the fixture
(and still is, if you have an older googletest installation), and TearDownTestCase
was run after the last of all tests derived from the fixture.
As of googletest 1.8, all of the tests that derived from a fixture - which we'd
naturally call its test cases - are collectively regarded as a test suite;
so the set-up function that runs before all of them is now called SetUpTestSuite
and the tear-down function that runs after of them is called TearDownTestSuite.
I will stick with the new improved names as I have the latest rev of googletest.
Look at this gtest code that demonstrates the calling order of a fixture's
constructor and destructor, SetUp and TearDown, SetUpTestSuite and TearDownTestSuite:
gtester.cpp
#include <gtest/gtest.h>
#include <iostream>
struct fixture : ::testing::Test
{
fixture() {
std::cout << __PRETTY_FUNCTION__ << std::endl;
}
~fixture() override {
std::cout << __PRETTY_FUNCTION__ << std::endl;
}
void SetUp() override {
std::cout << __PRETTY_FUNCTION__ << std::endl;
}
void TearDown() override {
std::cout << __PRETTY_FUNCTION__ << std::endl;
}
static void SetUpTestSuite() {
std::cout << __PRETTY_FUNCTION__ << std::endl;
}
static void TearDownTestSuite() {
std::cout << __PRETTY_FUNCTION__ << std::endl;
}
};
TEST_F(fixture, One) {
ASSERT_EQ(1,1);
}
TEST_F(fixture, Two) {
ASSERT_NE(1,0);
}
TEST_F(fixture, Three) {
ASSERT_LT(1,2);
}
You can get a useful peek under the hood if you just preprocess this
file (and preferably, pipe through a pretty-printer) and look at what comes out:
$ g++ -E -P gtester.cpp | clang-format > gtester.ii
In gtester.ii you'll be able to find:
...
class fixture_One_Test : public fixture {
public:
fixture_One_Test() {}
private:
virtual void TestBody();
static ::testing::TestInfo *const test_info_ __attribute__((unused));
fixture_One_Test(fixture_One_Test const &) = delete;
void operator=(fixture_One_Test const &) = delete;
};
::testing::TestInfo *const fixture_One_Test::test_info_ =
::testing::internal::MakeAndRegisterTestInfo(
"fixture", "One", nullptr, nullptr,
::testing::internal::CodeLocation("gtester.cpp", 31),
(::testing::internal::GetTypeId<fixture>()),
::testing::internal::SuiteApiResolver<fixture>::GetSetUpCaseOrSuite(),
::testing::internal::SuiteApiResolver<
fixture>::GetTearDownCaseOrSuite(),
new ::testing::internal::TestFactoryImpl<fixture_One_Test>);
void fixture_One_Test::TestBody() {
switch (0)
case 0:
default:
if (const ::testing::AssertionResult gtest_ar =
(::testing::internal::EqHelper<
decltype(::testing::internal::IsNullLiteralHelper(
1, ::testing::internal::TypeIsValidNullptrConstant<decltype(
1)>()))::value>::Compare("1", "1", 1, 1)))
;
else
return ::testing::internal::AssertHelper(
::testing::TestPartResult::kFatalFailure, "gtester.cpp", 32,
gtest_ar.failure_message()) = ::testing::Message();
}
...
That is what the test case:
TEST_F(fixture, One) {
ASSERT_EQ(1,1);
}
expands to after preprocessing. When googletest runs test case fixture.One:
It constructs a new instance of fixture_One_Test - which by inheritance, is
a new instance of fixture.
The base fixture constructor is called.
The derived fixture_One_Test constructor is called. (But it's empty.)
Googletest calls fixture::Setup() through the fixture_One_Test instance.
It calls fixture_One_Test::TestBody() through the fixture_One_Test instance,
which executes the payload of the test case.
It calls calls fixture::TearDown() through the fixture_One_Test instance.
It destroys the fixture_One_Test instance, which -
Calls the fixture_One_Test destructor, which simply -
Calls the fixture destructor.
Similarly for fixture_Two_Test and fixture_Three_Test.
And just before all of that, googletest calls fixture::SetupTestSuite, if it's
is defined.
And just after all of that, googletest calls fixture::TearDownTestSuite, if it's
is defined.
Now let's see that in action:
$ g++ -Wall -Wextra -o gtester gtester.cpp -lgtest_main -lgtest -pthread
$ ./gtester
Running main() from /home/imk/Downloads/googletest-master/googletest/src/gtest_main.cc
[==========] Running 3 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 3 tests from fixture
static void fixture::SetUpTestSuite()
[ RUN ] fixture.One
fixture::fixture()
virtual void fixture::SetUp()
virtual void fixture::TearDown()
virtual fixture::~fixture()
[ OK ] fixture.One (0 ms)
[ RUN ] fixture.Two
fixture::fixture()
virtual void fixture::SetUp()
virtual void fixture::TearDown()
virtual fixture::~fixture()
[ OK ] fixture.Two (0 ms)
[ RUN ] fixture.Three
fixture::fixture()
virtual void fixture::SetUp()
virtual void fixture::TearDown()
virtual fixture::~fixture()
[ OK ] fixture.Three (0 ms)
static void fixture::TearDownTestSuite()
[----------] 3 tests from fixture (0 ms total)
[----------] Global test environment tear-down
[==========] 3 tests from 1 test case ran. (0 ms total)
[ PASSED ] 3 tests.
SetUpTestSuite() and TearDownTestSuite() each run just once. The constructor,
destructor, SetUp() and TearDown() each run 3 times = the number of test cases
in the test suite.
So to your two questions:
When I run the test case CreateReaderAndGetTopic does the Constructor of InjectorTest gets called before executing the test case? Or should it be called explictly ?
The InjectorTest constructor is necessarily called to construct an instance of the class CreateReaderAndGetTopic_InjectorTest_Test,
which Googletest does for you. It is called before SetUp(), which is called before TestBody(), which is the payload of the test case.
Should my class InjectorTest have a constructor...
InjectorTest will have a constructor, either by default or as defined by you; so you're
wondering whether you should define one.
You should define one if you have something to do that needs to be done anew in preparation
for each test case, and that had better be done before the SetUp() for each test case.
... or should the code inside the constructor be moved to SetUpTestCase?
The code you have inside the constructor:
m_reader = CreateReader();
should be moved out to SetUpTestCase - a.k.a SetUpTestSuite - if it is something
that does not need to be done anew in preparation for each test case but needs to be
done once just before all test cases of InjectorTest.
As it stands, you cannot move that constructor code into SetUpTestSuite because it initializes a non-static
class member of InjectorTest, m_reader. But do you in fact need or want a new XMLReader
to be created by CreateReader() for each test case? Only you can decide.
If you do want to create a new reader for each test case, then you face the choice
of creating it in the constructor or in SetUp(). Here you can be guided by the
Googletest FAQ Should I use the constructor/destructor of the test fixture or SetUp()/TearDown()?
If you want to make sure your test cases in a fixture have the same setup, you should implement the SetUp and TearDown methods (notice the case)
Your InjectorTest fixture only gets constructed once, but the SetUp step (resp. the TearDown step) will be run before (resp. after) your test case is executed.
The constructor should only deal with things that should be only done once (if any), any behavior you want to enforce for your test cases should go in these 2 methods. In your example, if the m_reader member can be shared across all your test cases, you can initialize it in the constructor like you did.
To sumup, here is the sequence of operations that will be run:
InjectorTest::InjectorTest(): test fixture is constructed
For each test case:
InjectorTest::SetUp(): setup for test case
Test case is run (in your example InjectorTest::CreateReaderAndGetTopic)
InjectorTest::TearDown(): setup is undone for next case
InjectorTest::~InjectorTest(): destruction of the fixture object
I came across some odd behavior when mocking a c++ destructor, as documented in google mock cookbook. The class and mock are as follows:
// System under test
class Base {
public:
virtual ~Base(){};
};
// Mocked class
class MockBase : public Base {
public:
MOCK_METHOD0(Die, void());
virtual ~MockBase() { Die(); }
};
Test 1. Test that a mock object is destructed once. This test passes just fine.
// Test 1.
MockBase * mb1 = new MockBase();
EXPECT_CALL(*mb1, Die()).Times(1);
delete mb1;
Test 2. Expecting an object to be destructed twice, fails. This also makes sense.
// Test 2. This fails -> ok.
MockBase * mb2 = new MockBase();
EXPECT_CALL(*mb2, Die()).Times(2);
delete mb2;
Test 3. Testing that a non-deleted object is destructed. This test does not seem to fail, even though I expected it to fail. (notice that I commented out the delete command). At the end of the program, there are warnings that some mocked objects are never deleted.
// Test 3. This does not fail
MockBase * mb3 = new MockBase();
EXPECT_CALL(*mb3, Die()).Times(1);
//delete mb3;
Test 4. Testing that a non-deleted object is destructed twice. This test does not seem to fail either. Similar to test 3, I didn't delete this mockBase either, so I'd expect this to fail as well. I even increased the number of times that I expect this to be called, but it still doesn't fail.
// This does not fail
MockBase * mb4 = new MockBase();
EXPECT_CALL(*mb4, Die()).Times(2);
//delete mb4;
Can someone explain why Tests 3 and 4 pass?
From the Google Mock Cheat Sheet:
Google Mock will verify the expectations on a mock object when it is destructed, or you can do it earlier
In other words, Test 3 and 4 in your examples never have their expectations verified because the mock object doesn't get destroyed. As per the Cheat Sheet, you can force the expectations to be verified manually. For example, Test 3 becomes the following:
MockBase * mb3 = new MockBase();
EXPECT_CALL(*mb3, Die());
Mock::VerifyAndClearExpectations(mb3);
Maybe a finesse question, my problem is that if I write:
EXPECT_CALL(mock, handleMessage(_)).Times(0); // expectation #1
EXPECT_CALL(mock, handleMessage(Pointee(IsLike(aSpecificMessage)))); // expectation #2
... and method handleMessage is called once, but with a different argument (not aSpecificMessage), then the failure looks like:
Mock function called more times than expected - returning default value.
Function call: handleMessage(0x8b5378)
Returns: false
Expected: to be never called
Actual: called once - over-saturated and active
Google Mock doesn't print the diagnostic on why the argument didn't match the predicate in expectation #2.
This is presumably because expectation #1 is the first to fail (?).
If I omit expectation #1, then the failure is verbose, along the lines of:
Google Mock tried the following 1 expectation, but it didn't match:
../../test/Test.cpp:143: EXPECT_CALL(mock, handleMessage(Pointee(IsLike(aSpecificMessage))))...
Expected arg #0: points to a value that <....>
Actual: 0xfeaf18, which points to <.....>
I'm using a custom Matcher IsLike and I went through the trouble of generating very explicit mismatch reasons, I'd like them to be printed. I also would not want to give up expectation #1 because that goes in a "default" section and by default I want no calls on that mock throughout the rest of the tests.
Looks like you should use the StrictMock template modifier. Let's use this simple class and mock:
struct Foo
{
virtual void function(int) {
}
};
struct MockFoo: public Foo
{
MOCK_METHOD1(function, void(int x));
};
Let's start with a basic test that exercises that method:
TEST(MockTest, basic_one_expectation)
{
MockFoo foo;
EXPECT_CALL(foo, function(4));
Foo& foo1(foo);
foo1.function(3);
}
Output:
[ RUN ] MockTest.basic_one_expectation
unknown file: Failure
Unexpected mock function call - returning directly.
Function call: function(3)
Google Mock tried the following 1 expectation, but it didn't match:
mock-test.cpp:298: EXPECT_CALL(foo, function(4))...
Expected arg #0: is equal to 4
Actual: 3
Expected: to be called once
Actual: never called - unsatisfied and active
mock-test.cpp:298: Failure
Actual function call count doesn't match EXPECT_CALL(foo, function(4))...
Expected: to be called once
Actual: never called - unsatisfied and active
[ FAILED ] MockTest.basic_one_expectation (1 ms)
That's one of the alternatives you've already considered, but you don't want it because you have other tests that don't have any particular expectations about whether the function is called, and you want those tests to fail if the function gets called anyway. As a reminder, let's see what happened when we try such a test:
TEST(MockTest, basic_no_expectation)
{
MockFoo foo;
Foo& foo1(foo);
foo1.function(3);
}
Output:
[ RUN ] MockTest.basic_no_expectation
GMOCK WARNING:
Uninteresting mock function call - returning directly.
Function call: function(3)
Stack trace:
[ OK ] MockTest.basic_no_expectation (1 ms)
We get a warning, but the test still passes. That's no good for you. Let's see what effect StrictMock has:
TEST(MockTest, strict_no_expectation)
{
::testing::StrictMock<MockFoo> foo;
Foo& foo1(foo);
foo1.function(3);
}
Output:
[ RUN ] MockTest.strict_no_expectation
unknown file: Failure
Uninteresting mock function call - returning directly.
Function call: function(3)
[ FAILED ] MockTest.strict_no_expectation (0 ms)
We didn't have to explicitly say that we don't want the function to be called, but when the function gets called anyway, the test correctly fails. Exactly what you wanted.
Finally, let's look at what happens with StrictMock in the case where there are explicit expectations for the function's argument:
TEST(MockTest, strict_one_expectation)
{
::testing::StrictMock<MockFoo> foo;
EXPECT_CALL(foo, function(4));
Foo& foo1(foo);
foo1.function(3);
}
Output:
[ RUN ] MockTest.strict_one_expectation
unknown file: Failure
Unexpected mock function call - returning directly.
Function call: function(3)
Google Mock tried the following 1 expectation, but it didn't match:
mock-test.cpp:307: EXPECT_CALL(foo, function(4))...
Expected arg #0: is equal to 4
Actual: 3
Expected: to be called once
Actual: never called - unsatisfied and active
mock-test.cpp:307: Failure
Actual function call count doesn't match EXPECT_CALL(foo, function(4))...
Expected: to be called once
Actual: never called - unsatisfied and active
[ FAILED ] MockTest.strict_one_expectation (0 ms)
The diagnostics show the reason the argument didn't match, just like the original basic_one_expectation test shown above.