With the help of googlemock and googletest, I set up a test that checks that different mocked errors are correctly handled by the method under test. Basically my code looks like this:
// setup mock object, and object under test
// setup initial EXPECT_CALL expectations
// this expected method call in the middle mocks a failure
EXPECT_CALL(*mock, method(arg_in)).
Times(1).
WillOnce(Throw(exception_of_type_A));
// setup cleanup EXPECT_CALL expectations
// Now invoke method in object under test.
// Expect exception has been translated to different type.
EXPECT_THROW(x.method_under_test(), exception_type_B);
// destructor of mock object will check the mock method invocations
Now my mocked method that fails here can not only fail by throwing an exception of type A, but also by throwing an exception of type B, or by returning an unexpected return value.
I can implement this easily enough by copying and pasting the complete TEST() and just changing what the misbehaving mocked method1 will do. But this will make the code messy. Even if I document that these 3 tests are exactly the same except for how the mocked method1 fails in the WillOnce() action specification, a human reader would still have to compare carefully if this is still true.
What would be the correct way in googletest/googlemock to share the common code between the three TESTS and just have them differ in the WillOnce() action?
To my mind comes: Macros, loops over a container with WillOnce() actions, googletest fixtures, static helper methods for setup and cleanup.
I am still new to googletest and not sure how to address this.
For now, I implement the test logic in a templated static function that accepts an action as a parameter:
template <typename A>
static void paramererizeable_test(A failingAction) {
// set everything up
EXPECT_CALL(*mock, method(arg_in)).
Times(1).
WillOnce(failingAction);
// set up more expectations
// trigger the calls
}
TEST(Section, Method) {
paramererizeable_test(Throw(exception_of_type_A));
paramererizeable_test(Throw(exception_of_type_B));
paramererizeable_test(Return(unexpected_return_value));
}
Not sure if this is how it should be done, or if there is a better way, but it works and is readable.
Related
In all the documentation about gmock I always find the mock object to be instantiated inside a test, like that:
TEST(Bim, Bam)
{
MyMockClass myMockObj;
EXPECT_CALL(MyMockObj, foo(_));
...
}
So, the object is created and destroyed per test. I believe it's also perfectly fine to create and destroy the object per test fixture. But I'm wondering if it's also ok to have a file-global instance of the mock object, like that:
MyMockClass myMockObj;
TEST(Bim, Bam)
{
EXPECT_CALL(MyMockObj, foo(_))
...
}
I tried it and I have absolutely no problems so far, it all seems to work fine. But maybe I should be aware of anything? Just because I stumbled about this question, where the only answer states:
... the problem is that you're instantiating a global instance of FooMock. Googlemock/googletest expect the mock to be defined either within the body of the test, or within a test fixture class.
But I could not find anything in the documentation or anywhere else that confirms this (did I overlook it?).
Thanks, Georg
PS: The reason why I need to use a global mock instance would be the topic of another discussion (see this posting of mine).
You can, but it is not a good idea.
Doing such a thing is violate the isolation principle of UT.
This violation may cause an unexpected failure/pass in your tests.
Gtest uses the destructor of the fake objects to verify that the expectation occurred, this is the reason behind the expectation that each fake object will create and release in the body of the test, or within a test fixture class.
If you make the fake object global then it won't release at the end of each UT, then the verification won't execute and the test will pass even when it should fail. more over some of your UTs may fass/fail when you execute all your tests together; in one test you expect the method x won't call and in the other you expect that the method will call; in one UT you expect the method x will call 3 times, but the method was call twice in the test + one in other test(the test should fail but it won't...)
So the bottom line you should never use a global mock unless this global mock is being in use only to prevent null pointer(you didn't set a behaviour..)
Just stumbled across this question while chasing a bug related to my mock objects. In my case the problem was that the mock object's constructor was being called before InitGoogleMock, and that seemed to send things off into the weeds.
Note: I'm using Google Mock with CppUnitTestFramework.
Fail:
MockObject mock;
TEST_MODULE_INITIALIZE(ModuleInitialize)
{
InitGoogleMock(argc, argv);
}
Win:
MockObject *mock = nullptr;
TEST_MODULE_INITIALIZE(ModuleInitialize)
{
InitGoogleMock(argc, argv);
mock = new MockObject;
}
TEST_MODULE_CLEANUP(ModuleCleanup)
{
delete mock;
}
Not saying it's best practice or anything, but if you need global mock objects I'd say pay attention to when your constructors are being called.
In addition to the accepted answer, if you're using GTest, global variables will also be tagged as leakage when they're not destroyed after test cases are executed.
The idea behind leakage is in this reference: https://google.github.io/googletest/gmock_cook_book.html#forcing-a-verification
If you don't want manual verification, the closest solution would be to have the mock object as a member of your fixture class. And if for some reason you would need to dynamically allocate mock, you can have a pointer and create/destroy the instance on SetUp and TearDown (the same concept as #Chris Olsen's answer). Or if you're in C++11, you can use shared_ptr:
class Fixture : public ::testing::Test
{
std::shared_ptr<ObjT> mPtr;
...
void SetUp()
{
mPtr = std::make_shared<ObjT>();
}
...
}
I have below code, just putting one scenario here.
class A{
public JSONObject m1(type1,type2,type3){
callmethod2(type3);
}
public Map callmethod2(type3){
//some jobs
return myMap;
}
}
#Test
assertequals(JSONObjectTest,m1(type1,type2,type3))
In my test, I am creating a mock with dummy data and passing them to actual function type1,type2,type3 Now there is callmethod2(type3) as a dependency so i do not want to execute that method. so in my test case i written
when(mockA.callmethod2(any(type3.class))).thenReturn(mockMap);
But i can see my callmethod2 is getting execute, how can i solve this and set some expected result for that method in my test case, so it will not execute.
Your code scenario at the time I am writing this is a little incomplete, but I will try to read between the lines. I assume you are somewhere creating a mock instance of class A. I am also assuming you are using that mocked instance to invoke the instance method "m1".
If either of these things are not true, then you will have problems. If you do not create a mock instance, then the "when(...)" clause will not function properly, and in fact while you are invoking the "when" clause you will also actually be invoking the real (i.e., unmocked) instance of A.m1(). The same will be true if you properly mock A but then don't use the mocked instance in the "when" clause, or if you don't use the mocked version in your execute/assert statement(s).
But the bigger problem is that if you actually ARE properly creating and using a mock of A, then even the default mocked functionality of method "m1" becomes empty -- do nothing. It would never get executed. The only way to do what you seem to be proposing is to use what Mockito calls "partial mocking", using a Spy. You instantiate a real instance of A, then wrap it in a Spy to make a "Spied"-instance of A. Now the Spied instance of A inherits all of the default implementations of methods, and you can mock out specific methods if you like. You then must use the spied instance in both yur "whe(...)" clauses as well as your execute/assert statements.
But even that endeavor is questionable. You should try to test classes as a whole rather than mocking out partial behavior. In general (practically a rule),you should either mock nor spy the class under test. If you see a need to do so then there is probably a coherency problem with your class, and it is pointing to design issues or a misunderstanding of good testing practices.
I often face the problem that mock objects need to be brought in a certain state before the "interesting" part of a test can start.
For example, let's say I want to test the following class:
struct ToTest
{
virtual void onEnable();
virtual void doAction();
};
Therefore, I create the following mock class:
struct Mock : ToTest
{
MOCK_METHOD0(onEnable, void());
MOCK_METHOD0(doAction, void());
};
The first test is that onEnable is called when the system that uses a ToTest object is enabled:
TEST(SomeTest, OnEnable)
{
Mock mock;
// register mock somehow
// interesting part of the test
EXPECT_CALL(mock, onEnable());
EnableSystem();
}
So far, so good. The second test is that doAction is called when the system performs an action and is enabled. Therefore, the system should be enabled before the interesting part of the test can start:
TEST(SomeTest, DoActionWhenEnabled)
{
Mock mock;
// register mock somehow
// initialize system
EnableSystem();
// interesting part of the test
EXPECT_CALL(mock, doAction());
DoSomeAction();
}
This works but gives an annoying warning about an uninteresting call to onEnable. There seem to be two common fixes of this problem:
Using NiceMock<Mock> to suppress all such warnings; and
Add an EXPECT_CALL(mock, onEnable()) statement.
I don't want to use the first method since there might be other uninteresting calls that really should not happen. I also don't like the second method since I already tested (in the first test) that onEnable is called when the system is enabled; hence, I don't want to repeat that expectation in all tests that work on enabled systems.
What I would like to be able to do is say that all mock calls up to a certain point should be completely ignored. In this example, I want expectations to be only checked starting from the "interesting part of the test" comment.
Is there a way to accomplish this using Google Mock?
The annoying thing is that the necessary functions are there: gmock/gmock-spec-builders.h defines Mock::AllowUninterestingCalls and others to control the generation of warnings for a specific mock object. Using these functions, it should be possible to temporarily disable warnings about uninteresting calls.
That catch is, however, that these functions are private. The good thing is that class Mock has some template friends (e.g., NiceMock) that can be abused. So I created the following workaround:
namespace testing
{
// HACK: NiceMock<> is a friend of Mock so we specialize it here to a type that
// is never used to be able to temporarily make a mock nice. If this feature
// would just be supported, we wouldn't need this hack...
template<>
struct NiceMock<void>
{
static void allow(const void* mock)
{
Mock::AllowUninterestingCalls(mock);
}
static void warn(const void* mock)
{
Mock::WarnUninterestingCalls(mock);
}
static void fail(const void* mock)
{
Mock::FailUninterestingCalls(mock);
}
};
typedef NiceMock<void> UninterestingCalls;
}
This lets me access the private functions through the UninterestingCalls typedef.
The flexibility you're looking for is not possible in gmock, by design. From the gmock Cookbook (emphasis mine):
[...] you should be very cautious about when to use naggy or strict mocks, as they tend to make tests more brittle and harder to maintain. When you refactor your code without changing its externally visible behavior, ideally you should't need to update any tests. If your code interacts with a naggy mock, however, you may start to get spammed with warnings as the result of your change. Worse, if your code interacts with a strict mock, your tests may start to fail and you'll be forced to fix them. Our general recommendation is to use nice mocks (not yet the default) most of the time, use naggy mocks (the current default) when developing or debugging tests, and use strict mocks only as the last resort.
Unfortunately, this is an issue that we, and many other developers, have encountered. In his book, Modern C++ Programming with Test-Driven Development, Jeff Langr writes (Chapter 5, on Test Doubles):
What about the test design? We split one test into two when we changed from a hand-rolled mock solution to one using Google Mock. If we expressed everything in a single test, that one test could set up the expectations to cover all three significant events. That’s an easy fix, but we’d end up with a cluttered test.
[...]
By using NiceMock, we take on a small risk. If the code later somehow changes to invoke another method on the [...] interface, our tests aren’t going to know about it. You should use NiceMock when you need it, not habitually. Seek to fix your design if you seem to require it often.
You might be better off using a different mock class for your second test.
class MockOnAction : public ToTest {
// This is a non-mocked function that does nothing
virtual void onEnable() {}
// Mocked function
MOCK_METHOD0(doAction, void());
}
In order for this test to work, you can have onEnable do nothing (as shown above). Or it can do something special like calling the base class or doing some other logic.
virtual void onEnable() {
// You could call the base class version of this function
ToTest::onEnable();
// or hardcode some other logic
// isEnabled = true;
}
Just trying the EasyMock for the first time.
I seem to get it going but I am immediately halted with the fact that the mocked class runs a method "returning" void (EntityManager.remove(abc)).
I am able to mock the EntityManger partly to begin testing, i.e.
EasyMock.expect(this.mockManager.find(Some.class, id)).andReturn(mock);
, but how do I do the same for the 'remove' case?
I can't do (for example):
EasyMock.expect(this.mockManager.remove(rek)).andReturn(Boolean(true));
And if I do nothing, I get:
java.lang.AssertionError:
Unexpected method call EntityManager.remove(EasyMock for class my.package.Some)...
I need to test the logic before getting to remove part, but I don't care if it actually succeeds (would be a different thing).
You don't need to call EasyMock.expect(). Just use
this.mockManager.remove(rek);
while in the recording phase (before calling replay()).
If you want the mocked method, for example, to throw an exception or to be called twice, use expectLastCal():
this.mockManager.remove(rek);
expectLastCall().andThrow(new RuntimeException());
//or expectLastCall().times(2);
I don't understand the difference between ON_CALL and EXPECT_CALL when using it to
specify the default action.
So far I noticed/learned there are two ways to adjust the default action of a mock:
ON_CALL(mock, methodX(_)).WillByDefault(Return(0x01));
or
EXPECT_CALL(mock, methodX(_)).WillRepeatedly(Return(0x01));
Could someone explain to me:
The difference between the two methods
The ups and downs of each one
When is it appropriate to use them (what kind of setup ...)
There are subtle but significant differences between the two statements. EXPECT_CALL sets expectation on a mock calls. Writing
EXPECT_CALL(mock, methodX(_)).WillRepeatedly(do_action);
tells gMock that methodX may be called on mock any number of times with any arguments, and when it is, mock will perform do_action. On the other hand,
ON_CALL(mock, methodX(_)).WillByDefault(do_action);
tells gMock that whenever methodX is invoked on mock, it should perform do_action. That feature is helpful in a scenario where you have to write many expectations on your mock, and most/all of them have to specify the same action -- especially if it's complex. You can specify that action in ON_CALL, and then write EXPECT_CALLs without specifying the action explicitly. E.g.,
ON_CALL(mock, Sign(Eq(0), _))
.WillByDefault(DoAll(SetArgPointee<1>("argument is zero"), Return(0)));
ON_CALL(mock, Sign(Gt(0), _))
.WillByDefault(DoAll(SetArgPointee<1>("argument is positive"), Return(1)));
ON_CALL(mock, Sign(Lt(0), _))
.WillByDefault(DoAll(SetArgPointee<1>("argument is negative"), Return(-1)));
Now, if you have to write a lot of EXPECT_CALLs, you don't have to mock's specify the behavior every time:
EXPECT_CALL(mock, Sign(-4, _));
EXPECT_CALL(mock, Sign(0, _));
EXPECT_CALL(mock, Sign(1, _)).Times(2);
EXPECT_CALL(mock, Sign(2, _));
EXPECT_CALL(mock, Sign(3, _));
EXPECT_CALL(mock, Sign(5, _));
In another example, assuming Sign returns int, if you write
ON_CALL(mock, Sign(Gt(0), _)).WillByDefault(Return(1));
EXPECT_CALL(mock, Sign(10, _));
the call mock.Sign(10) will return 1 as ON_CALL provides default behavior for a call specified by EXPECT_CALL. But if you write
EXPECT_CALL(mock, Sign(Gt(0), _).WillRepeatedly(Return(1));
EXPECT_CALL(mock, Sign(10, _));
the invocation of mock.Sign(10, p) will return 0. It will be matched against the second expectation. That expectation specifies no explicit action and gMock will generate a default action for it. That default action is to return a default value of the return type, which is 0 for int. The first expectation will be totally ignored in this case.
ON_CALL(mock, methodX(_)).WillByDefault(Return(0x01));
EXPECT_CALL(mock, methodX(_)).WillRepeatedly(Return(0x01));
As you said, these two lines are doing exactly the same thing, therefore there are no differences at all. Use either way to set a default action as you please.
However, there is a logical difference :
ON_CALL(mock, methodX(_)).WillByDefault(Return(0x01)); means that the method might be called, and if that happens, every call will return 0x01
EXPECT_CALL(mock, methodX(_)).WillRepeatedly(Return(0x01)); means that it is expected that the method will be called, and every call will return 0x01
By the way, there is a Setting default actions in their cheat sheet, which says :
To customize the default action for a particular method, use ON_CALL():
ON_CALL(mock_object, method(matchers))
.With(multi_argument_matcher) ?
.WillByDefault(action);
Here is the "official" explanation about the most important differences between ON_CALL and EXPECT_CALL as explained in the gMock Cookbook.
There are basically two constructs for defining the behavior of a mock object: ON_CALL and EXPECT_CALL.
The difference?
ON_CALL defines what happens when a mock method is called, but doesn't imply any expectation on the method being called.
EXPECT_CALL not only defines the behavior, but also sets an expectation that the method must be called with the given arguments, for the given number of times (and in the given order when you specify the order too).
Since EXPECT_CALL does more, isn't it better than ON_CALL?
Not really. Every EXPECT_CALL adds a constraint on the behavior of the code under test. Having more constraints than necessary is bad - even worse than not having enough constraints.
This may be counter-intuitive. How could tests that verify more be worse than tests that verify less? Isn't verification the whole point of tests?
The answer, lies in what a test should verify. A good test verifies the contract of the code. If a test over-specifies, it doesn't leave enough freedom to the implementation. As a result, changing the implementation without breaking the contract (e.g. refactoring and optimization), which should be perfectly fine to do, can break such tests. Then you have to spend time fixing them, only to see them broken again the next time the implementation is changed.
Keep in mind that one doesn't have to verify more than one property in one test. In fact, it's a good style to verify only one thing in one test. If you do that, a bug will likely break only one or two tests instead of dozens (which case would you rather debug?). If you are also in the habit of giving tests descriptive names that tell what they verify, you can often easily guess what's wrong just from the test log itself.
So use ON_CALL by default, and only use EXPECT_CALL when you actually intend to verify that the call is made.
See here https://github.com/google/googletest/blob/master/docs/gmock_cook_book.md#knowing-when-to-expect-useoncall
There are basically two constructs for defining the behavior of a mock object: ON_CALL and EXPECT_CALL. The difference? ON_CALL defines what happens when a mock method is called, but doesn't imply any expectation on the method being called. EXPECT_CALL not only defines the behavior, but also sets an expectation that the method will be called with the given arguments, for the given number of times (and in the given order when you specify the order too).
One difference is that the ON_CALL behavior (default behavior) and the EXPECT_CALL expectations are cleared differently.
https://github.com/google/googletest/blob/master/googlemock/docs/cheat_sheet.md#verifying-and-resetting-a-mock
using ::testing::Mock;
...
// Verifies and removes the expectations on mock_obj;
// returns true if and only if successful.
Mock::VerifyAndClearExpectations(&mock_obj);
...
// Verifies and removes the expectations on mock_obj;
// also removes the default actions set by ON_CALL();
// returns true if and only if successful.
Mock::VerifyAndClear(&mock_obj);
This can be used to clear expectations at some point in your test, but still keep the default behavior of the mock object. Notice that this is not the case for StrictMock objects since they will not allow the test to pass without actual expectations, even with a defined default behavior setup with ON_CALL.