I don't understand the difference between ON_CALL and EXPECT_CALL when using it to
specify the default action.
So far I noticed/learned there are two ways to adjust the default action of a mock:
ON_CALL(mock, methodX(_)).WillByDefault(Return(0x01));
or
EXPECT_CALL(mock, methodX(_)).WillRepeatedly(Return(0x01));
Could someone explain to me:
The difference between the two methods
The ups and downs of each one
When is it appropriate to use them (what kind of setup ...)
There are subtle but significant differences between the two statements. EXPECT_CALL sets expectation on a mock calls. Writing
EXPECT_CALL(mock, methodX(_)).WillRepeatedly(do_action);
tells gMock that methodX may be called on mock any number of times with any arguments, and when it is, mock will perform do_action. On the other hand,
ON_CALL(mock, methodX(_)).WillByDefault(do_action);
tells gMock that whenever methodX is invoked on mock, it should perform do_action. That feature is helpful in a scenario where you have to write many expectations on your mock, and most/all of them have to specify the same action -- especially if it's complex. You can specify that action in ON_CALL, and then write EXPECT_CALLs without specifying the action explicitly. E.g.,
ON_CALL(mock, Sign(Eq(0), _))
.WillByDefault(DoAll(SetArgPointee<1>("argument is zero"), Return(0)));
ON_CALL(mock, Sign(Gt(0), _))
.WillByDefault(DoAll(SetArgPointee<1>("argument is positive"), Return(1)));
ON_CALL(mock, Sign(Lt(0), _))
.WillByDefault(DoAll(SetArgPointee<1>("argument is negative"), Return(-1)));
Now, if you have to write a lot of EXPECT_CALLs, you don't have to mock's specify the behavior every time:
EXPECT_CALL(mock, Sign(-4, _));
EXPECT_CALL(mock, Sign(0, _));
EXPECT_CALL(mock, Sign(1, _)).Times(2);
EXPECT_CALL(mock, Sign(2, _));
EXPECT_CALL(mock, Sign(3, _));
EXPECT_CALL(mock, Sign(5, _));
In another example, assuming Sign returns int, if you write
ON_CALL(mock, Sign(Gt(0), _)).WillByDefault(Return(1));
EXPECT_CALL(mock, Sign(10, _));
the call mock.Sign(10) will return 1 as ON_CALL provides default behavior for a call specified by EXPECT_CALL. But if you write
EXPECT_CALL(mock, Sign(Gt(0), _).WillRepeatedly(Return(1));
EXPECT_CALL(mock, Sign(10, _));
the invocation of mock.Sign(10, p) will return 0. It will be matched against the second expectation. That expectation specifies no explicit action and gMock will generate a default action for it. That default action is to return a default value of the return type, which is 0 for int. The first expectation will be totally ignored in this case.
ON_CALL(mock, methodX(_)).WillByDefault(Return(0x01));
EXPECT_CALL(mock, methodX(_)).WillRepeatedly(Return(0x01));
As you said, these two lines are doing exactly the same thing, therefore there are no differences at all. Use either way to set a default action as you please.
However, there is a logical difference :
ON_CALL(mock, methodX(_)).WillByDefault(Return(0x01)); means that the method might be called, and if that happens, every call will return 0x01
EXPECT_CALL(mock, methodX(_)).WillRepeatedly(Return(0x01)); means that it is expected that the method will be called, and every call will return 0x01
By the way, there is a Setting default actions in their cheat sheet, which says :
To customize the default action for a particular method, use ON_CALL():
ON_CALL(mock_object, method(matchers))
.With(multi_argument_matcher) ?
.WillByDefault(action);
Here is the "official" explanation about the most important differences between ON_CALL and EXPECT_CALL as explained in the gMock Cookbook.
There are basically two constructs for defining the behavior of a mock object: ON_CALL and EXPECT_CALL.
The difference?
ON_CALL defines what happens when a mock method is called, but doesn't imply any expectation on the method being called.
EXPECT_CALL not only defines the behavior, but also sets an expectation that the method must be called with the given arguments, for the given number of times (and in the given order when you specify the order too).
Since EXPECT_CALL does more, isn't it better than ON_CALL?
Not really. Every EXPECT_CALL adds a constraint on the behavior of the code under test. Having more constraints than necessary is bad - even worse than not having enough constraints.
This may be counter-intuitive. How could tests that verify more be worse than tests that verify less? Isn't verification the whole point of tests?
The answer, lies in what a test should verify. A good test verifies the contract of the code. If a test over-specifies, it doesn't leave enough freedom to the implementation. As a result, changing the implementation without breaking the contract (e.g. refactoring and optimization), which should be perfectly fine to do, can break such tests. Then you have to spend time fixing them, only to see them broken again the next time the implementation is changed.
Keep in mind that one doesn't have to verify more than one property in one test. In fact, it's a good style to verify only one thing in one test. If you do that, a bug will likely break only one or two tests instead of dozens (which case would you rather debug?). If you are also in the habit of giving tests descriptive names that tell what they verify, you can often easily guess what's wrong just from the test log itself.
So use ON_CALL by default, and only use EXPECT_CALL when you actually intend to verify that the call is made.
See here https://github.com/google/googletest/blob/master/docs/gmock_cook_book.md#knowing-when-to-expect-useoncall
There are basically two constructs for defining the behavior of a mock object: ON_CALL and EXPECT_CALL. The difference? ON_CALL defines what happens when a mock method is called, but doesn't imply any expectation on the method being called. EXPECT_CALL not only defines the behavior, but also sets an expectation that the method will be called with the given arguments, for the given number of times (and in the given order when you specify the order too).
One difference is that the ON_CALL behavior (default behavior) and the EXPECT_CALL expectations are cleared differently.
https://github.com/google/googletest/blob/master/googlemock/docs/cheat_sheet.md#verifying-and-resetting-a-mock
using ::testing::Mock;
...
// Verifies and removes the expectations on mock_obj;
// returns true if and only if successful.
Mock::VerifyAndClearExpectations(&mock_obj);
...
// Verifies and removes the expectations on mock_obj;
// also removes the default actions set by ON_CALL();
// returns true if and only if successful.
Mock::VerifyAndClear(&mock_obj);
This can be used to clear expectations at some point in your test, but still keep the default behavior of the mock object. Notice that this is not the case for StrictMock objects since they will not allow the test to pass without actual expectations, even with a defined default behavior setup with ON_CALL.
Related
I have two mocks. Only one of them should be called for a single run and I would like to use the expectations to figure out if the execute() function is successful without knowing from the given preconditions.
How can this be achieved?
Mock1 successMock;
Mock2 failMock;
EXPECT_CALL(successMock, performOnSuccess()).Times(1);
EXPECT_CALL(failMock, performOnFail()).Times(0);
execute(successMock, failMock);
Either above should be valid or below expectations but only one of them.
EXPECT_CALL(successMock, performOnSuccess()).Times(0);
EXPECT_CALL(failMock, performOnFail()).Times(1);
What you expect is that one (and only one) of the collaborators has been called.
One viable solution (as presented in other languages) is to make these mocks to increment a shared counter in the test scope.
You can achieve this with GoogleMock by defining actions. It would be something like this:
SuccessMock successMock;
FailMock failMock;
int callCounter = 0;
ON_CALL(successMock, performOnSuccess())
.WillByDefault(InvokeWithoutArgs([&]()
{
callCounter++;
});
ON_CALL(failMock, performOnFailure())
.WillByDefault(InvokeWithoutArgs([&]()
{
callsCounter++;
});
execute(successMock, failMock);
ASSERT_THAT(callsCounter, Eq(1));
Having said this, this test has some randomness in it that I don't really like. You should be end up having one test expecting failure and another one expecting success.
I wrote an simplified gist for this.
With the help of googlemock and googletest, I set up a test that checks that different mocked errors are correctly handled by the method under test. Basically my code looks like this:
// setup mock object, and object under test
// setup initial EXPECT_CALL expectations
// this expected method call in the middle mocks a failure
EXPECT_CALL(*mock, method(arg_in)).
Times(1).
WillOnce(Throw(exception_of_type_A));
// setup cleanup EXPECT_CALL expectations
// Now invoke method in object under test.
// Expect exception has been translated to different type.
EXPECT_THROW(x.method_under_test(), exception_type_B);
// destructor of mock object will check the mock method invocations
Now my mocked method that fails here can not only fail by throwing an exception of type A, but also by throwing an exception of type B, or by returning an unexpected return value.
I can implement this easily enough by copying and pasting the complete TEST() and just changing what the misbehaving mocked method1 will do. But this will make the code messy. Even if I document that these 3 tests are exactly the same except for how the mocked method1 fails in the WillOnce() action specification, a human reader would still have to compare carefully if this is still true.
What would be the correct way in googletest/googlemock to share the common code between the three TESTS and just have them differ in the WillOnce() action?
To my mind comes: Macros, loops over a container with WillOnce() actions, googletest fixtures, static helper methods for setup and cleanup.
I am still new to googletest and not sure how to address this.
For now, I implement the test logic in a templated static function that accepts an action as a parameter:
template <typename A>
static void paramererizeable_test(A failingAction) {
// set everything up
EXPECT_CALL(*mock, method(arg_in)).
Times(1).
WillOnce(failingAction);
// set up more expectations
// trigger the calls
}
TEST(Section, Method) {
paramererizeable_test(Throw(exception_of_type_A));
paramererizeable_test(Throw(exception_of_type_B));
paramererizeable_test(Return(unexpected_return_value));
}
Not sure if this is how it should be done, or if there is a better way, but it works and is readable.
I have a Unit test code where I have a mock to a function foo. When I use ON_CALL construct to specify the action, it does not perform my_action but just executes foo.
ON_CALL(my_mock, foo(my_matcher()))
.WillByDefault(DoAll(my_action(), Return(0)));
But when I do EXPECT_CALL, it works. In my case, foo() gets called only once. I tried to see how it could be different. But googling or searching stackoverflow didn't help. Below is the EXPECT_CALL that works:
EXPECT_CALL(my_mock, foo(my_matcher()))
.WillOnce(DoAll(my_action(), Return(0)));
I dont understand how they are different given the same my_matcher and my_action.
ON_CALL provides a default action to run when an expectation is met. So an action defined in ON_CALL will not fire unless an expectation defined in EXPECT_CALL is fulfilled. The typical use case for ON_CALL is to replace a bunch of identical WillOnce/WillRepeatedly clauses in your EXPECT_CALLs. To make your snippet work, it will need to look like this:
ON_CALL(my_mock, foo(_))
.WillByDefault(DoAll(my_action(), Return(0)));
EXPECT_CALL(my_mock, foo(my_matcher())).Times(1);
EXPECT_CALL(my_mock, foo(my_other_matcher())).Times(1);
Is it necessary to verify the interactions with the Mock objects? So let's say I have a class:
Class A{
B b;
public A(B b){
this.b = b;
}
int getObjectFromDatabase(int id){
Object o = b.get(id);
// do some extra work
return result
}
}
Now I'm testing the getObjectFromDatabase method. I have passed the Mock object of Class B. Do I need to verify the interaction that b.get(id) is being called or not? Or is it a good idea to just check the input and output result I get?
In general, verifying that a stubbed call happened is not necessary, and leads to brittle tests (tests that fail even when the implementation remains correct). In your case, it probably doesn't matter whether get(id) is called, or how many times; it only matters that the object returned is correct, and it's likely that a correct result will require calling b.get(id) at some point.
Mockito has some explicit advice in its verify(T) Javadoc:
Although it is possible to verify a stubbed invocation, usually it's just redundant. Let's say you've stubbed foo.bar(). If your code cares what foo.bar() returns then something else breaks(often before even verify() gets executed). If your code doesn't care what get(0) returns then it should not be stubbed. Not convinced? See here.
Though it may make sense to verify(b).get(id) a certain number of times (including zero) when testing caching or lazy-load behavior, or in other circumstances where the interaction is a crucial part of the tested behavior, strive to test for the correct output/state instead of verifying your expected interactions.
I know that many mocking libraries let the programmer check that a method was called only once. But why is that useful?
Also, why is it useful to verify the parameters to a mock's method?
When doing unit tests you are testing a isolated method: all other methods are supposed to work correctly, and you test only that your method behaves in the expected (specified...) way.
But in many occasions the expected way implies calling methods of classes you depend on (via dependency injection, if you want to do unit testing).
For these reason you need to assure that this calls are really done... and of course they are called with the expected parameters.
Example:
In your real application you have a repository class that stores all your changes in the database (and does only this!). But for unit test your "business" class (where all your business rules are defined), you should mock that "repository" class: then you must check that this mocked class receives the correct updated data.
In general, the answers to both of those questions are the same.
It's useful if the requirements of the unit/method you're testing specify that that behavior is required. If that behavior is required, then that's what you need to verify is actually happening.
If it's important to ensure that a particular method is only called once, then you can do that. If it doesn't matter that a method is called more than once, then don't test for it explicitly. Note that the default for the Mockito "verify" method is "times(1)", which means that it confirms that the method was called once and only once.
Concerning the second question, you might want to verify the parameters if it's entirely possible the method could be called with different parameters, and you don't want to count those occurrences, you only care about a specific set of parameter values.