I am trying to use OCMock for testing my app. But I am confused where should we use expect and where to use stub? Can anyone help please?
The basic difference is this: you expect things that must happen, and stub things that might happen.
There are 2 ways mock objects fail: either an unexpected/unstubbed method is called, or an expected method is not called.
Unexpected invocations. When a mock object receives a message that hasn't been either stubbed or expected, it throws an exception immediately and your test fails.
Expected invocations. When you call verify on your mock (generally at the end of your test), it checks to make sure all of the methods you expected were actually called. If any were not, your test will fail.
There are a couple of types of mocks that change this behavior: nice mocks and partial mocks. Nice mocks prevent you having to stub methods--basically they let unexpected invocations occur. Partial mocks are a way of intercepting messages sent to actual objects. Any messages you expect or stub on a partial mock will be sent to the mock object. All other messages are sent to the actual object. For both nice mocks and partial mocks, you won't get a test failure on unexpected invocations (rule #1 above).
Related
How can I return multiple object from a mocked factory returning unique_ptr, when the calls cannot be identified through different input parameters to the called function?
I'm doing this:
EXPECT_CALL(MyFactoryMock, create())
.WillRepeatedly(Return(ByMove(std::make_unique<MyObjectTypeMock>())));
And run-time error is:
[ FATAL ]
/.../tools/googletest/1.8.0-57/include/gmock/gmock-actions.h:604::
Condition !performed_ failed. A ByMove() action should only be
performed once.
Doing the same thing only once, using WillOnce, works fine.
Follow-up question: Return multiple mock objects from a mocked factory function which returns std::unique_ptr
ByMove is designed to move a predefined value that you prepared in your test, so it can only be called once. If you need something else, you'll need to write it yourself explicitly.
Here's an excerpt from the googletest documentation:
Quiz time! What do you think will happen if a Return(ByMove(...))
action is performed more than once (e.g. you write ...
.WillRepeatedly(Return(ByMove(...)));)? Come think of it, after the
first time the action runs, the source value will be consumed (since
it’s a move-only value), so the next time around, there’s no value to
move from – you’ll get a run-time error that Return(ByMove(...)) can
only be run once.
If you need your mock method to do more than just moving a pre-defined
value, remember that you can always use a lambda or a callable object,
which can do pretty much anything you want:
EXPECT_CALL(mock_buzzer_, MakeBuzz("x"))
.WillRepeatedly([](StringPiece text) {
return MakeUnique<Buzz>(AccessLevel::kInternal);
});
EXPECT_NE(nullptr, mock_buzzer_.MakeBuzz("x"));
EXPECT_NE(nullptr, mock_buzzer_.MakeBuzz("x"));
Every time this EXPECT_CALL fires, a new unique_ptr<Buzz> will be
created and returned. You cannot do this with Return(ByMove(...)).
I have two unit tests that share some state (unfortunately I can't change this since the point is to test the handling of this very state).
TEST(MySuite, test1)
{
shared_ptr<MockObject> first(make_shared<MockObject>());
SubscribeToFooCallsGlobal(first);
EXPECT_CALL(*first, Foo(_));//.RetiresOnSaturation();
TriggerFooCalls(); // will call Foo in all subscribed
}
TEST(MySuite, test2)
{
shared_ptr<MockObject> second(make_shared<MockObject>());
SubscribeToFooCallsGlobal(second);
EXPECT_CALL(*second, Foo(_)).Times(1);
TriggerFooCalls(); // will call Foo in all subscribed
}
If I run the tests separately, both are successful. If I run them in the order test1, test2, I will get the following error in test2:
mytest.cpp(42): error: Mock function called more times than expected - returning directly.
Function call: Foo(0068F65C)
Expected: to be called once
Actual: called twice - over-saturated and active
The expectation that fails is the one in test1. The call does take place, but I would like to tell GoogleMock to not care after test1 is complete (in fact, I only want to check expectations in a test while the test is running).
I was under the impression that RetiresOnSaturation would do this, but with it I get:
Unexpected mock function call - returning directly.
Function call: Foo(005AF65C)
Google Mock tried the following 1 expectation, but it didn't match:
mytest.cpp(42): EXPECT_CALL(first, Foo(_))...
Expected: the expectation is active
Actual: it is retired
Expected: to be called once
Actual: called once - saturated and retired
Which I have to admit, confuses me. What does it mean? How can I solve this?
You can read in documentation of Mock almost literally described your case:
Forcing a Verification
When it's being destoyed, your friendly mock object will automatically
verify that all expectations on it have been satisfied, and will
generate Google Test failures if not. This is convenient as it leaves
you with one less thing to worry about. That is, unless you are not
sure if your mock object will be destoyed.
How could it be that your mock object won't eventually be destroyed?
Well, it might be created on the heap and owned by the code you are
testing. Suppose there's a bug in that code and it doesn't delete the
mock object properly - you could end up with a passing test when
there's actually a bug.
So you shall not expect, that at the end of the test case, in some magic way expectations will be "deactivated". As cited above - the mock destructor is the point of verification.
In your case - you mocks are not local variable - they are created in dynamic memory (heap in cited doc) and kept in your tested code via SubscribeToFooCallsGlobal(), So for sure mock created in one test is still alive in next test.
The easy, and proper solution is to unsubscribe at the end of each TC - I do not know if you have any UnsubscribeToFooCallsGlobal() - if not - create such function. To be sure that it will be always called - use ScopeGuard pattern.
There is one function to manually enforce verification Mock::VerifyAndClearExpectations(&mock_object) - but use it only if you need this verification not in the last line of your testcase, because that should be point of destruction.
edit: Fixed the googlemock link.
Assume we have an Order class with a method called Approve. When this method is called, it checks certain conditions and either puts the Order in the state of Approved or throws an exception. In the service layer, we've got something like this:
var order = _repository.Single(o => o.ID == orderID);
order.Approve();
_context.SaveChanges(); // or _session.SaveChanges();
There are 2 ways to test this method and I'd like to hear your insight on this:
Solution 1: Stub the repository to return an Order object. Then assert the Order is in the state of "Approved".
Solution 2: Stub the repository to return a Mock Order object. Assert that Approve() method was called.
Solution 1 is easier and I personally favor state-based testing to interaction-based testing, as the latter can target implementation details and should be avoided. However, I believe testing that the given Order is in the state of Approved is not the concern of this service method. I think we need a separate test method for the Order class to test whether an exception is thrown or the Order's state is changed to Approved.
Solution 2 may sound logical as we are delegating the responsibility of Approving an Order to the Order class itself. So perhaps we need 2 tests for this service method: One to ensure it delegates the task of Approving an Order to the Order class and one to ensure it saves the changes.
What's your insight on this? Which solution do you prefer?
Cheers
Unit tests are to test whether the observed behavior is conforming to expectations/specification.
The answer to your question boils down to what you consider "expected behavior" in this case: a) if the expected behavior is that the Order is in approved state after calling the service method, then test the state; b) if the expected behavior is that the approve action is delegated, then test the method call.
You will need to test the Order object's behavior as well (so that calling Approve() changes the state to approved) in either case.
The second solution plays well as it decouples the behavior of the two objects, but if there are more than one ways that the order can be in approved state (and that's what you are testing -- case a)), then you limit the accepted behavior needlessly.
Also, I would create a separate test for testing the saving part, if that is not essential to the approval part
I had to start writing some unit tests, using QualityTools.UnitTestFramework, for a web service layer we have developed, when my approach seemed to be incorrect from the beginning.
It seems that unit tests should be able to run in any order and not rely on other tests.
My initial thought was to have the something similar to the following tests (a simplified expample) which would run as an ordered test in the same order.
AddObject1SuccessTest
AddObject2WithSameUniqueCodeTest(relies on first test having created object1 first then expects fail)
AddObject2SuccessTest
UpdateObject2WithSameUniqueCodeTest(relies on first test having created object1 and thrid test having created object2 first then expects fail)
UpdateObject2SuccessTest
GetObjectListTest
DeleteObjectsTest(using added IDs)
However, there is no state between tests and no apparent way of passing say added IDs to the deletetest for example.
So, is it then the case that the correct approach for unit testing complex interactions is by scenario?
For example
AddObjectSuccessTest
(which creates an object, gets it to validate the data and then deletes it)
AddObjectWithSameUniqueCodeTest
(which creates object 1 then attempts to create object 2 with a fail and then deletes object 1)
UpdateObjectWithSameUniqueCodeTest
(which creates object 1 then creates object 2 and then attempts to update object 2 to have the same unique code as object 1 with a fail and then deletes object 1 and object 2)
Am I coming at this wrong?
Thanks
It is a tenet of unit testing that each test case should be independent of any other test case. MSTest (as well as all other unit testing frameworks) enforce this by not guaranteeing the order in which tests are run - some (xUnit.NET) even go so far as to randomize the order between each test run.
It is also a recommended best practice that units are condensed into simple interactions. Although no hard and fast rule can be provided, it's not a unit test if the interaction is too complex. In any case, complex tests are brittle and have a very high maintainance overhead, which is why simple tests are preferred.
It sounds like you have a case of shared state between your tests. This leads to interdependent tests and should be avoided. Instead you can write reusable code that sets up the pre-condition state for each test, ensuring that this state is always correct.
Such a pre-condition state is called a Fixture. The book xUnit Test Patterns contains lots of information and guidance on how to manage Fixtures in many different scenarios.
As a complement to what Mark said, yes, each test should be completely independent from the others, and, to use your terms, each test should be a self-contained scenario, which can run independently of the others.
I assume from what you describe that you are testing persistence, because you have in your steps the deletion of the entities you created at the end of the test, to clean up the state. Ideally, a unit test is running completely in memory, with no shared state between each test. One way to achieve that is to use Mocks. I assume you have something like a Repository in place, so that your class calls Repository.Add(myNewObject), which calls something like Repository.ValidateObjectCanBeAdded(myNewObject). Rather than testing against the real repository, which will add objects in the database and require to delete them to clean the state after the test, you can create an interface IRepository, with the two same methods, and use a Mock to check that when your class calls IRepository, it is exercising the right methods, with the right arguments, in the right order. It also gives you the ability to set the "fake" repository to any state you want, in memory, without having to physically add or delete records from a real storage.
Hope this helps!
The question may be a little vague but here's an example of what I want to know (pseudocode):
//start test-case for CreateObject function
{
// initialization of parameters
MyObject *obj = CreateObject();
// test results
}
//end test-case for CreateObject function
Is it necessary in this case to also deallocate the memory by calling "DestroyObject" function? [this is the particular case that gave birth to this question]
My personal opinion would be no, that I shouldn't undo what the function did, but if many tests would be carried out I could remain without memory/resources for that test-suite (not likely to happen but ...).
What do you think? In this case and also in a general case.
Thanks,
Iulian
In this case, you should deallocate the memory your test case has allocated. That way, you can use a tool that lets you run your tests and confirms that no memory was leaked. Letting your test code leak memory means that this would fail, and you wouldn't be able to tell for certain that the leak was in the test and not in your production code.
As to the more general situation, tests should clean up what they've done. Most unit testing frameworks let you implement a tearDown() method to do this. That way, if one test fails you'll know it's an issue with that test and not an interaction with another test.
You should really try to create all the mock objects on the stack (or use smart pointers). That way they get automatically destructed when test function goes out of scope.
Not directly to do with testing, but if you have C++ code that does stuff like:
MyObject *obj = CreateObject();
where "obj" is not a smart pointer or is not being managed by a class, then you have problems. If I were writing the test, I would say:
MyObject obj;
// tests on obj here
No matter what the results of your tests, obj will be correctly destroyed. Never create an object dynamically in C++ if you can possibly avoid it.
Typically, you want to test an isolated code path and an isolated piece of functionality, and you want to make it a fair test each time. That means starting fresh, setting up exactly what you need, and then discarding the testing environment when you're through. This avoids the problem of different test cases leaving behind side effects that might alter the results or behaviour of subsequent runs. It also means you can guarantee that your tests are all independent of each other, and that you can run any subset, in any order.
You'll find, however, that it's also quite common to have pre and post-suite set-up and tear-down methods, which set up a whole testing environment (such as a database or whatever) that a whole bunch of unit tests can execute against.