Partially mocking the internal wirings of SUT to DRY up unit-tests - unit-testing

Question

I'm sorry, it's a very long post. I've read almost everything on the subject, and I'm not yet convinced that it's a bad idea to partially mock the SUT to just DRY up the tests. So, I need to first address all the reasonings against it in order to avoid repetitive answers. Please bear with me.
Have you ever felt the urge to partially mock out the SUT itself, in order to make tests more DRY? Less mocks, less madness, more readable tests?!
Let's have an example at hand to discuss the subject more clearly:
class Sut
{
public function fn0(...)
{
// Interacts with: Dp0
}
public function fn1(...)
{
// Calls: fn0
// Interacts with: Dp1
}
public function fn2(...)
{
// Calls: fn0
// Interacts with: Dp1
}
public function fn3(...)
{
// Calls: fn2
}
public function fn4(...)
{
// Calls: fn1(), fn2(), fn3()
// Interacts with: Dp2
}
}
Now, let's test the behavior of the SUT. Each fn*() is representing a behavior of the class under test. Here, I'm not trying to unit-test each single method of the SUT, but its exposed behaviors.
class SutTest extends \PHPUnit_Framework_Testcase
{
/**
* #covers Sut::fn0
*/
public function testFn0()
{
// Mock Dp0
}
/**
* #covers Sut::fn1
*/
public function testFn1()
{
// Mock Dp1, which is a direct dependency
// Mock Dp0, which is an indirect dependency
}
/**
* #covers Sut::fn2
*/
public function testFn2()
{
// Mock Dp1 with different expectations than testFn1()
// Mock Dp0 with different expectations
}
/**
* #covers Sut::fn3
*/
public function testFn3()
{
// Mock Dp1, again with different expectations
// Mock Dp0, with different expectations
}
/**
* #covers Sut::fn4
*/
public function testFn4()
{
// Mock Dp2 which is a direct dependency
// Mock Dp0, Dp1 as indirect dependencies
}
}
You get the terrible idea! You need to keep repeating yourself. It's not DRY at all. And as the expectations of each mock object may differ for each test, you can't just mock out all the dependencies and set expectations once for the whole testcase. You need to be explicit about the behavior of the mock for each test.
Let's also have some real code to see what it would look like when a test needs to mock out all dependencies through the code path it's testing:
/** #test */
public function dispatchesActionsOnACollectionOfElementsFoundByALocator()
{
$elementMock = $this->mock(RemoteWebElement::class)
->shouldReceive('sendKeys')
->times(3)
->with($text = 'some text...')
->andReturn(Mockery::self())
->mock();
$this->inject(RemoteWebDriver::class)
->shouldReceive('findElements')
->with(WebDriverBy::class)
->andReturn([$elementMock, $elementMock, $elementMock])
->shouldReceive('getCurrentURL')
->zeroOrMoreTimes();
$this->inject(WebDriverBy::class)
->shouldReceive('xpath')
->once()
->with($locator = 'someLocatorToMatchMultipleElements')
->andReturn(Mockery::self());
$this->inject(Locator::class)
->shouldReceive('isLocator', 'isXpath')
->andReturn(true, false);
$this->type($text, $locator);
}
Madness! In order to test a small method, you find yourself writing such a test that is extremely non-readable and coupled to the implementation details of 3 or 4 other dependant methods in its way up the chain. It's even more terrible when you see the whole testcase; lot's of those mock blocks, duplicated to set different expectations, covering different paths of code. The test is mirroring the implementation details of few others. It's painful.
Workaround
OK, back to the first pseudo-code; While testing fn3(), you start thinking, what if I could mock the call to fn2() and put a stop to all the mocking madness? I make a partial mock of the SUT, set expectations for the fn2() and I make sure that the method under test interacts with fn2(), correctly.
In other words, to avoid excessive mocking of external dependencies, I focus on just one behavior of the SUT (might be one or couple of methods) and make sure it behaves correctly. I mockout all other methods that belong to other behaviors of the SUT. No worries about them, they all have their own tests.
Contrary reasonings
One might discuss that:
The problem with stubbing/mocking methods in a class is that you are violating the encapsulation. Your test should be checking to see whether or not the external behaviour of the object matches the specifications. Whatever happens inside the object is none of its business. By mocking the public methods in order to test the object, you are making an assumption about how that object is implemented.
When unit-testing, it's rare that you always deal with such behaviors that are fully testable by providing input and expecting output; seeing them as black boxes. Most of the time you need to test how they interact with one another. So, we need to have, at least, some information about the internal implementation of the SUT, to be able to fully test it.
When we mock a dependency, we're ALREADY making an assumption about how the SUT works. We're binding the test to the implementation details of the SUT. So, now that we're deep in mud, why not mocking an internal method to make our lives easier?!
Some may say:
Mocking the methods is taking the object (SUT) and breaking it into two pieces. One piece is being mocked while the other piece is being tested. What you are doing is essentially an ad-hoc breaking up of the object. If that's the case, just break up the object already.
Unit tests should treat the classes they test as black boxes. The only thing which matters is that its public methods behave the way it is expected. How the class achieves this through internal state and private methods does not matter. When you feel that it is impossible to create meaningful tests this way, it is a sign that your classes are too powerful and do too much. You should consider to move some of their functionality to separate classes which can be tested separately.
If there are arguments that support this separation (partially mock the SUT), the same arguments can be used to refactor the class into two classes and this is exactly what you should to then.
If it's a smell for SRP, yeah right, the functionality can be extracted into another class and then you can easily mock that class and go happily home. But it's not the case. The SUT's design is ok, it has no SRP issues, it's small, it does one job and ad-heres to SOLID principles. When looking at SUT's code, there's no reason you want to break the functionality into some other classes. It's already broken into very fine pieces.
How come when you look at SUT tests, you decide to break the class? How come it's ok to mock out aaaaall those dependencies along the way, when testing fn3() but it's not ok to mock the only real dependency it has (even though it's an internal one)? fn2(). Either way, we're bound to the implementation details of the SUT. Either way, the tests are fragile.
It's important to notice why we want to mock those methods. We just want easier testing, less mocks, while maintaining the absolute isolation of the SUT (more on this later).
Some other might reason:
The way I see it, an object has external and internal behaviors. External behavior includes returns values, calls into other objects, etc. Obviously, anything in that category should be tested. But internal behavior shouldn't really be tested. I don't write tests directly on the internal behavior, only indirectly through the external behavior.
Right, I do that, too. But we're not testing the internals of SUT, we're just exercising its public API and we want to avoid excessive mocking.
Reasoning states that, External behavior includes calls into other objects; I agree. We're tryinng to test the SUT's external calls here as well, just by early-mocking the internal method that makes the interaction. That mocked method(s) has the its tests already.
One other reasons:
Too many mocks and already perfectly broken into multiple classes? You are over-unit-testing by unit-testing what should be integration tested.
The example code has just 3 external dependencies and I don't think that it's too much. Again, it's important to notice why I want to partially mock the SUT; only and only for easier testing, avoiding excessive mocks.
By the way, the reasoning might be true somehow. I might need to do integration testing in some cases. More on this in the next section.
The last one says:
These are all tests man, not production code, they don't need to be DRY!
I've actually read something like this! And I simply don't think so. I need to put my lifetime into use! You, too!
Bottom-line: To mock, or not to mock?
When we choose to mock, we're writing whitebox unit-tests. We're bounding tests to the implementation details of the SUT, more or less. Then, if we decide to go down the PURE way, and radically maintain the isolation of the SUT, soon or later, we find ourselves in the hell of mocking madness... and fragile tests. Ten months into the maintenance, you found yourself serving unit-tests instead of them serving you! You find yourself re-implementing multiple tests for a single change in the implementation of one SUT method. Painful right?
So, if we're going this way, why not partially mock the SUT? Why not making our lives waaaaaaaaay easier? I see no reason not to do so? Do you?
I've read and read, and finally came across this article by Uncle Bob:
https://8thlight.com/blog/uncle-bob/2014/05/10/WhenToMock.html
To qoute the most important part:
Mock across architecturally significant boundaries, but not within those boundaries.
I think it's the remedy to all the mocking madness I told you about. There's no need to radically maintain the isolation of the SUT as I've learnt blindly. Even though it may work for most of the time, it also may force you to live in your private mocking hell, banging your head against the wall.
This little gem piece of advice, is the only reasoning that makes sense to not to partially mock the SUT. In fact, it's the exact opposite of doing so. But now the question would be, isn't that integration testing? Is that still called unit-testing? What's the UNIT here? Architecturally significant boundaries?
Here's another article by Google Testing team, implicitly suggesting the same practice:
https://testing.googleblog.com/2013/05/testing-on-toilet-dont-overuse-mocks.html
To recap
If we're going down the pure isolation way, assuming that the SUT is already broken into fine pieces, with minimum possible external deps, is there any reason not to partially mock the SUT? In order to avoid the excessive mocking and to make unit-tests more DRY?
If we take the advice of Uncle Bob to the heart and only "Mock across architecturally significant boundaries, but not within those boundaries.", is that still considered unit testing? What's the unit here?
Thank you for reading.
P.S. Those contrary reasonings are more or less from existing SO answers or articles I found on the subject. Unfortunately, I don't currently have the refs to link to.

Answer 1

Unit tests don't have to be isolated unit tests, at least if you accept the definition promoted by authors like Martin Fowler and Kent Beck. Kent is the creator of JUnit, and probably the main proponent of TDD. These guys don't do mocking.
In my own experience (as long-time developer of an advanced Java mocking library), I see programmers misusing and abusing mocking APIs all the time. In particular, when they think that partially mocking the SUT is a valid idea. It is not. Making test code more DRY shouldn't be an excuse for over-mocking.
Personally, I favor integration tests with minimal or (preferably) no mocking. As long as your tests are stable and run fast enough, they are ok. The important thing is that tests don't become a pain to write, maintain, and more importantly don't discourage the programmers from running them. (This is why I avoid functional UI-driven tests - they tend to be a pain to run.)