Related
I'd like to brush my brain to avoid confusions. In few words, what can be said about Mocking process in TDD
What's the GREAT idea behind MOCKING?
Mocking frameworks are meant to be used only to avoid accessing DB during tests or they can be used for something else?
For new comers (like me), are all the frameworks equal or I need to choose one for this or that reason?
In addition to eliminating databases and other slow or ancillary concerns from the unit being tested, mocking allows you to start writing tests for a class without having to implement any collaborating classes.
As you design some piece of functionality, you'll realize that you need some other class or service, in order to stick to the single responsibility principle, but then you'll have to implement those to get the first one working, which in turn will demonstrate the need for still more classes.
If you can mock or stub those dependencies, then you can create the interfaces upon which that first class will rely, without actually having to implement anything outside of that class -- just return canned results from stubs of the interfaces.
This is an essential component to a test-first approach.
The GREAT idea: LIMIT THE SCOPE OF YOUR TESTS. By removing dependencies you remove the risk of test failures because of dependencies. That way you can focus on the correctness of the code that USES those dependencies.
Mocking DB's is very common but you can mock any dependency with an interface. In a recent project we mocked a web service, for example. You might even want to mock another business object just to make sure that you aren't relying on the correctness of the logic in that object.
I'd choose whichever one seems easiest to use. Moq is really nice.
I suggest you start here:
Mocks are not Stubs
It probably is the article that got me thinking the right way about Mocks. Sure the mocked object is usually heavy (otherwise it may not be worth mocking) but it doesn't have to be heavy in the sense that has some strong reliance on an external system like a database. It can be just a complex piece that you need to isolate to effectively be testing only your class and not the dependency.
I am trying to change my unit testing of ArcGIS, and start using mocks (I use rhino).
When I started to get into writing the tests, I noticed I have to start mocking a lot of objects, and stub a lot of methods for even a single test to pass.
For example - my controller first gets a RelationshipClass (so I need to stub the IWorkspace and the returned IRelationshipClass), then also gets an IFeature (A stub), and finally calls stubRelClass.GetRelatedObjects(stubFeature), to return an ISet of other IFeatures.
Is it normal to have to stub so many objects and methods just to make it pass? I also feel
like I really need to step over the code (yeah - I know I should have written the tests first, I am still trying this one), in order to figure out what to stub out next, and what I should return.
I am also having problem with mocking com classes which implement more than one interface. In the production code I QI them between the interfaces. How can I create a mock that implements both interfaces at runtime?
Depending on your injection chain, yes, sometimes you have to mock a lot of objects. If you're going multiple levels deep though, it may be indicative of a design fault - objects that are relying on data that is three layers down into your API may not be loosely coupled. You should be able to nip the chain in the bud by just returning a fake object of some kind at some point that has the necessary properties that the layer you're testing at needs.
You should also be able to do most of your mocking in a [SetUp] method and then have each test just change one or two things.
For mocking multiple interfaces, Rhino has the concept of a MultiMock. I believe the syntax you're after is:
var mock =
MockRepository.DynamicMultiMock<MyType>(
typeof(Interface1),
typeof(Interface2),
....);
It might be a sign of high coupling - which in turn implies a need to reduce dependencies (which will improve design and testability). As a rough guideline, an object should have around 4-6 collaborators max. Anything over that would set off my alarms.
How are Mocks meant to be used?
To me it sounds like untestable code, which is a smell :-(
I would recommend reading http://misko.hevery.com/code-reviewers-guide/. The author is coach responsible for teaching google developers in the testing area. In the article he shows how you can write testable and untestable code.
Further recommended reading:
Clean Code (Robert C. Martin) - main focus on how to write clean (which corresponds to testable) code.
Working effectively with legacy code (Michael Feather) - shows ways to get untested and untestable code under control.
This question already has answers here:
When should I mock?
(4 answers)
Closed 9 years ago.
Classes that use other classes (as members, or as arguments to methods) need instances that behave properly for unit test. If you have these classes available and they introduce no additional dependencies, isn't it better to use the real thing instead of a mock?
I say use real classes whenever you can.
I'm a big believer in expanding the boundaries of "unit" tests as much as possible. At this point they aren't really unit tests in the traditional sense, but rather just an automated regression suite for your application. I still practice TDD and write all my tests first, but my tests are a little bigger than most people's and my green-red-green cycles take a little longer. But now that I've been doing this for a little while I'm completely convinced that unit tests in the traditional sense aren't all they're cracked up to be.
In my experience writing a bunch of tiny unit tests ends up being an impediment to refactoring in the future. If I have a class A that uses B and I unit test it by mocking out B, when I decide to move some functionality from A to B or vice versa all of my tests and mocks have to change. Now if I have tests that verify that the end to end flow through the system works as expected then my tests actually help me to identify places where my refactorings might have caused a change in the external behavior of the system.
The bottom line is that mocks codify the contract of a particular class and often end up actually specifying some of the implementation details too. If you use mocks extensively throughout your test suite your code base ends up with a lot of extra inertia that will resist any future refactoring efforts.
It is fine to use the "real thing" as long as you have absolute control over the object. For example if you have an object that just has properties and accessors you're probably fine. If there is logic in the object you want to use, you could run into problems.
If a unit test for class a uses an instance of class b and an change introduced to b breaks b, then the tests for class a are also broken. This is where you can run into problems where as with a mock object you could always return the correct value. Using "the real thing" Can kind of convolute tests and hide the real problem.
Mocks can have downsides too, I think there is a balance with some mocks and some real objects you will have to find for yourself.
There is one really good reason why you want to use stubs/mocks instead of real classes. I.e. to make your unit test's (pure unit test) class under test isolated from everything else. This property is extremely useful and the benefits for keeping tests isolated are plentiful:
Tests run faster because they don't need to call the real class implementation. If the implementation is to run against file system or relational database then the tests will become sluggish. Slow tests make developers not run unit tests as often. If you're doing Test Driven Development then time hogging tests are together a devastating waste of developers time.
It will be easier to track down problems if the test is isolated to the class under test. In contrast to a system test it will be much more difficult to track down nasty bugs that are not apparently visible in stack traces or what not.
Tests are less fragile on changes done on external classes/interfaces because you're purely testing the class that is under test. Low fragility is also an indication of low coupling, which is a good software engineering.
You're testing against external behaviour of a class rather than the internal implementation which is more useful when deciding code design.
Now if you want to use real class in your test, that's fine but then it is NOT a unit test. You're doing a integration test instead, which is useful for the purpose of validating requirements and overall sanity check. Integration tests are not run as often as unit tests, in practice it is mostly done before committing to favorite code repository, but is equally important.
The only thing you need to have in mind is the following:
Mocks and stubs are for unit tests.
Real classes are for integration/system tests.
Extracted and extended from an answer of mine How do I unit-test inheriting objects?">here:
You should always use real objects where possible.
You should only use mock objects if the real objects do something you dont want to set up (like use sockets, serial ports, get user input, retrieve bulky data etc). Essentially, mock objects are for when the estimated effort to implement and maintain a test using a real object is greater than that to implement and maintain a test using a mock object.
I dont buy into the "dependant test failure" argument. If a test fails because a depended-on class broke, the test did exactly what it should have done. This is not a smell! If a depended-on interface changes, I want to know!
Highly mocked testing environments are very high-maintenance, particularly early in a project when interfaces are in flux. Ive always found it better to start integration testing ASAP.
I always use a mock version of a dependency if the dependency accesses an external system like a database or web service.
If that isn't the case, then it depends on the complexity of the two objects. Testing the object under test with the real dependency is essentially multiplying the two sets of complexities. Mocking out the dependency lets me isolate the object under test. If either object is reasonably simple, then the combined complexity is still workable and I don't need a mock version.
As others have said, defining an interface on the dependency and injecting it into the object under test makes it much easier to mock out.
Personally, I'm undecided about whether it's worth it to use strict mocks and validate every call to the dependency. I usually do, but it's mostly habit.
You may also find these related questions helpful:
What is object mocking and when do I need it?
When should I mock?
How are mocks meant to be used?
And perhaps even, Is it just me, or are interfaces overused?
Use the real thing only if it has been unit tested itself first. If it introduces dependencies that prevent that (circular dependencies or if it requires certain other measures to be in place first) then use a 'mock' class (typically referred to as a "stub" object).
If your 'real things' are simply value objects like JavaBeans then thats fine.
For anything more complex I would worry as mocks generated from mocking frameworks can be given precise expectations about how they will be used e.g. the number of methods called, the precise sequence and the parameters expected each time. Your real objects cannot do this for you so you risk losing depth in your tests.
I've been very leery of mocked objects since I've been bitten by them a number of times. They're great when you want isolated unit tests, but they have a couple of issues. The major issue is that if the Order class needs a a collection of OrderItem objects and you mock them, it's almost impossible to verify that the behavior of of the mocked OrderItem class matches the real-world example (duplicating the methods with appropriate signatures is generally not enough). More than once I've seen systems fail because the mocked classes don't match the real ones and there weren't enough integration tests in place to catch the edge cases.
I generally program in dynamic languages and I prefer merely overriding the specific methods which are problematic. Unfortunately, this is sometimes hard to do in static languages. The downside of this approach is that you're using integration tests rather than unit tests and bugs are sometimes harder to track down. The upside is that you're using the actual code that is written, rather than a mocked version of that code.
If you don't care for verifying expectations on how your UnitUnderTest should interact with the Thing, and interactions with the RealThing have no other side-effects (or you can mock these away) then it is in my opinion perfectly fine to just let your UnitUnderTest use the RealThing.
That the test then covers more of your code base is a bonus.
I generally find it is easy to tell when I should use a ThingMock instead of a RealThing:
When I want to verify expectations in the interaction with the Thing.
When using the RealThing would bring unwanted side-effects.
Or when the RealThing is simply too hard/troublesome to use in a test setting.
If you write your code in terms of interfaces, then unit testing becomes a joy because you can simply inject a fake version of any class into the class you are testing.
For example, if your database server is down for whatever reason, you can still conduct unit testing by writing a fake data access class that contains some cooked data stored in memory in a hash map or something.
It depends on your coding style, what you are doing, your experience and other things.
Given all that, there's nothing stopping you from using both.
I know I use the term unit test way too often. Much of what I do might be better called integration test, but better still is to just think of it as testing.
So I suggest using all the testing techniques where they fit. The overall aim being to test well, take little time doing it and personally have a solid feeling that it's right.
Having said that, depending on how you program, you might want to consider using techniques (like interfaces) that make mocking less intrusive a bit more often. But don't use Interfaces and injection where it's wrong. Also if the mock needs to be fairly complex there is probably less reason to use it. (You can see a lot of good guidance, in the answers here, to what fits when.)
Put another way: No answer works always. Keep your wits about you, observe what works what doesn't and why.
When I originally was introduced to Mocks I felt the primary purpose was to mock up objects that come from external sources of data. This way I did not have to maintain an automated unit testing test database, I could just fake it.
But now I am starting to think of it differently. I am wondering if Mocks are more effective used to completely isolate the tested method from anything outside of itself. The image that keeps coming to mind is the backdrop you use when painting. You want to keep the paint from getting all over everything. I am only testing that method, and I only want to know how it reacts to these faked up external factors?
It seems incredibly tedious to do it this way but the advantage I am seeing is when the test fails it is because it is screwed up and not 16 layers down. But now I have to have 16 tests to get the same testing coverage because each piece would be tested in isolation. Plus each test becomes more complicated and more deeply tied to the method it is testing.
It feels right to me but it also seems brutal so I kind of want to know what others think.
I recommend you take a look at Martin Fowler's article Mocks Aren't Stubs for a more authoritative treatment of Mocks than I can give you.
The purpose of mocks is to unit test your code in isolation of dependencies so you can truly test a piece of code at the "unit" level. The code under test is the real deal, and every other piece of code it relies on (via parameters or dependency injection, etc) is a "Mock" (an empty implementation that always returns expected values when one of its methods is called.)
Mocks may seem tedious at first, but they make Unit Testing far easier and more robust once you get the hang of using them. Most languages have Mock libraries which make mocking relatively trivial. If you are using Java, I'll recommend my personal favorite: EasyMock.
Let me finish with this thought: you need integration tests too, but having a good volume of unit tests helps you find out which component contains a bug, when one exists.
Don't go down the dark path Master Luke. :) Don't mock everything. You could but you shouldn't... here's why.
If you continue to test each method in isolation, you have surprises and work cut out for you when you bring them all together ala the BIG BANG. We build objects so that they can work together to solve a bigger problem.. By themselves they are insignificant. You need to know if all the collaborators are working as expected.
Mocks make tests brittle by introducing duplication - Yes I know that sounds alarming. For every mock expect you setup, there are n places where your method signature exists. The actual code and your mock expectations (in multiple tests). Changing actual code is easier... updating all the mock expectations is tedious.
Your test is now privy to insider implementation information. So your test depends on how you chose to implement the solution... bad. Tests should be a independent spec that can be met by multiple solutions. I should have the freedom to just press delete on a block of code and reimplement without having to rewrite the test suite.. coz the requirements still stay the same.
To close, I'll say "If it quacks like a duck, walks like a duck, then it probably is a duck" - If it feels wrong.. it probably is. *Use mocks to abstract out problem children like IO operations, databases, third party components and the like.. Like salt, some of it is necessary.. too much and :x *
This is the holy war of State based vs Iteraction based testing.. Googling will give you deeper insight.
Clarification: I'm hitting some resistance w.r.t. integration tests here :) So to clarify my stand..
Mocks do not figure in the 'Acceptance tests'/Integration realm. You'll only find them in the Unit Testing world.. and that is my focus here.
Acceptance tests are different and are very much needed - not belittling them. But Unit tests and Acceptance tests are different and should be kept different.
All collaborators within a component or package do not need to be isolated from each other.. Like micro-optimization that is Overkill. They exist to solve a problem together.. cohesion.
Yes, I agree. I see mocking as sometimes painful, but often necessary, for your tests to truly become unit tests, i.e. only the smallest unit that you can make your test concerned with is under test. This allows you to eliminate any other factors that could potentially affect the outcome of the test. You do end up with a lot more small tests, but it becomes so much easier to work out where a problem is with your code.
My philosophy is that you should write testable code to fit the tests,
not write tests to fit the code.
As for complexity, my opinion is that tests should be simple to write, simply because you write more tests if they are.
I might agree that could be a good idea if the classes you're mocking doesn't have a test suite, because if they did have a proper test suite, you would know where the problem is without isolation.
Most of them time I've had use for mock objects is when the code I'm writing tests for is so tightly coupled (read: bad design), that I have to write mock objects when classes they depend on is not available. Sure there are valid uses for mock objects, but if your code requires their usage, I would take another look at the design.
Yes, that is the downside of testing with mocks. There is a lot of work that you need to put in that it feels brutal. But that is the essence of unit testing. How can you test something in isolation if you don't mock external resources?
On the other hand, you're mocking away slow functionality (such as databases and i/o operations). If the tests run faster then that will keep programmers happy. There is nothing much more painful than waiting for really slow tests, that take more than 10 seconds to finish running, while you're trying to implement one feature.
If every developer in your project spent time writing unit tests, then those 16 layers (of indirection) wouldn't be that much of a problem. Hopefully you should have that test coverage from the beginning, right? :)
Also, don't forget to write a function/integration test between objects in collaboration. Or else you might miss something out. These tests won't need to be run often, but are still important.
On one scale, yes, mocks are meant to be used to simulate external data sources such as a database or a web service. On a more finely grained scale however if you're designing loosely coupled code then you can draw lines throughout your code almost arbitrarily as to what might at any point be an 'outside system'. Take a project I'm working on currently:
When someone attempts to check in, the CheckInUi sends a CheckInInfo object to a CheckInMediator object which validates it using a CheckInValidator, then if it is ok, it fills a domain object named Transaction with CheckInInfo using CheckInInfoAdapter then passes the Transaction to an instance of ITransactionDao.SaveTransaction() for persistence.
I am right now writing some automated integration tests and obviously the CheckInUi and ITransactionDao are windows unto external systems and they're the ones which should be mocked. However, whose to say that at some point CheckInValidator won't be making a call to a web service? That is why when you write unit tests you assume that everything other than the specific functionality of your class is an external system. Therefore in my unit test of CheckInMediator I mock out all the objects that it talks to.
EDIT: Gishu is technically correct, not everything needs to be mocked, I don't for example mock CheckInInfo since it is simply a container for data. However anything that you could ever see as an external service (and it is almost anything that transforms data or has side-effects) should be mocked.
An analogy that I like is to think of a properly loosely coupled design as a field with people standing around it playing a game of catch. When someone is passed the ball he might throw a completely different ball to the next person, he might even throw a multiple balls in succession to different people or throw a ball and wait to receive it back before throwing it to yet another person. It is a strange game.
Now as their coach and manager, you of course want to check how your team works as a whole so you have team practice (integration tests) but you also have each player practice on his own against backstops and ball-pitching machines (unit tests with mocks). The only piece that this picture is missing is mock expectations and so we have our balls smeared with black tar so they stain the backstop when they hit it. Each backstop has a 'target area' that the person is aiming for and if at the end of a practice run there is no black mark within the target area you know that something is wrong and the person needs his technique tuned.
Really take the time to learn it properly, the day I understood Mocks was a huge a-ha moment. Combine it with an inversion of control container and I'm never going back.
On a side note, one of our IT people just came in and gave me a free laptop!
As someone said before, if you mock everything to isolate more granular than the class you are testing, you give up enforcing cohesion in you code that is under test.
Keep in mind that mocking has a fundamental advantage, behavior verification. This is something that stubs don't provide and is the other reason that makes the test more brittle (but can improve code coverage).
Mocks were invented in part to answer the question: How would you unit test objects if they had no getters or setters?
These days, recommended practice is to mock roles not objects. Use Mocks as a design tool to talk about collaboration and separation of responsibilities, and not as "smart stubs".
Mock objects are 1) often used as a means to isolate the code under test, BUT 2) as keithb already pointed out, are important to "focus on the relationships between collaborating objects". This article gives some insights and history related to the subject: Responsibility Driven Design with Mock Objects.
Which scenarios, areas of an application/system, etc. are best suited for 'classic' state based testing versus using mock objects?
I'm going to tackle it from a TDD/BDD perspective, where the tests are driving the designs.
First off it depends what style of design you buy into, because remember this is about design first. As Martin Fowler discusses in this excellent article Mocks Aren't Stubs there are two school of thoughts and they do produce different types of designs.
If you want to buy into the mockist approach then I highly suggest you start by looking at the mockobjects site and their article Mock Roles, Not Objects. They also have a book coming out.
The truth is even if you do believe that the mock-first style of design is not for you, or that you don't want to do it right through your application (e.g. not when testing your domain/service layer) then you will still want to use test doubles. xUnit Test Patterns explains the different types of test doubles and their purposes.
Personally I never mock domain classes, so never mocking entities/value objects. However I've been trying out the mock objects style approach a lot recently, it does change my designs a bit and I find the working style comfortable. Working in this style in an MVC app I'll probably start with an automated acceptance test, then I'll write a controller test that mocks out any non-domain objects (e.g. repositories/services), then I'll move down to testing those repositories/services again mocking out their dependencies. I stop when I reach a class with no troublesome dependencies such as a domain entities/value object. I could go on and test against specific role interfaces which are then implemented by my domain classes, which is what the mockobjects guys would recommend, but I don't currently see a lot of value in that approach.
Obviously it is worth adding that design for test is important here, but remember that although 90% of IoC/mocking/DIP examples show interface-implementation pairs (ICustomerRepository/CustomerRepository) there is a lot of value in instead looking for role interfaces.
You should be using mocks for dependencies. I don't think that its an either-or; Usually you will create mocks for dependencies, set expectations (whether it is calls or state) on them, then run the unit under test. Then you would check its state, and verify the expectations on the mocks, afterwards.
Using mock objects doesnt mean you're not doing state based testing.
When using services, whether my own or third party, I design to interfaces as a matter of course, so I tend to focus on the interactions there. This encourages me to design minimal interfaces.
I check state on anything focusing on value objects, just as simple calculations, lookups, and the like.
The next time you find yourself designing something that typically follows Model/View/Controller-or-Presenter, I highly recommend trying the Presenter First approach (Google it) using interfaces for the Model and View. This will give you a great feel for how to use stubs/mocks effectively.
Its a matter of style.. Mockist vs Classic TDDers.
Personally.. I'd go testing real classes as far as possible. Tone down on Mocks as much as possible; only to decouple things like IO (filesystems, DB Connections, network), Third party components, etc. things that are slow/difficult to get under test.
As an experienced TDD'er, this is a question that I'm frequently asked by other developers. For me, it's a mistake to get sucked into a Mockist vs. Classicist debate, as such a discussion is misleading. State-based and behaviuour-based unit testing are two different tools in your toolbox, and there's no reason why they should be mutually exclusive.
State-based unit testing is suitable when you want to query the internal properties of an object after talking to its external interface. If one or more of its collaborators involve potentially expensive calls to other objects, then by all means stub those calls and simply disregard collaborators.
Behaviour-based unit testing is a good idea when you want to consider the "how" of a unit test and focus upon discoving relationships between objects, as opposed to the traditional "what" questions of a state-based unit test. If you wish to assert that collaborators are used in a certain way and/or sequential order, use mocks - stubs that provide assertions.
I suggest you focus upon standard unit testing practices - exercise as little production code as possible, and have one assertion per test. This will force you to think "what exactly is it that I want to exercise and assert in this test?", and the answer to that question will help you choose the correct unit testing tools.