I have read this: http://martinfowler.com/articles/mocksArentStubs.html
My concepts about a stub and a mock are clear. I understand the need of isolation frameworks like moq, rhinomocks and like to create a mock object. As mocks, participate in actual verfication of expectations. But why do we need these frameworks to create stubs. I would rather prefer rolling out a hand created stub and use it in various fixtures.
Have you tried using a library like Rhino Mocks for a day or two before deciding that hand-rolled stubs is a better option?
#chibacity: The whole idea about mocking libraries is to avoid implementations. If I only need to set a single property's value, I don't want to create a whole implementation of an interface. Calling code like
MyObj obj = MockRepository.GenerateStub<MyObj>();
obj.MyProperty = 33;
seems much simpler to me. Not to mention situations where I only need a dumb stub to be passed as a parameter where I don't care what happens to it (so no set up is required).
This doesn't mean you shouldn't roll out your own stubs for more complex scenarios, but in my experience such cases are rare.
My recommendation is to learn how to use a good mocking library (like Rhino) with all its little tricks, my bet is that you'll soon learn to appreciate the reasons for its existence.
Strictly speaking, no, you do not need an isolation framework to create stubs. In fact, Microsoft even has a Stubs Framework that only generates stubs, not mocks.
I've never written an isolation framework myself, but it would seem that it once you have object mocking in place, stubs would be a breeze to create. That might be the main reason that most/all isolation frameworks include stub objects.
In regards to your last sentence ("I would rather prefer rolling out a hand created stub and use it in various fixtures"), have you actually tried that on any sizable project? Sure, maybe you have an interface with a single method that returns a nullable bool -- you only have to write three stubs for that interface, and that's not so bad.
But once you start looking at dozens of interfaces and classes to stub, it simply becomes a mess to keep track of all the different stubs. If you are using each stub in several tests, you can certainly justify hand-writing a stub and putting it aside; but when you only use one particular stub once or twice, it's much easier to keep it as an "anonymous" stub directly generated by the framework, for simplicity's sake.
After using mocking frameworks for a while, I have found that my code design takes on a totally different slant. This slant seems to be directed more in an interaction style. In other words, I become more interested in messages and behavior than state. My objects become services rather than stateful objects as would happen when using stubs. With stubs I end up passing state around to other objects.
The problem for me then becomes more of creating abstraction layers. I have to question whither something should be interacting at certain levels. This helps with creating that 'Last responsible Moment'. So I end up with objects that have producers and consumers. And everything in between is just message channels.
I hope this helps.
Related
I'm curious as to what method people like to use for mocking and why. The two methods that I know of are using hard coded mock objects and a mocking framework. To demonstrate, I'll outline an example using C#.
Suppose we have an IEmployeeRepository interface with a method called GetEmployeeById.
public interface IEmployeeRepository
{
Employee GetEmployeeById(long id);
}
We can easily create a mock of this:
public class MockEmployeeRepository : IEmployeeRepository
{
public Employee GetEmployeeById(long id)
{
Employee employee = new Employee();
employee.FirstName = "First";
employee.LastName = "Last";
...
return employee;
}
}
Then, in our tests we can explicitly tell our services to use the MockEmployeeRepository, either using a setter or dependency injection. I'm new to mocking frameworks so I'm curious as to why we use them, if we can just do the above?
That's not a Mock, it's a Stub. For stubbing, your example is perfectly acceptable.
From Martin Fowler:
Mocks are what we are talking about here: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.
When you're mocking something, you usually call a "Verify" method.
Look at this for the diff between Mocks and Stubs
http://martinfowler.com/articles/mocksArentStubs.html
I think the choice between writing dummy objects by hand or by using a framework depends a lot upon the types of components that you are testing.
If it is part of the contract for the component under test to communicate with its collaborators following a precise protocol, then instrumented dummy objects ("Mocks") are just the thing to use. It is frequently much easier to test such protocols using a mocking framework than by hand-coding. Consider a component that is required to open a repository, perform some reads and writes in a prescribed order, and then close the repository -- even in the face of an exception. A mocking framework would make it easier to set up all of the necessary tests. Applications related to telecommunications and process control (to pick a couple of random examples) are full of components that need to be tested in this fashion.
On the other hand, many components in general business applications have no particular constraints on how they communicate with their collaborators. Consider a component that performs some kind of analysis of, say, university course loads. The component needs to retrieve instructor, student and course information from a repository. But it does not matter what order it retrieves the data: instructor-student-course, student-course-instructor, all-at-once, or whatever. There is no need to test for and enforce a data access pattern. Indeed, it would likely be harmful to test that pattern as it would be demanding a particular implementation unnecessarily. In this context, simple uninstrumented dummy objects ("Stubs") are adequate and a mocking framework is probably overkill.
I should point out that even when stubbing, a framework can still make your life a lot easier. One doesn't always have the luxury of dictating the signatures of one's collaborators. Imagine unit-testing a component that is required to process data retrieved from a thick interface like IDataReader or ResultSet. Hand-stubbing such interfaces is unpleasant at best -- especially if the component under test only actually uses three of the umpteen methods in the interface.
For me, the projects that have required mocking frameworks were almost invariably of a systems-programming nature (e.g. database or web infrastructure projects, or low-level plumbing in a business application). For applications-programming projects, my experience has been that there were few mocks in sight.
Given that we always strive to hide the messy low-level infrastructure details as much as possible, it would seem that we should aim to have the simple stubs far outnumber the mocks.
Some distinguish between mocks and stubs. A mock object may verify that it has been interacted with in the expected way. A mocking framework can make it easy to generate mocks and stubs.
In your example, you've stubbed out a single method in an interface. Consider an interface with n methods, where n can change over time. A hand-stubbed implementation may require more code and more maintenance.
A mocked interface can have different outputs per test - One test you may have a method return null, another test has the method return an object, another test has the method throw an exception. This is all configured in the unit test, whereas your version would require several hand-written objects.
Psuedocode:
//Unit Test One
MockObject.Expect(m => m.GetData()).Return(null);
//Unit Test Two
MockObject.Expect(m => m.GetData()).Return(new MyClass());
//Unit Test Three
MockObject.Expect(m => m.GetData()).ThrowException();
I tend to write stubs and mocks by hand, first. Then if it can be easily expressed using a mock object framework, I rewrite it so that I have less code to maintain.
I have been writing them by hand. I was having trouble using Moq, but then I read TDD: Introduction to Moq, and I think I get what they say about classical vs. mockist approaches now. I'll be giving Moq another try this evening, and I think understanding the "mockist" approach will give me what I need to make Moq work better for me.
I'd like to brush my brain to avoid confusions. In few words, what can be said about Mocking process in TDD
What's the GREAT idea behind MOCKING?
Mocking frameworks are meant to be used only to avoid accessing DB during tests or they can be used for something else?
For new comers (like me), are all the frameworks equal or I need to choose one for this or that reason?
In addition to eliminating databases and other slow or ancillary concerns from the unit being tested, mocking allows you to start writing tests for a class without having to implement any collaborating classes.
As you design some piece of functionality, you'll realize that you need some other class or service, in order to stick to the single responsibility principle, but then you'll have to implement those to get the first one working, which in turn will demonstrate the need for still more classes.
If you can mock or stub those dependencies, then you can create the interfaces upon which that first class will rely, without actually having to implement anything outside of that class -- just return canned results from stubs of the interfaces.
This is an essential component to a test-first approach.
The GREAT idea: LIMIT THE SCOPE OF YOUR TESTS. By removing dependencies you remove the risk of test failures because of dependencies. That way you can focus on the correctness of the code that USES those dependencies.
Mocking DB's is very common but you can mock any dependency with an interface. In a recent project we mocked a web service, for example. You might even want to mock another business object just to make sure that you aren't relying on the correctness of the logic in that object.
I'd choose whichever one seems easiest to use. Moq is really nice.
I suggest you start here:
Mocks are not Stubs
It probably is the article that got me thinking the right way about Mocks. Sure the mocked object is usually heavy (otherwise it may not be worth mocking) but it doesn't have to be heavy in the sense that has some strong reliance on an external system like a database. It can be just a complex piece that you need to isolate to effectively be testing only your class and not the dependency.
I am trying to change my unit testing of ArcGIS, and start using mocks (I use rhino).
When I started to get into writing the tests, I noticed I have to start mocking a lot of objects, and stub a lot of methods for even a single test to pass.
For example - my controller first gets a RelationshipClass (so I need to stub the IWorkspace and the returned IRelationshipClass), then also gets an IFeature (A stub), and finally calls stubRelClass.GetRelatedObjects(stubFeature), to return an ISet of other IFeatures.
Is it normal to have to stub so many objects and methods just to make it pass? I also feel
like I really need to step over the code (yeah - I know I should have written the tests first, I am still trying this one), in order to figure out what to stub out next, and what I should return.
I am also having problem with mocking com classes which implement more than one interface. In the production code I QI them between the interfaces. How can I create a mock that implements both interfaces at runtime?
Depending on your injection chain, yes, sometimes you have to mock a lot of objects. If you're going multiple levels deep though, it may be indicative of a design fault - objects that are relying on data that is three layers down into your API may not be loosely coupled. You should be able to nip the chain in the bud by just returning a fake object of some kind at some point that has the necessary properties that the layer you're testing at needs.
You should also be able to do most of your mocking in a [SetUp] method and then have each test just change one or two things.
For mocking multiple interfaces, Rhino has the concept of a MultiMock. I believe the syntax you're after is:
var mock =
MockRepository.DynamicMultiMock<MyType>(
typeof(Interface1),
typeof(Interface2),
....);
It might be a sign of high coupling - which in turn implies a need to reduce dependencies (which will improve design and testability). As a rough guideline, an object should have around 4-6 collaborators max. Anything over that would set off my alarms.
How are Mocks meant to be used?
To me it sounds like untestable code, which is a smell :-(
I would recommend reading http://misko.hevery.com/code-reviewers-guide/. The author is coach responsible for teaching google developers in the testing area. In the article he shows how you can write testable and untestable code.
Further recommended reading:
Clean Code (Robert C. Martin) - main focus on how to write clean (which corresponds to testable) code.
Working effectively with legacy code (Michael Feather) - shows ways to get untested and untestable code under control.
I recently discussed with a colleague about mocking. He said that mocking classes is very bad and should not be done, only in few cases.
He says that only interfaces should be mocked, otherwise it's an architecture fault.
I wonder why this statement (I fully trust him) is so correct? I don't know it and would like to be convinced.
Did I miss the point of mocking (yes, I read Martin Fowler's article)
Mocking is used for protocol testing - it tests how you'll use an API, and how you'll react when the API reacts accordingly.
Ideally (in many cases at least), that API should be specified as an interface rather than a class - an interface defines a protocol, a class defines at least part of an implementation.
On a practical note, mocking frameworks tend to have limitations around mocking classes.
In my experience, mocking is somewhat overused - often you're not really interested in the exact interaction, you really want a stub... but mocking framework can be used to create stubs, and you fall into the trap of creating brittle tests by mocking instead of stubbing. It's a hard balance to get right though.
IMHO, what your colleague means is that you should program to an interface, not an implementation. If you find yourself mocking classes too often, it's a sign you broke the previous principle when designing your architecture.
Mocking classes (in contrast to mocking interfaces) is bad because the mock still has a real class in the background, it is inherited from, and it is possible that real implementation is executed during the test.
When you mock (or stub or whatever) an interface, there is no risk of having code executed you actually wanted to mock.
Mocking classes also forces you to make everything, that could possibly be mocked, to be virtual, which is very intrusive and could lead to bad class design.
If you want to decouple classes, they should not know each other, this is the reason why it makes sense to mock (or stub or whatever) one of them. So implementing against interfaces is recommended anyway, but this is mentioned here by others enough.
I would suggest to stay away from mocking frameworks as far as possible. At the same time, I would recommend to use mock/fake objects for testing, as much as possible. The trick here is that you should create built-in fake objects together with real objects. I explain it more in detail in a blog post I wrote about it: http://www.yegor256.com/2014/09/23/built-in-fake-objects.html
Generally you'd want to mock an interface.
While it is possible to mock a regular class, it tends to influence your class design too much for testability. Concerns like accessibility, whether or not a method is virtual, etc. will all be determined by the ability to mock the class, rather than true OO concerns.
There is one faking library called TypeMock Isolator that allows you to get around these limitations (have cake, eat cake) but it's pretty expensive. Better to design for testability.
The answer, like most questions about practices, is "it depends".
Overuse of mocks can lead to tests that don't really test anything. It can also lead to tests which are virtual re-implementations of the code under test, tightly bound to a specific implementation.
On the other hand, judicious use of mocks and stubs can lead to unit tests which are neatly isolated and test one thing and one thing alone - which is a good thing.
It's all about moderation.
It makes sense to mock classes so tests can be written early in the development lifecycle.
There is a tendency to continue to use mock classes even when concrete implementations become available. There is also the tendency to develop against mock classes (and stubs) necessary early in a project when some parts of the system have not been built.
Once a piece of the system has been built it is necessary to test against it and continue to test against it (for regression). In this case starting with mocks is good but they should be discarded in favour of the implementation as soon as possible. I have seen projects struggle because different teams continue to develop against the behaviour of the mock rather than the implementation (once it is available).
By testing against mocks you are assuming that the mock is characteristic of the system. Often this involves guessing what the mocked component will do. If you have a specification of the system you are mocking then you don't have to guess, but often the 'as-built' system doesn't match the original specification due to practical considerations discovered during construction. Agile development projects assume this will always happen.
You then develop code that works with the mock. When it turns out that the mock does not truly represent the behaviour of the real as-built system (eg. latency issues not seen in the mock, resource and efficiency issues not seen in the mock, concurrency issues, performance issues etc) you then have a bunch of worthless mocking tests you must now maintain.
I consider the use of mocks to be valuable at the start of development but these mocks should not contribute to project coverage. It is best later if the mocks are removed and proper integration tests are created to replace them otherwise your system will not be getting tested for the variety of conditions which your mock did not simulate (or simulates incorrectly relative to the real system).
So, the question is whether or not to use mocks, it is a matter of when to use them and when to remove them.
It depends how often you use (or are forced by bad design) mocks.
If instantiating the object becomes too hard (and it happens more than often), then it is a sign the code may need some serious refactoring or change in design (builder? factory?).
When you mock everything you end up with tests that know everything about your implementation (white box testing). Your tests no longer document how to use the system - they are basically a mirror of its implementation.
And then comes potential code refactoring..
From my experience it's one of the biggest issues related to overmocking. It becomes painful and takes time, lots of it.
Some developers become fearful of refactoring their code knowing how long will it take.
There is also question of purpose - if everything is mocked, are we really testing the production code?
Mocks of course tend to violate DRY principle by duplicating code in two places: once in the production code and once in the tests.
Therefore, as I mentioned before, any change to code has to be made in two places (if tests aren't written well, it can be in more than that..).
Edit: Since you have clarified that your colleague meant mock class is bad but mock interface is not, the answer below is outdated. You should refer to this answer.
I am talking about mock and stub as defined by Martin Fowler, and I assume that's what your colleague meant, too.
Mocking is bad because it can lead to overspecification of tests. Use stub if possible and avoid mock.
Here's the diff between mock and stub (from the above article):
We can then use state verification on
the stub like this.
class OrderStateTester...
public void testOrderSendsMailIfUnfilled() {
Order order = new Order(TALISKER, 51);
MailServiceStub mailer = new MailServiceStub();
order.setMailer(mailer);
order.fill(warehouse);
assertEquals(1, mailer.numberSent());
}
Of course this is a very simple test -
only that a message has been sent.
We've not tested it was send to the
right person, or with the right
contents, but it will do to illustrate
the point.
Using mocks this test would look quite
different.
class OrderInteractionTester...
public void testOrderSendsMailIfUnfilled() {
Order order = new Order(TALISKER, 51);
Mock warehouse = mock(Warehouse.class);
Mock mailer = mock(MailService.class);
order.setMailer((MailService) mailer.proxy());
mailer.expects(once()).method("send");
warehouse.expects(once()).method("hasInventory")
.withAnyArguments()
.will(returnValue(false));
order.fill((Warehouse) warehouse.proxy());
}
}
In order to use state verification on the stub, I need to make some extra methods on the >stub to help with verification. As a result the stub implements MailService but adds extra >test methods.
Recently there has been quite some hype around all the different mocking frameworks in the .NET world. I still haven't quite grasped what is so great about them. It doesn't seem to be to hard to write the mocking objects I need myself. Especially with the help of Visual Studio I quickly can write a class that implements the interface I want to mock (it auto-generates almost everything for me) and then write an implementation for the method(s) I need for my test. Done! Why going through the hassle of understanding a mocking framework for the sole purpose of saving a few lines of code. Or is a mocking framework not only about saving lines of code?
Once I finally got the hang of mock objects, I realized that they're essential for unit testing for the same reason that double blind testing or control groups are essential for scientific trials: they isolate what you're actually testing.
If you're testing a class which has quite a bit of interaction via other interfaces, you not only save the lines of code on having to mock each and every interface, but you also gain the ability to do things like "throw an exception if an unexpected method is called" or "exception if these methods are called out of order". You can get remarkably sophisticated with mock frameworks, and though I'll quickly admit there's a large learning curve, when you get up to speed they'll help make your unit tests more thorough without being bloated.
You actually identified one of the key points of a mock framework in your question. The fact that you code the mocks yourself is not something the developer should be concerned with. The mocking frameworks give you implementations of interfaces programatically, plus they are functional (based on your setup of the mock).
What do you do if you are testing an ICustomerDAO, for example, and you want to test some method 14 times each with different outcomes? Implement 14 different classes manually? I doubt that anyone would want to do that.
Mocks give you the power to define what will happen with parts of your classes when you are not concerned with whether or not they will actually work, like throwing exceptions whenever you want them to, returning zero results and making sure you handle that correctly, etc...
They are a great unit testing tool.
Previous questions that may help:
What is a mock and when should you use it?
Mockist vs classical TDD
I find that using a mocking framework allows me to generate tests a lot faster and with better verification that what I expect to happen in the test actually is happengin. I have in the past implemented stubs or fakes myself. I found that I needed to generate stubs specific to the test that I wanted and this took a lot of time. I can create the same test much faster using a mocking framework. The good ones support the generation of fakes, stubs or mocks with straightforward syntax.
It takes a while to get the hang of it, I avoided it for a while but now wouldn't try to work without a mocking framework for the reasons #Chamelaeon states.
Roy Osherove had a poll about Mock Frameworks and down in the comment section, there is a discussion (albeit brief) about whether one needs a Mock Framework or not.
I personally have been manually doing exactly as you stated and it has worked well enough, but this has mainly been out of habit rather than a closely-held opinion on mock frameworks in general.
Well I certainly don't think that you NEED a mocking framework. It's a framework like any other, and it's ultimately designed to save you some time and effort. You can also do things like roll your own common data structures like stacks and queues, but isn't it generally easier to just use the ones built into the class libraries that ship with the compiler/IDE of your language of choice?
I'm sure there are other compelling reasons for using mocking frameworks, though I'd leave it to the TDD and unit testing gurus to answer.
For the same reason you wouldn't try to write unit tests without NUnit. A mocking framework will assist you in verifying state and behavior over hundreds of unit tests. It's worth the 2 weeks or so of pain to get up to speed and really helps you focus on what needs to be tested.
One thing that troubles me about a mocking framework is that "what a function should o/p given an i/p" via
when(mock.someMethod("some arg")).thenReturn("something");
statement is spread across many unit test classes.
Let me elaborate with an example. Lets say there was a DAO Interface function getEmp(int EmpID) which was returning an Employee Object when passed an Employee ID as a parameter. Assume that this function was being mocked by 10 different unit test classes. Now if in the future, this function were changed to return a newer version of the Employee Object, one would have to go to each of the 10 different classes to update this change.
The disadvantages are as follows...
a) I don't know how to figure out all the classes which mock this function so that I can go update this change.
b) My existing test cases which consumes the mock DAO object continue to be blissfully unaware of the changes that have happened to the DAO Interface because the mock has not changed and hence continue to be green.
Ideally, if I were to have coded a single mock class myself and consumed it everywhere, I would have just one place to update for the newer version of the Employee object. Also, once I update at this one place, all my existing test cases which consume the mock would break and I would then know exactly what places I need to go and do an update for the new Employee Object.
Any thoughts on my views..
One of the good things about a mocking framework is that it allows setting expectations on the objects being mocked. With the expectations I can then set up all sorts of conditions to exercise the code thats being tested.
An isolation framework or mocking framework allows you to test the code you want, without its dependencies. It makes for short running tests, allows you to debug quickly, and easily build a safety net of tests around the code. Different frameworks have different features, and as said before - it's a tool, and you should select the right tool for the job.
I've use rhino mocks for a mocking framework. I and 5 other developers used it on a large enterprise application that was an 8 month project. We used tdd on the project. Was it worth it? I guess. Was there such a massive huge selling point to using mocks that I have to use it on every project? In my opinion, no. It is not something that is necessary, it is just a tool that you can use if you want to try it out. Some projects you can roll out your own mock classes as some here say they prefer - it is easier. Other projects are larger and may require a mocking framework. The key word (in my opinion) is MAY require... how much code coverage do you require? To me, that is another consideration to using mocks. The project I did with tdd/rhino mocks we were required to have 80% code coverage so the mocks helped us attain that. If our code coverage requirements were less, for example 40%, we probably would have not used a mocking framework and just wrote our own mock classes as others mention they do.