My company is trying to decide if we are going to standardize on Moq, Rhino Mocks or MS Moles and Stubs.
I know Rhino Mocks and Moles and Stubs fairly well. But I am unfamiliar with Moq. How does the syntax work? Does it support Arrange Act Assert (AAA) like Rhino Mocks (I hear they created it, but I am not sure). Does it have strong typing?
Basically, I am leaning towards Rhino Mocks (using Moles where needed). But I don't want to do that just because I am familiar with Rhino Mocks. If Moq is in fact better or (even more important) easier to use, then I want to learn it and pick that one.
So, any one out there that has used both and feels like giving me a syntax comparison?
I've done a multi-part blog series on the differences between a number of mocking frameworks. Feel free to check it out at http://www.richard-banks.org/2010/07/mocking-comparison-part-1-basics.html
Have you looked at http://code.google.com/p/moq/wiki/QuickStart ?
Moq supports AAA and strong typing (via use of lambdas)
This project compares many mock framework against each other: mocking frameworks compare
Related
A bit confused on the differences of Mock frameworks like NMock vs the VS 2011 Fakes Framework.
Going through MSDN, what I understand is that Fakes allow you to mock your dependencies just like RhinoMock or NMock, however the approach is different, Fakes generates code to achive this functionality but Mocks framework does not. So is my understanding correct? Is Fakes just another Mock framework
Your question was about how the MS Fakes framework is different from NMock and it appears the other answers have resolved some of that, but here is some more information regarding how they are the same and how they are different. NMock is also similar to RhinoMocks and Moq, so I'm grouping them in with NMock.
There are 3 major differences I see right off between NMock/RhinoMocks/Moq and the MS Fakes Framework:
The MS fakes framework uses generated code, much like Accessors in prior versions of Visual Studio instead of generic types. When you want to use the fakes framework for a dependency, you add the assembly that contains the dependency to the references of the test project and then right-click on it to generate the test doubles (stubs or shims). Then when you are testing, you are actually using these generated classes instead. NMock uses generics to accomplish the same thing (i.e. IStudentRepository studentRepository = mocks.NewMock<IStudentRepository>()). In my opinion, the MS Fakes framework approach inhibits code navigation and refactoring from within the tests since you are actually working against a generated class, not your real interface.
The MS fakes framework supplies stubs and moles (shims), whereas NMock, RhinoMocks, and Moq all provide stubs and mocks. I really don't understand MS's decision to not include mocks and I am, personally not a fan of moles for reasons described below.
With the MS fakes framework, you supply an alternative implementation of the methods you want to stub. Within these alternate implementations, you can specify the return values and track information about how or if the method was called. With NMock, RhinoMocks and Moq, you generate a mock object and then use that object to specify stubbed return values or to track interactions (whether and how the methods were called). I find the MS fakes approach more complex and less expressive.
To clarify the difference in what the frameworks provide: NMock, RhinoMocks and Moq all provide two types of test doubles (stubs and mocks). The fakes framework provides stubs and moles (they call them shims), and unfortunately does not include mocks. In order to understand the differences and similarities between NMock and MS Fakes, it is helpful to understand what these different types of test doubles are:
Stubs: Stubs are used when you need to provide a values for methods or properties that will be asked of your test doubles by the method under test. For example, when my method under test calls the DoesStudentExist() method of the IStudentRepository test double, I want it to return true.
The idea of stubs in NMock and MS fakes is the same, but with NMock you would do something like this:
Stub.On(mockStudentRepository).Method("DoesStudentExist").Will(Return.Value(true));
And with MSFakes you would do somethign like this:
IStudentRepository studentRepository = new DataAccess.Fakes.StubIStudentRepository() // Generated by Fakes.
{
DoesStudentExistInt32 = (studentId) => { return new Student(); }
};
Notice in the MS Fakes example you create an entirely new implementation for the DoesStudentExist method (Note that it is called DoesStudentExistInt32 because the fakes framework appends the parameter data types to the method names when it generates the stub objects, I think this obscures the clarity of the tests). To be honest the NMock implementation also bugs me because it uses a string to identify the method name. (Forgive me if I've misunderstood how NMock is intended to be used.) This approach really inhibits refactoring and I'd highly recommend RhinoMocks or Moq over NMock for this reason.
Mocks: Mocks are used to verify interaction between your method under test and its dependencies. With NMock, you do this by setting expectations similar to this:
Expect.Once.On(mockStudentRepository).Method("Find").With(123);
This is another reason why I'd prefer RhinoMocks and Moq over NMock, NMock uses the older expectation style whereas RhinoMocks and Moq both support the Arrange/Act/Assert approach where you specify you expected interactions as assertions at the end of the test like this:
stubStudentRepository.AssertWasCalled( x => x.Find(123));
Again, note that RhinoMocks uses a lambda instead of a string to identify the method. The ms fakes framework does not provide mocks at all. This means that in your stubbed out implementations (see the description of stubs above) you have to set variables that you later verify were set correctly. That would look something like this:
bool wasFindCalled = false;
IStudentRepository studentRepository = new DataAccess.Fakes.StubIStudentRepository()
{
DoesStudentExistInt32 = (studentId) =>
{
wasFindCalled = true;
return new Student();
}
};
classUnderTest.MethodUnderTest();
Assert.IsTrue(wasFindCalled);
I find this approach to be a little convoluted since you have to track the call up in the stub and then assert it later in the test. I find the NMock, and especially the RhinoMocks, examples to be more expressive.
Moles (Shims): To be frank, I do not like moles, because of their potential for misuse. One of the things I like so much about unit testing (and TDD in particular) is that testing your code helps you to understand where you have written poor code. This is because testing poorly written code is difficult. This is not true when using moles because moles are actually designed to allow you to test against dependencies that are not injected or to test private methods. They work similarly to stubs, except that you use a ShimsContext like this:
using (ShimsContext.Create())
{
System.Fakes.ShimDateTime.NowGet = () => { return new DateTime(fixedYear, 1, 1); };
}
My worry with shims is that people will start seeing them as "an easier way to unit test" because it doesn't force you to write code the way you should. For a more complete write-up on this concept see this post of mine:
https://jcoop.io/2012/05/21/solid-code-for-solid-reasons/
For more information on some concerns related to the fakes frameworks take a look at these posts:
https://jcoop.io/2012/03/16/38/
If you're interested in learning RhinoMocks here's a Pluralsight training video (full disclosure - I wrote this course and get paid royalties for views, but I think it applies to this discussion so I'm including it here):
http://www.pluralsight.com/training/Courses/TableOfContents/rhinomock-fundamentals
You are correct, but there's more to the story. The most important things to take away from this answer are:
Your architecture should make proper use of stubs and dependency injection, rather than relying on the crutch of Fakes and mocks
Fakes and mocks are useful for testing code you shouldn't or can't change, such as:
Legacy code that does not make use (or efficient use) of stubs
3rd party APIs
Resources for which you have no source code
Shims (known as "Moles", during development) is indeed a mocking framework that operates by way of detouring calls. Instead of painstakingly building a mock (yes, even using Moq is relatively painful!), shims simply use the production code object already in place. Shims simply re-route the call from the production target to the test delegate.
Stubs are generated from interfaces in the target project. The Stub object is just that -- an implementation of the interface. The benefit to using the Stub type is that you can quickly generate a stub without cluttering up your test project with many one-time use stubs, not to mention wasting time creating them. Of course, you should still create concrete stubs, for use across many tests.
Efficiently implementing Fakes (Shims, Mocks and Stub types) takes a little getting used to, but is well worth the effort. I have personally saved weeks of development time, through use of the Shims/Mole, Mocks, and Stub types. I hope you have as much fun with the technology as I have!
As I understand it, the Visual Studio team wanted to avoid competing with the various mock libraries available for .NET. MS often faces hard decisions like this. They are dammed if they don't provide certain functionality ("why doesn't MS provide us with a mock library; mocks are such a common requirement?") and damned if they do ("why is Microsoft acting so aggressively and driving its natural supporters out of the market?") Very often, but not always, they decide to hold back from simply providing their own alternative to available and well-received technologies. That seems to be the case here.
The shim feature of Fakes is really, really useful. Sure, there are dangers. It takes some discipline to ensure you only use this where necessary. However, it fills a big gap. My main complaint is that it is only delivered with the Ultimate edition of VS 2012 and therefore will only be available to a subsection of the .NET development community. What a pity.
Fakes includes two different kinds of "fake" object. The first, called a "stub", is essentially an auto-generated dummy whose default behaviour can (and usually would) be overridden to make it a more interesting mock. It does, however, lack some of the features that most of the currently available mocking frameworks offer. For example, if you want to check that a method on a stub instance was invoked, you would need to add the logic for this yourself. Basically, if you're authoring your own mocks manually now, stubs would probably seem like an improvement. However, if you're already using a more full-featured mocking framework, you might feel like there are some important pieces missing from Fakes stubs.
The other category of object offered by Fakes, called a "shim", exposes a mechanism for replacing behaviour of dependencies that have not been (or cannot be) decoupled adequately for standard replacement via mocks. AFAIK, TypeMock is the only one of the major mocking frameworks that currently offers this sort of functionality.
BTW, if you have tried out Moles before, Fakes is essentially the same thing, finally making its way out of Microsoft Research and into an actual product.
Regarding Fake (Shim + Stub) objects, it has been well defined above, though I guess the last paragraph in the last comment summarizes the whole situation pretty well.
Though a lot of people will argue that Fake (Shim + Stub) objects are good assets to have in some unit testing cases, the downside is that no matter if you're using Visual Studio 2012 or Visual Studio 2013, these options are ONLY available with Premium or Ultimate versions. IOW, this means that you WILL NOT be to run ANY of those Fakes (Shim + Stub) on any Pro version.
You may probably see the Fakes (Shim + Stub) menu option on Pro versions, but beware, there are some pretty strong chances that you will end up with absolutely nothing... It won't generate any compilation error telling you that something important is missing, options are just not there, so don't waste your time...
It's an important factor to consider in a dev team, especially if one is the only one using Ultimate version while everybody else uses Pro version... Moq on the other hand can easily be installed through Nuget no matter which Visual Studio version you use. I had no problem using Moq, the key with any tool is to know what they're used for and how to use them properly ;)
I'm using Rhino Mocks in my unit test.
I would like to know the difference between STUBS and MOCKS (mocks.Stub<T>() and mocks.StrictMock<T>()).
I think it had been asked before.
It is generally the same with the following differences:
Strict Mocks throw exceptions on each call which had not been expected
Dynamic Mocks accept unexpected calls und just return default values (eg. null)
Stubs are like dynamic mocks but have "property behaviour" turned on by default. This allows writing and reading properties like fields, but doesn't allow Stub, Expect nor AssertWasCalled on properties. This behaviour can also be configured on a normal Mock too. But for stubs it is the default.
Since Rhino changed to AAA syntax, it is discouraged to use Strict Mocks, since they make test code very hard to maintain.
Similar questions:
What are the differences between mocks and stubs on Rhino Mocks?
Rhino Mocks - Difference between GenerateStub<T> & GenerateMock<T>
I can't find the information about the Rhino implementation, so I don't mark this question as duplicate.
I just watched this funny YouTube Video about unit testing (it's Hitler with fake subtitles chewing out his team for not doing good unit tests--skip it if you're humor impaired) where stubs get roundly criticized. But I don't understand what wrong with stubs.
I haven't started using a mocking framework and I haven't started feeling the pain from not using one.
Am I in for a world a hurt sometime down the line, having chosen handwritten stubs and fakes instead of mocks (like Rhinomock etc)? (using Fowler's taxonomy)
What are the considerations for picking between a mock and handwritten stub?
There is nothing wrong with stubs, there is room for stubs, mocks... and spies. All are "test doubles", but with different purposes as explained in Mocks and Stubs aren't Spies:
[...] Before moving on, I'd like to
clarify and define some terms in use
here, which I originally discovered in
Gerard Meszaros' xUnit Patterns
book.
A Dummy Object is a placeholder object passed to the system under test
but never used.
A Test Stub provides the system under test with indirect input
A Test Spy provides a way to verify that the system under test performed
the correct indirect output
A Mock Object provides the system under test with both indirect input
and a way to verify indirect output
[...] And you can let this handy chart
guide your decisions:
PS: Mockito - The New Mock Framework on the Block is worth the read too.
I use the following terminology (introduced by Roy Osherove, the author of the Art of Unit-Testing):
A fake is called a stub if you tell it to fake something in case a method is called with such and such parameters. But if you also verify that such call actually took place or took place exactly N times, then such fake is called a mock. In short. a fake is a stub unless you call Verify() on it and then it's a mock.
Obviously, you will need to use stubs in some cases and mocks in others. So, criticizing stubs roundly is probably wrong and using stubs exclusively is probably wrong as well.
If you haven't started using a mocking framework (alternative term: isolation framework), you should keep an eye on them and reevaluate your options frequently. I went from manual mocks to NMock2 to Moq very quickly. Here's an interesting poll of programmers that shows what they use. Manual mocks/stubs are in the minority, but aren't that uncommon.
Mocks are just a lot easier to throw in. They are a real instance of your class, pre-stubbed with the ability to override the action of any method with minimal boilerplate.
There are lots of little considerations such as: If you don't want to deal with any of the methods, you can either have it act as a no-op or fail the test--your choice--but either way virtually no code.
How much boilerplate do you get when you stub out a class? How do you handle it if your class is final? Do you play tricks to get your stub on the classpath first, or do you use different source?
I recommend just starting with the mocks--It's easier all around.
There is nothing wrong with using stubs instead of mocks.
If you want to get technical, mocks are "smart" objects with expectations that can be verified. Stubs are dummy objects that return preset values. See Mocks Aren't Stubs.
But many people (myself included) prefer to do state testing with stubs rather than behavior testing with mocks. You inject a stub into the class under test, you call a method, then you check the state of the class under test. It tends to make for less brittle tests than asserting that the class's internals called method X of a mock object with argument Y.
I don't think you're in for a world of hurt. If you haven't started feeling the pain, you probably don't need an isolation/mocking framework yet. When and if you do, having handwritten stubs/fakes around isn't going to hurt anything.
If you have a lot of interfaces, or if your interfaces have a lot of methods, an isolation/mocking framework can save a lot of time over hand-coding stubs.
I like Moq a lot; I find it easier to use than Rhino Mocks for creating stubs.
Mocks and stubs is used to achieve a real unit test. You just mock all the dependencies, and unit test your class in isolation.
I'm currently using MOQ for mocking and stubbing.
I am just starting to do Test Driven Development, and I am wondering the major differences between RhinoMock, TypeMock, and NUnit's built-in mocking?
Any information would be greatly appreciated!
TypeMock is a commercial product (meaning you'll have to pay for it) but will allow you to mock concrete objects - unlike RhinoMocks/NUnit/MoQ which can only mock an interface/abstract class. How it achieves this is borderline black magic, but it does some very clever things with the CLR.
This can be particularly useful when you use libraries in your project that don't use many interfaces. So you could, for example, use TypeMock to mock out a LINQtoSQL datacontext, or Sharepoint objects. However, if you are using TypeMock this is no excuse for bad design in your application.
As far as I'm aware, aside from minor syntax differences, most of the mocking frameworks have moved away from the old record/playback model. Commonly, you set up your mocks by writing expectations using a Fluent Interface.
Personally, I have only used MoQ and I <3 it.
A video called TDD - Understanding Mock Objects by Roy Osherove is very helpful in learning the differences of the different mocking libraries. He doesn't go in great detail of every aspect, but enough for you to understand. I hope this helps. Roy is also the Chief Architect for TypeMock and is a very influential figure in the unit testing arena. I couldn't recommend this video enough for someone who wants to learn how to use mocking and also learn about the library's available.
The main difference between TypeMock and the open-source library's is that TypeMock uses the Profiler API provided by Microsoft instead of a dynamic proxy. This allows TypeMock to mock concrete classes and static methods. In case you aren't sure what the profiler is, it is the same API that is used by tools like JetBrain's dotTrace and RedGate's Ants .Net profilers. TypeMock just uses the API in a different way to fake(mock) what you tell it to.
#RichardOD, thanks for the reminder, his book "The Art of Unit Testing" goes into greater detail where the video doesn't. I own the book and it is very informative.
Rhino.Mocks is an open source, continuously developed and improving framework by one of the industry's most prolific developers. It has been around for a while and hence supports quite a few different paradigms for mocking. It can be a little tougher to learn therefore in the sense that you might find tutorials for the "old" way of doing things. Here's a tip, SetUpResultFor() and Expect.Call() are the old ways of doing things. The new way is mockObject.AssertWasCalled().
I have not had any personal experience with these others but...
MOQ is an open source, continuously developed and improving framework by one of the industry's somewhat less prolific developers (as compared to Ayende). It is newer and therefore lacks some features that Rhino.Mocks has. This is usually not a problem since these features tend to be ones that are somewhat deprecated in Rhino. I have heard that because of this it is slightly easier to learn (mocking frameworks aren't hard to learn by the way).
NUnit Mocks is very quaint as far as mocking goes. It doesn't support the currently preferred Arrange-Act-Assert syntax relying instead on Expect-Verify (record/replay). It also relies on strings to identify method and property names instead of lambdas. This makes it significantly resistant to refactoring. This is a serious problem. I would not recommend it.
TypeMock Isolater is a hardcore for-pay mocking framework from a company (owned by?) Roy Osherove - a guy who knows his testing but also has some mildly controversial opinions as to how to apply it. It is really intense as far as what it can do - getting down to the low level and modifying how CLR objects work. The philosophy behind TypeMock isn't really 100% TDD however. Part of the benefits of TDD is that by embracing the limitations of Mocking frameworks you will design better code. TypeMock blasts those limitations to pieces. As far as I know it is mostly used by people who are trying to get code they have no control over under test.
I use TypeMock all the time and have found it to be a very powerful tool that can improve the coverage of my unit tests. This is because I work with SharePoint and only TypeMock can allow me to mock out SharePoint classes - since they are concrete classes and not interfaces.
Mocking SharePoint classes is not possible with RhinoMock, Moq, NUNit, etc since (I believe) they require interfaces to mock objects, rather then being able to mock the actual concrete classes.
If your code does use a lot of interfaces, and you don't require mocking concrete classes then TypeMock is a bit pricey, but for the power you get, it's worth it.
Recently there has been quite some hype around all the different mocking frameworks in the .NET world. I still haven't quite grasped what is so great about them. It doesn't seem to be to hard to write the mocking objects I need myself. Especially with the help of Visual Studio I quickly can write a class that implements the interface I want to mock (it auto-generates almost everything for me) and then write an implementation for the method(s) I need for my test. Done! Why going through the hassle of understanding a mocking framework for the sole purpose of saving a few lines of code. Or is a mocking framework not only about saving lines of code?
Once I finally got the hang of mock objects, I realized that they're essential for unit testing for the same reason that double blind testing or control groups are essential for scientific trials: they isolate what you're actually testing.
If you're testing a class which has quite a bit of interaction via other interfaces, you not only save the lines of code on having to mock each and every interface, but you also gain the ability to do things like "throw an exception if an unexpected method is called" or "exception if these methods are called out of order". You can get remarkably sophisticated with mock frameworks, and though I'll quickly admit there's a large learning curve, when you get up to speed they'll help make your unit tests more thorough without being bloated.
You actually identified one of the key points of a mock framework in your question. The fact that you code the mocks yourself is not something the developer should be concerned with. The mocking frameworks give you implementations of interfaces programatically, plus they are functional (based on your setup of the mock).
What do you do if you are testing an ICustomerDAO, for example, and you want to test some method 14 times each with different outcomes? Implement 14 different classes manually? I doubt that anyone would want to do that.
Mocks give you the power to define what will happen with parts of your classes when you are not concerned with whether or not they will actually work, like throwing exceptions whenever you want them to, returning zero results and making sure you handle that correctly, etc...
They are a great unit testing tool.
Previous questions that may help:
What is a mock and when should you use it?
Mockist vs classical TDD
I find that using a mocking framework allows me to generate tests a lot faster and with better verification that what I expect to happen in the test actually is happengin. I have in the past implemented stubs or fakes myself. I found that I needed to generate stubs specific to the test that I wanted and this took a lot of time. I can create the same test much faster using a mocking framework. The good ones support the generation of fakes, stubs or mocks with straightforward syntax.
It takes a while to get the hang of it, I avoided it for a while but now wouldn't try to work without a mocking framework for the reasons #Chamelaeon states.
Roy Osherove had a poll about Mock Frameworks and down in the comment section, there is a discussion (albeit brief) about whether one needs a Mock Framework or not.
I personally have been manually doing exactly as you stated and it has worked well enough, but this has mainly been out of habit rather than a closely-held opinion on mock frameworks in general.
Well I certainly don't think that you NEED a mocking framework. It's a framework like any other, and it's ultimately designed to save you some time and effort. You can also do things like roll your own common data structures like stacks and queues, but isn't it generally easier to just use the ones built into the class libraries that ship with the compiler/IDE of your language of choice?
I'm sure there are other compelling reasons for using mocking frameworks, though I'd leave it to the TDD and unit testing gurus to answer.
For the same reason you wouldn't try to write unit tests without NUnit. A mocking framework will assist you in verifying state and behavior over hundreds of unit tests. It's worth the 2 weeks or so of pain to get up to speed and really helps you focus on what needs to be tested.
One thing that troubles me about a mocking framework is that "what a function should o/p given an i/p" via
when(mock.someMethod("some arg")).thenReturn("something");
statement is spread across many unit test classes.
Let me elaborate with an example. Lets say there was a DAO Interface function getEmp(int EmpID) which was returning an Employee Object when passed an Employee ID as a parameter. Assume that this function was being mocked by 10 different unit test classes. Now if in the future, this function were changed to return a newer version of the Employee Object, one would have to go to each of the 10 different classes to update this change.
The disadvantages are as follows...
a) I don't know how to figure out all the classes which mock this function so that I can go update this change.
b) My existing test cases which consumes the mock DAO object continue to be blissfully unaware of the changes that have happened to the DAO Interface because the mock has not changed and hence continue to be green.
Ideally, if I were to have coded a single mock class myself and consumed it everywhere, I would have just one place to update for the newer version of the Employee object. Also, once I update at this one place, all my existing test cases which consume the mock would break and I would then know exactly what places I need to go and do an update for the new Employee Object.
Any thoughts on my views..
One of the good things about a mocking framework is that it allows setting expectations on the objects being mocked. With the expectations I can then set up all sorts of conditions to exercise the code thats being tested.
An isolation framework or mocking framework allows you to test the code you want, without its dependencies. It makes for short running tests, allows you to debug quickly, and easily build a safety net of tests around the code. Different frameworks have different features, and as said before - it's a tool, and you should select the right tool for the job.
I've use rhino mocks for a mocking framework. I and 5 other developers used it on a large enterprise application that was an 8 month project. We used tdd on the project. Was it worth it? I guess. Was there such a massive huge selling point to using mocks that I have to use it on every project? In my opinion, no. It is not something that is necessary, it is just a tool that you can use if you want to try it out. Some projects you can roll out your own mock classes as some here say they prefer - it is easier. Other projects are larger and may require a mocking framework. The key word (in my opinion) is MAY require... how much code coverage do you require? To me, that is another consideration to using mocks. The project I did with tdd/rhino mocks we were required to have 80% code coverage so the mocks helped us attain that. If our code coverage requirements were less, for example 40%, we probably would have not used a mocking framework and just wrote our own mock classes as others mention they do.