I have seen the following questions already:
"Is it possible to mock out a .NET HttpWebResponse?"
"Mocking WebResponse's from a WebRequest"
These do not help me because I need:
To test existing code which uses WebRequest/WebResponse directly, with no interfaces
The existing code depends on HttpWebRequest / HttpWebResponse, so I can't use my own derived WebRequest and WebResponse classes.
I know about WebRequest.RegisterPrefix, and have a successful unit test which proves that it works (once you get the prefix right). But that test simply tests WebRequest and WebResponse.
I tried the following, but it gives a compile error (not a warning):
public class MockWebRequest : HttpWebRequest
{
public MockWebRequest()
// : base(new SerializationInfo(typeof (HttpWebRequest), new FormatterConverter()),
// new StreamingContext())
{
}
}
With the two lines commented-out, I get a compile error:
error CS0619: 'System.Net.HttpWebRequest.HttpWebRequest()' is obsolete: 'This API supports the .NET Framework infrastructure and is not intended to be used directly from your code.'
When I uncomment those lines, I get a 618 warning, but the code crashes at runtime:
System.Runtime.Serialization.SerializationException: Member '_HttpRequestHeaders' was not found.
at System.Runtime.Serialization.SerializationInfo.GetElement(String name, Type& foundType)
at System.Runtime.Serialization.SerializationInfo.GetValue(String name, Type type)
at System.Net.HttpWebRequest..ctor(SerializationInfo serializationInfo, StreamingContext streamingContext)
at UnitTests.MockWebRequest..ctor(String responseJson)
at UnitTests.MockWebRequestCreator.Create(Uri uri)
at System.Net.WebRequest.Create(Uri requestUri, Boolean useUriBase)
at System.Net.WebRequest.Create(Uri requestUri)
at code under test
I'm at a loss about how to proceed with this (other than the default, which is to just punt on these unit tests and wish the code had been implemented with testing in mind).
I could probably get "down and dirty" and do something nasty with ISerializable, or something like that, but this brings up the third requirement:
3. The code must be maintainable, and must not be too much more complex than the code being tested!
OBTW, I can't use TypeMock for this, and in fact there's a bit of a time limit. The project is almost done, so a new, purchased mocking framework is out of the question.
I just ran into the same problem. Frankly, the way Microsoft has set this up doesn't make much sense - in WebRequest.Create they have an abstract factory, but they neither provide interfaces for HttpWebRequest and HttpWebResponse that we can implement, nor do they provide a protected constructor that we can use to inherit from these classes.
The solution I have come up with requires some minor changes to the main program, but it works. Basically, I am doing what Microsoft should have done.
Create interfaces IHttpWebRequest and IHttpWebResponse
Create mock classes which implement the interfaces
Create wrappers for HttpWebRequest and HttpWebResponse which implement the interfaces
Replace calls to WebRequest.Create with my own factory method that returns the wrappers or the mocks, as needed
Depending on how your code is structured, the above may be relatively trivial to implement, or it may require making changes throughout your project. In my case, we have all of our HTTP requests going through a common service, so it hasn't been too bad.
Anyway, probably too late for your project, but maybe this will help someone else down the road.
I generally prefer to use a mocking framework like MOQ when mocking asp.net singletons like httpcontext.
There are other frameworks out there for mocking, but I find MOQ to be very flexible and intuitive
https://github.com/Moq/moq4/wiki/Quickstart
You can do something similar to this
http://www.syntaxsuccess.com/viewarticle/how-to-mock-httpcontext
Another approach is to abstract the call to httpcontext through a virtual method:
Similar to this technique:
http://unit-testing.net/CurrentArticle/How-To-Remove-Data-Dependencies-In-Unit-Tests.html
Related
I have read it here. It is close to my problem and I need more guidance on this part.
Given,
class Journey{
fun start(){
Service.getInstance().getProductInfo(activity,object: Service.Callback<Product>{
override fun onSuccess(data: Product) {
showProductInfo(activity, data, customer)
}
override fun onError(e: Throwable) {
showError(e)
}
})
}
}
and I want to mock Service.getProductInfo and perform happy path and unhappy path handling, how do I do it with mockk?
Additional question, is there a way to exclude certain function like start above in jacoco? Sometimes some functions has not much meaning for unit test and excluding it in jacoco makes much more sense.
UPDATE:
I learnt from Gradle website that it does not currently support method exclusion.
Callback interface is below
interface Callback<T>{
fun onSuccess(data: T)
fun onError(e: Throwable)
}
I'm using retrofit for Service.
First, you don't want to exclude your code from code coverage check, because it is your own code and it has to be tested.
For the main problem. In this particular (because mainly I would not use this design at all) case I would go with the way of having ability to mock getInstance method first, so it would return any mock you want to use in your tests. First thing that come to mind is using PowerMockito (or similar mocking frameworks).
PowerMockito allows you to mock static method (see https://github.com/powermock/powermock/wiki/mockstatic) or, which is a bit worse in my mind, but is totally up to you) mock construction of object inside factory method with https://github.com/powermock/powermock/wiki/MockConstructor
My least favorite approach (if you somehow cannot use PowerMock in your code) to do so (as far as I understand it is static method of Service class) can be found here: https://hackernoon.com/mocking-static-method-in-kotlin-614df89482ae
It is pretty simple but you will need to rewrite you code to use it. But if can rewrite your code, it is better to redesign it to make it more testable.
Edit: just realized that you needed to do it with mockk. I'm not really experienced with it, but AFAIK it has almost same fucntionality with staticMockk. You can see this question to check it: Mock static java methods using Mockk
My colleagues and I are currently introducing unit tests to our legacy Java EE5 codebase. We use mostly JUnit and Mockito. In the process of writing tests, we have noticed that several methods in our EJBs were hard to test because they did a lot of things at once.
I'm fairly new to the whole testing business, and so I'm looking for insight in how to better structure the code or the tests. My goal is to write good tests without a headache.
This is an example of one of our methods and its logical steps in a service that manages a message queue:
consumeMessages
acknowledgePreviouslyDownloadedMessages
getNewUnreadMessages
addExtraMessages (depending on somewhat complex conditions)
markMessagesAsDownloaded
serializeMessageObjects
The top-level method is currently exposed in the interface, while all sub-methods are private. As far as I understand it, it would be bad practice to just start testing private methods, as only the public interface should matter.
My first reaction was to just make all the sub-methods public and test them in isolation, then in the top-level method just make sure that it calls the sub-methods. But then a colleague mentioned that it might not be a good idea to expose all those low-level methods at the same level as the other one, as it might cause confusion and other developers might start using when they should be using the top-level one. I can't fault his argument.
So here I am.
How do you reconcile exposing easily testable low-level methods versus avoiding to clutter the interfaces? In our case, the EJB interfaces.
I've read in other unit test questions that one should use dependency injection or follow the single responsibility principle, but I'm having trouble applying it in practice. Would anyone have pointers on how to apply that kind of pattern to the example method above?
Would you recommend other general OO patterns or Java EE patterns?
At first glance, I would say that we probably need to introduce a new class, which would 1) expose public methods that can be unit tested but 2) not be exposed in the public interface of your API.
As an example, let's imagine that you are designing an API for a car. To implement the API, you will need an engine (with complex behavior). You want to fully test your engine, but you don't want to expose details to the clients of the car API (all I know about my car is how to push the start button and how to switch the radio channel).
In that case, what I would do is something like that:
public class Engine {
public void doActionOnEngine() {}
public void doOtherActionOnEngine() {}
}
public class Car {
private Engine engine;
// the setter is used for dependency injection
public void setEngine(Engine engine) {
this.engine = engine;
}
// notice that there is no getter for engine
public void doActionOnCar() {
engine.doActionOnEngine();
}
public void doOtherActionOnCar() {
engine.doActionOnEngine();
engine.doOtherActionOnEngine(),
}
}
For the people using the Car API, there is no way to access the engine directly, so there is no risk to do harm. On the other hand, it is possible to fully unit test the engine.
Dependency Injection (DI) and Single Responsibility Principle (SRP) are highly related.
SRP is basicly stating that each class should only do one thing and delegate all other matters to separate classes. For instance, your serializeMessageObjects method should be extracted into its own class -- let's call it MessageObjectSerializer.
DI means injecting (passing) the MessageObjectSerializer object as an argument to your MessageQueue object -- either in the constructor or in the call to the consumeMessages method. You can use DI frameworks to do this for, but I recommend to do it manually, to get the concept.
Now, if you create an interface for the MessageObjectSerializer, you can pass that to the MessageQueue, and then you get the full value of the pattern, as you can create mocks/stubs for easy testing. Suddenly, consumeMessages doesn't have to pay attention to how serializeMessageObjects behaves.
Below, I have tried to illustrate the pattern. Note, that when you want to test consumeMessages, you don't have to use the the MessageObjectSerializer object. You can make a mock or stub, that does exactly what you want it to do, and pass it instead of the concrete class. This really makes testing so much easier. Please, forgive syntax errors. I did not have access to Visual Studio, so it is written in a text editor.
// THE MAIN CLASS
public class MyMessageQueue()
{
IMessageObjectSerializer _serializer;
//Constructor that takes the gets the serialization logic injected
public MyMessageQueue(IMessageObjectSerializer serializer)
{
_serializer = serializer;
//Also a lot of other injection
}
//Your main method. Now it calls an external object to serialize
public void consumeMessages()
{
//Do all the other stuff
_serializer.serializeMessageObjects()
}
}
//THE SERIALIZER CLASS
Public class MessageObjectSerializer : IMessageObjectSerializer
{
public List<MessageObject> serializeMessageObjects()
{
//DO THE SERILIZATION LOGIC HERE
}
}
//THE INTERFACE FOR THE SERIALIZER
Public interface MessageObjectSerializer
{
List<MessageObject> serializeMessageObjects();
}
EDIT: Sorry, my example is in C#. I hope you can use it anyway :-)
Well, as you have noticed, it's very hard to unit test a concrete, high-level program. You have also identified the two most common issues:
Usually the program is configured to use specific resources, such as a specific file, IP address, hostname etc. To counter this, you need to refactor the program to use dependency injection. This is usually done by adding parameters to the constructor that replace the ahrdcoded values.
It's also very hard to test large classes and methods. This is usually due to the combinatorical explosion in the number of tests required to test a complex piece of logic. To counter this, you will usually refactor first to get lots more (but shorter) methods, then trying to make the code more generic and testable by extracting several classes from your original class that each have a single entry method (public) and several utility methods (private). This is essentially the single responsibility principle.
Now you can start working your way "up" by testing the new classes. This will be a lot easier, as the combinatoricals are much easier to handle at this point.
At some point along the way you will probably find that you can simplify your code greatly by using these design patterns: Command, Composite, Adaptor, Factory, Builder and Facade. These are the most common patterns that cut down on clutter.
Some parts of the old program will probably be largely untestable, either because they are just too crufty, or because it's not worth the trouble. Here you can settle for a simple test that just checks that the output from known input has not changed. Essentially a regression test.
So I have a factory class and I'm trying to work out what the unit tests should do. From this question I could verify that the interface returned is of a particular concrete type that I would expect.
What should I check for if the factory is returning concrete types (because there is no need - at the moment - for interfaces to be used)? Currently I'm doing something like the following:
[Test]
public void CreateSomeClassWithDependencies()
{
// m_factory is instantiated in the SetUp method
var someClass = m_factory.CreateSomeClassWithDependencies();
Assert.IsNotNull(someClass);
}
The problem with this is that the Assert.IsNotNull seems somewhat redundant.
Also, my factory method might be setting up the dependencies of that particular class like so:
public SomeClass CreateSomeClassWithDependencies()
{
return new SomeClass(CreateADependency(), CreateAnotherDependency(),
CreateAThirdDependency());
}
And I want to make sure that my factory method sets up all these dependencies correctly. Is there no other way to do this then to make those dependencies public/internal properties which I then check for in the unit test? (I'm not a big fan of modifying the test subjects to suit the testing)
Edit: In response to Robert Harvey's question, I'm using NUnit as my unit testing framework (but I wouldn't have thought that it would make too much of a difference)
Often, there's nothing wrong with creating public properties that can be used for state-based testing. Yes: It's code you created to enable a test scenario, but does it hurt your API? Is it conceivable that other clients would find the same property useful later on?
There's a fine line between test-specific code and Test-Driven Design. We shouldn't introduce code that has no other potential than to satisfy a testing requirement, but it's quite alright to introduce new code that follow generally accepted design principles. We let the testing drive our design - that's why we call it TDD :)
Adding one or more properties to a class to give the user a better possibility of inspecting that class is, in my opinion, often a reasonable thing to do, so I don't think you should dismiss introducing such properties.
Apart from that, I second nader's answer :)
If the factory is returning concrete types, and you're guaranteeing that your factory always returns a concrete type, and not null, then no, there isn't too much value in the test. It does allows you to make sure, over time that this expectation isn't violated, and things like exceptions aren't thrown.
This style of test simply makes sure that, as you make changes in the future, your factory behaviour won't change without you knowing.
If your language supports it, for your dependencies, you can use reflection. This isn't always the easiest to maintain, and couples your tests very tightly to your implementation. You have to decide if that's acceptable. This approach tends to be very brittle.
But you really seem to be trying to separate which classes are constructed, from how the constructors are called. You might just be better off with using a DI framework to get that kind of flexibility.
By new-ing up all your types as you need them, you don't give yourself many seams (a seam is a place where you can alter behaviour in your program without editing in that place) to work with.
With the example as you give it though, you could derive a class from the factory. Then override / mock CreateADependency(), CreateAnotherDependency() and CreateAThirdDependency(). Now when you call CreateSomeClassWithDependencies(), you are able to sense whether or not the correct dependencies were created.
Note: the definition of "seam" comes from Michael Feather's book, "Working Effectively with Legacy Code". It contains examples of many techniques to add testability to untested code. You may find it very useful.
What we do is create the dependancies with factories, and we use a dependancy injection framework to substitute mock factories for the real ones when the test is run. Then we set up the appropriate expectations on those mock factories.
You can always check stuff with reflection. There is no need to expose something just for unit tests. I find it quite rare that I need to reach in with reflection and it may be a sign of bad design.
Looking at your sample code, yes the Assert not null seems redundant, depending on the way you designed your factory, some will return null objects from the factory as opposed to exceptioning out.
As I understand it you want to test that the dependencies are built correctly and passed to the new instance?
If I was not able to use a framework like google guice, I would probably do it something like this (here using JMock and Hamcrest):
#Test
public void CreateSomeClassWithDependencies()
{
dependencyFactory = context.mock(DependencyFactory.class);
classAFactory = context.mock(ClassAFactory.class);
myDependency0 = context.mock(MyDependency0.class);
myDependency1 = context.mock(MyDependency1.class);
myDependency2 = context.mock(MyDependency2.class);
myClassA = context.mock(ClassA.class);
context.checking(new Expectations(){{
oneOf(dependencyFactory).createDependency0(); will(returnValue(myDependency0));
oneOf(dependencyFactory).createDependency1(); will(returnValue(myDependency1));
oneOf(dependencyFactory).createDependency2(); will(returnValue(myDependency2));
oneOf(classAFactory).createClassA(myDependency0, myDependency1, myDependency2);
will(returnValue(myClassA));
}});
builder = new ClassABuilder(dependencyFactory, classAFactory);
assertThat(builder.make(), equalTo(myClassA));
}
(if you cannot mock ClassA you can assign a non-mock version to myClassA using new)
I'm basically trying to teach myself how to code and I want to follow good practices. There are obvious benefits to unit testing. There is also much zealotry when it comes to unit-testing and I prefer a much more pragmatic approach to coding and life in general. As context, I'm currently writing my first "real" application which is the ubiquitous blog engine using asp.net MVC. I'm loosely following the MVC Storefront architecture with my own adjustments. As such, this is my first real foray into mocking objects. I'll put the code example at the end of the question.
I'd appreciate any insight or outside resources that I could use to increase my understanding of the fundamentals of testing and mocking. The resources I've found on the net are typically geared towards the "how" of mocking and I need more understanding of the where, why and when of mocking. If this isn't the best place to ask this question, please point me to a better place.
I'm trying to understand the value that I'm getting from the following tests. The UserService is dependent upon the IUserRepository. The value of the service layer is to separate your logic from your data storage, but in this case most of the UserService calls are just passed straight to IUserRepository. The fact that there isn't much actual logic to test could be the source of my concerns as well. I have the following concerns.
It feels like the code is just testing that the mocking framework is working.
In order to mock out the dependencies, it makes my tests have too much knowledge of the IUserRepository implementation. Is this a necessary evil?
What value am I actually gaining from these tests? Is the simplicity of the service under test causing me to doubt the value of these tests.
I'm using NUnit and Rhino.Mocks, but it should be fairly obvious what I'm trying to accomplish.
[SetUp]
public void Setup()
{
userRepo = MockRepository.GenerateMock<IUserRepository>();
userSvc = new UserService(userRepo);
theUser = new User
{
ID = null,
UserName = "http://joe.myopenid.com",
EmailAddress = "joe#joeblow.com",
DisplayName = "Joe Blow",
Website = "http://joeblow.com"
};
}
[Test]
public void UserService_can_create_a_new_user()
{
// Arrange
userRepo.Expect(repo => repo.CreateUser(theUser)).Return(true);
// Act
bool result = userSvc.CreateUser(theUser);
// Assert
userRepo.VerifyAllExpectations();
Assert.That(result, Is.True,
"UserService.CreateUser(user) failed when it should have succeeded");
}
[Test]
public void UserService_can_not_create_an_existing_user()
{
// Arrange
userRepo.Stub(repo => repo.IsExistingUser(theUser)).Return(true);
userRepo.Expect(repo => repo.CreateUser(theUser)).Return(false);
// Act
bool result = userSvc.CreateUser(theUser);
// Assert
userRepo.VerifyAllExpectations();
Assert.That(result, Is.False,
"UserService.CreateUser() allowed multiple copies of same user to be created");
}
Essentially what you are testing here is that the methods are getting called, not whether or not they actually work. Which is what mocks are supposed to do. Instead of calling the method, they just check to see if the method got called, and return whatever is in the Return() statement. So in your assertion here:
Assert.That(result, Is.False, "error message here");
This assertion will ALWAYS succeed because your expectation will ALWAYS return false, because of the Return statement:
userRepo.Expect(repo => repo.CreateUser(theUser)).Return(false);
I'm guessing this isn't that useful in this case.
Where mocking is useful is when you want to, for example, make a database call somewhere in your code, but you don't want to actually call to the database. You want to pretend that the database got called, but you want to set up some fake data for it to return, and then (here's the important part) test the logic that does something with the fake data your mock returned. In the above examples you are omitting the last step. Imagine you had a method that displayed a message to the user that said whether the new user was created:
public string displayMessage(bool userWasCreated) {
if (userWasCreated)
return "User created successfully!";
return "User already exists";
}
then your test would be
userRepo.Expect(repo => repo.CreateUser(theUser)).Return(false);
Assert.AreEqual("User already exists", displayMessage(userSvc.CreateUser(theUser)))
Now this has some value, because you are testing some actual behavior. Of course, you could also just test this directly by passing in "true" or "false." You don't even need a mock for that test. Testing expectations is fine, but I've written plenty of tests like that, and have come to the same conclusion that you are reaching - it just isn't that useful.
So in short, mocking is useful when you want to abstract away externalities, like databases, or webservice calls, etc, and inject known values at that point. But it's not often useful to test mocks directly.
You are right: the simplicity of the service makes these tests uninteresting. It is not until you get more business logic in the service, that you will gain value from the tests.
You might consider some tests like these:
CreateUser_fails_if_email_is_invalid()
CreateUser_fails_if_username_is_empty()
Another comment: it looks like a code-smell, that your methods return booleans to indicate success or failure. You might have a good reason to do it, but usually you should let the exceptions propagate out. It also makes it harder to write good tests, since you will have problems detecting whether your method failed for the "right reason", f.x. you might write the CreateUser_fails_if_email_is_invalid()-test like this:
[Test]
public void CreateUser_fails_if_email_is_invalid()
{
bool result = userSvc.CreateUser(userWithInvalidEmailAddress);
Assert.That(result, Is.False);
}
and it would probably work with your existing code. Using the TDD Red-Green-Refactor-cycle would mitigate this problem, but it would be even better to be able to actually detect that the method failed because of the invalid email, and not because of another problem.
If you write your tests before you write your code, you'll gain much more from your unit tests. One of the reasons that it feels like your tests aren't worth much is that you're not deriving the value of having your tests drive the design. Writing your tests afterwards is mostly just an exercise in seeing if you can remember everything that can go wrong. Writing your tests first causes you to think about how you would actually implement the functionality.
These tests aren't all that interesting because the functionality that is being implemented is pretty basic. The way you are going about mocking seems pretty standard -- mock the things the class under test depends on, not the class under test. Testability (or good design sense) has already led you to implement interfaces and use dependency injection to reduce coupling. You might want to think about changing the error handling, as other have suggested. It would be nice to know why, if only to improve the quality of your tests, CreateUser has failed, for instance. You could do this with exceptions or with an out parameter (which is how MembershipProvider works, if I remember correctly).
You are facing the question of "classical" vs. "mockist" approaches to testing. Or "state-verification" vs. "behaviour-verification" as described by Martin Fowler: http://martinfowler.com/articles/mocksArentStubs.html#ClassicalAndMockistTesting
Another most excellent resource is Gerard Meszaros' book "xUnit Test Patterns: Refactoring Test Code"
I know how I use these terms, but I'm wondering if there are accepted definitions for faking, mocking, and stubbing for unit tests? How do you define these for your tests? Describe situations where you might use each.
Here is how I use them:
Fake: a class that implements an interface but contains fixed data and no logic. Simply returns "good" or "bad" data depending on the implementation.
Mock: a class that implements an interface and allows the ability to dynamically set the values to return/exceptions to throw from particular methods and provides the ability to check if particular methods have been called/not called.
Stub: Like a mock class, except that it doesn't provide the ability to verify that methods have been called/not called.
Mocks and stubs can be hand generated or generated by a mocking framework. Fake classes are generated by hand. I use mocks primarily to verify interactions between my class and dependent classes. I use stubs once I have verified the interactions and am testing alternate paths through my code. I use fake classes primarily to abstract out data dependencies or when mocks/stubs are too tedious to set up each time.
You can get some information :
From Martin Fowler about Mock and Stub
Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production
Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test. Stubs may also record information about calls, such as an email gateway stub that remembers the messages it 'sent', or maybe only how many messages it 'sent'.
Mocks are what we are talking about here: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.
From xunitpattern:
Fake: We acquire or build a very lightweight implementation of the same functionality as provided by a component that the SUT depends on and instruct the SUT to use it instead of the real.
Stub : This implementation is configured to respond to calls from the SUT with the values (or exceptions) that will exercise the Untested Code (see Production Bugs on page X) within the SUT. A key indication for using a Test Stub is having Untested Code caused by the inability to control the indirect inputs of the SUT
Mock Object that implements the same interface as an object on which the SUT (System Under Test) depends. We can use a Mock Object as an observation point when we need to do Behavior Verification to avoid having an Untested Requirement (see Production Bugs on page X) caused by an inability to observe side-effects of invoking methods on the SUT.
Personally
I try to simplify by using : Mock and Stub. I use Mock when it's an object that returns a value that is set to the tested class. I use Stub to mimic an Interface or Abstract class to be tested. In fact, it doesn't really matter what you call it, they are all classes that aren't used in production, and are used as utility classes for testing.
Stub - an object that provides predefined answers to method calls.
Mock - an object on which you set expectations.
Fake - an object with limited capabilities (for the purposes of testing), e.g. a fake web service.
Test Double is the general term for stubs, mocks and fakes. But informally, you'll often hear people simply call them mocks.
I am surprised that this question has been around for so long and nobody has as yet provided an answer based on Roy Osherove's "The Art of Unit Testing".
In "3.1 Introducing stubs" defines a stub as:
A stub is a controllable replacement for an existing dependency
(or collaborator) in the system. By using a stub, you can test your code without
dealing with the dependency directly.
And defines the difference between stubs and mocks as:
The main thing to remember about mocks versus stubs is that mocks are just like stubs, but you assert against the mock object, whereas you do not assert against a stub.
Fake is just the name used for both stubs and mocks. For example when you don't care about the distinction between stubs and mocks.
The way Osherove's distinguishes between stubs and mocks, means that any class used as a fake for testing can be both a stub or a mock. Which it is for a specific test depends entirely on how you write the checks in your test.
When your test checks values in the class under test, or actually anywhere but the fake, the fake was used as a stub. It just provided values for the class under test to use, either directly through values returned by calls on it or indirectly through causing side effects (in some state) as a result of calls on it.
When your test checks values of the fake, it was used as a mock.
Example of a test where class FakeX is used as a stub:
const pleaseReturn5 = 5;
var fake = new FakeX(pleaseReturn5);
var cut = new ClassUnderTest(fake);
cut.SquareIt;
Assert.AreEqual(25, cut.SomeProperty);
The fake instance is used as a stub because the Assert doesn't use fake at all.
Example of a test where test class X is used as a mock:
const pleaseReturn5 = 5;
var fake = new FakeX(pleaseReturn5);
var cut = new ClassUnderTest(fake);
cut.SquareIt;
Assert.AreEqual(25, fake.SomeProperty);
In this case the Assert checks a value on fake, making that fake a mock.
Now, of course these examples are highly contrived, but I see great merit in this distinction. It makes you aware of how you are testing your stuff and where the dependencies of your test are.
I agree with Osherove's that
from a pure maintainability perspective, in my tests using mocks creates more trouble than not using them. That has been my experience, but I’m always learning something new.
Asserting against the fake is something you really want to avoid as it makes your tests highly dependent upon the implementation of a class that isn't the one under test at all. Which means that the tests for class ActualClassUnderTest can start breaking because the implementation for ClassUsedAsMock changed. And that sends up a foul smell to me. Tests for ActualClassUnderTest should preferably only break when ActualClassUnderTest is changed.
I realize that writing asserts against the fake is a common practice, especially when you are a mockist type of TDD subscriber. I guess I am firmly with Martin Fowler in the classicist camp (See Martin Fowler's "Mocks aren't Stubs") and like Osherove avoid interaction testing (which can only be done by asserting against the fake) as much as possible.
For fun reading on why you should avoid mocks as defined here, google for "fowler mockist classicist". You'll find a plethora of opinions.
As mentioned by the top-voted answer, Martin Fowler discusses these distinctions in Mocks Aren't Stubs, and in particular the subheading The Difference Between Mocks and Stubs, so make sure to read that article.
Rather than focusing on how these things are different, I think it's more enlightening to focus on why these are distinct concepts. Each exists for a different purpose.
Fakes
A fake is an implementation that behaves "naturally", but is not "real". These are fuzzy concepts and so different people have different understandings of what makes things a fake.
One example of a fake is an in-memory database (e.g. using sqlite with the :memory: store). You would never use this for production (since the data is not persisted), but it's perfectly adequate as a database to use in a testing environment. It's also much more lightweight than a "real" database.
As another example, perhaps you use some kind of object store (e.g. Amazon S3) in production, but in a test you can simply save objects to files on disk; then your "save to disk" implementation would be a fake. (Or you could even fake the "save to disk" operation by using an in-memory filesystem instead.)
As a third example, imagine an object that provides a cache API; an object that implements the correct interface but that simply performs no caching at all but always returns a cache miss would be a kind of fake.
The purpose of a fake is not to affect the behavior of the system under test, but rather to simplify the implementation of the test (by removing unnecessary or heavyweight dependencies).
Stubs
A stub is an implementation that behaves "unnaturally". It is preconfigured (usually by the test set-up) to respond to specific inputs with specific outputs.
The purpose of a stub is to get your system under test into a specific state. For example, if you are writing a test for some code that interacts with a REST API, you could stub out the REST API with an API that always returns a canned response, or that responds to an API request with a specific error. This way you could write tests that make assertions about how the system reacts to these states; for example, testing the response your users get if the API returns a 404 error.
A stub is usually implemented to only respond to the exact interactions you've told it to respond to. But the key feature that makes something a stub is its purpose: a stub is all about setting up your test case.
Mocks
A mock is similar to a stub, but with verification added in. The purpose of a mock is to make assertions about how your system under test interacted with the dependency.
For example, if you are writing a test for a system that uploads files to a website, you could build a mock that accepts a file and that you can use to assert that the uploaded file was correct. Or, on a smaller scale, it's common to use a mock of an object to verify that the system under test calls specific methods of the mocked object.
Mocks are tied to interaction testing, which is a specific testing methodology. People who prefer to test system state rather than system interactions will use mocks sparingly if at all.
Test doubles
Fakes, stubs, and mocks all belong to the category of test doubles. A test double is any object or system you use in a test instead of something else. Most automated software testing involves the use of test doubles of some kind or another. Some other kinds of test doubles include dummy values, spies, and I/O blackholes.
The thing that you assert on it is called a mock object.
Everything else that just helped the test run is a stub.
To illustrate the usage of stubs and mocks, I would like to also include an example based on Roy Osherove's "The Art of Unit Testing".
Imagine, we have a LogAnalyzer application which has the sole functionality of printing logs. It not only needs to talk to a web service, but if the web service throws an error, LogAnalyzer has to log the error to a different external dependency, sending it by email to the web service administrator.
Here’s the logic we’d like to test inside LogAnalyzer:
if(fileName.Length<8)
{
try
{
service.LogError("Filename too short:" + fileName);
}
catch (Exception e)
{
email.SendEmail("a","subject",e.Message);
}
}
How do you test that LogAnalyzer calls the email service correctly when the web service throws an exception?
Here are the questions we’re faced with:
How can we replace the web service?
How can we simulate an exception from the web service so that we can
test the call to the email service?
How will we know that the email service was called correctly or at
all?
We can deal with the first two questions by using a stub for the web service. To solve the third problem, we can use a mock object for the email service.
A fake is a generic term that can be used to describe either a stub or a mock.In our test, we’ll have two fakes. One will be the email service mock, which we’ll use to verify that the correct parameters were sent to the email service. The other will be a stub that we’ll use to simulate an exception thrown from the web service. It’s a stub because we won’t be using the web service fake to verify the test result, only to make sure the test runs correctly. The email service is a mock because we’ll assert against it that it was called correctly.
[TestFixture]
public class LogAnalyzer2Tests
{
[Test]
public void Analyze_WebServiceThrows_SendsEmail()
{
StubService stubService = new StubService();
stubService.ToThrow= new Exception("fake exception");
MockEmailService mockEmail = new MockEmailService();
LogAnalyzer2 log = new LogAnalyzer2();
log.Service = stubService
log.Email=mockEmail;
string tooShortFileName="abc.ext";
log.Analyze(tooShortFileName);
Assert.AreEqual("a",mockEmail.To); //MOCKING USED
Assert.AreEqual("fake exception",mockEmail.Body); //MOCKING USED
Assert.AreEqual("subject",mockEmail.Subject);
}
}
Unit testing - is an approach of testing where the unit(class, method) is under control.
Test double - is not a primary object(from OOP world). It is a realisation which is created temporary to test, check or during development. And they are created for closing dependencies of tested unit(method, class...)
Test doubles types:
fake object is a real implementation of interface(protocol) or an extend which is using an inheritance or other approaches which can be used to create - is dependency. Usually it is created by developer as a simplest solution to substitute some dependency
stub object is a bare object(0, nil and methods without logic) with extra state which is predefined(by developer) to define returned values. Usually it is created by framework
class StubA: A {
override func foo() -> String {
return "My Stub"
}
}
mock object is very similar to stub object but the extra state is changed during program execution to check if something happened(method was called, arguments, when, how often...).
class MockA: A {
var isFooCalled = false
override func foo() -> String {
isFooCalled = true
return "My Mock"
}
}
spy object is a real object with a "partial mocking". It means that you work with a non-double object except mocked behavior
dummy object is object which is necessary to run a test but no one variable or method of this object is not called.
stub vs mock
Martin Fowler said
There is a difference in that the stub uses state verification while the mock uses behavior verification.
[Mockito mock vs spy]
All of them are called Test Doubles and used to inject the dependencies that your test case needs.
Stub:
It already has a predefined behavior to set your expectation
for example, stub returns only the success case of your API response
A mock is a smarter stub. You verify your test passes through it.
so you could make amock that return either the success or failure success depending on the condition could be changed in your test case.
If you are familiar with Arrange-Act-Assert, then one way of explaining the difference between stub and mock that might be useful for you, is that stubs belong to the arrange section as they are for arranging input state, and mocks belong to the assert section as they are for asserting results against.
Dummies don't do anything. They are just for filling up parameter lists, so that you don't get undefined or null errors. They also exist to satisfy the type checker in statically typed languages, so that you can be allowed to compile and run.
Stub, Fakes and Mocks have different meanings across different sources. I suggest you to introduce your team internal terms and agree upon their meaning.
I think it is important to distinguish between two approaches:
- behaviour validation (implies behaviour substitution)
- end-state validation (implies behaviour emulation)
Consider email sending in case of error. When doing behaviour validation - you check that method Send of IEmailSender was executed once. And you need to emulate return result of this method, return Id of the sent message. So you say: "I expect that Send will be called. And I will just return dummy (or random) Id for any call". This is behaviour validation:
emailSender.Expect(es=>es.Send(anyThing)).Return((subject,body) => "dummyId")
When doing state validation you will need to create TestEmailSender that implements IEmailSender. And implement Send method - by saving input to some data structure that will be used for future state verification like array of some objects SentEmails and then it tests you will check that SentEmails contains expected email. This is state validation:
Assert.AreEqual(1, emailSender.SentEmails.Count)
From my readings I understood that Behaviour validation usually called Mocks.
And State validation usually called Stubs or Fakes.
It's a matter of making the tests expressive. I set expectations on a Mock if I want the test to describe a relationship between two objects. I stub return values if I'm setting up a supporting object to get me to the interesting behaviour in the test.
stub and fake are objects in that they can vary their response based on input parameters. the main difference between them is that a Fake is closer to a real-world implementation than a stub. Stubs contain basically hard-coded responses to an expected request. Let see an example:
public class MyUnitTest {
#Test
public void testConcatenate() {
StubDependency stubDependency = new StubDependency();
int result = stubDependency.toNumber("one", "two");
assertEquals("onetwo", result);
}
}
public class StubDependency() {
public int toNumber(string param) {
if (param == “one”) {
return 1;
}
if (param == “two”) {
return 2;
}
}
}
A mock is a step up from fakes and stubs. Mocks provide the same functionality as stubs but are more complex. They can have rules defined for them that dictate in what order methods on their API must be called. Most mocks can track how many times a method was called and can react based on that information. Mocks generally know the context of each call and can react differently in different situations. Because of this, mocks require some knowledge of the class they are mocking. a stub generally cannot track how many times a method was called or in what order a sequence of methods was called. A mock looks like:
public class MockADependency {
private int ShouldCallTwice;
private boolean ShouldCallAtEnd;
private boolean ShouldCallFirst;
public int StringToInteger(String s) {
if (s == "abc") {
return 1;
}
if (s == "xyz") {
return 2;
}
return 0;
}
public void ShouldCallFirst() {
if ((ShouldCallTwice > 0) || ShouldCallAtEnd)
throw new AssertionException("ShouldCallFirst not first thod called");
ShouldCallFirst = true;
}
public int ShouldCallTwice(string s) {
if (!ShouldCallFirst)
throw new AssertionException("ShouldCallTwice called before ShouldCallFirst");
if (ShouldCallAtEnd)
throw new AssertionException("ShouldCallTwice called after ShouldCallAtEnd");
if (ShouldCallTwice >= 2)
throw new AssertionException("ShouldCallTwice called more than twice");
ShouldCallTwice++;
return StringToInteger(s);
}
public void ShouldCallAtEnd() {
if (!ShouldCallFirst)
throw new AssertionException("ShouldCallAtEnd called before ShouldCallFirst");
if (ShouldCallTwice != 2) throw new AssertionException("ShouldCallTwice not called twice");
ShouldCallAtEnd = true;
}
}
According to the book "Unit Testing Principles, Practices, and Patterns by Vladimir Khorikov" :
Mocks: help to emulate and examine outcoming interactions. These interactions are calls the SUT makes to its dependencies to change their state. In other words it helps to examine the interaction (behaviour) of SUT and its dependencies. mocks could be :
Spy : created manually
Mocks : created using framework
Stubs: helps to emulate incoming interactions. These interactions are calls the SUT makes to its dependencies to get input data. IN other words it helps to test the data passed to SUT. It could be 3 types
Fake: is usually implemented to replace a dependency that doesn’t yet exist.
Dummy: is hard-coded value.
Stubs: Fledged dependency that you configure to return different values for different scenarios.
In xUnit Test Patterns book by Gerard Meszaros There is a nice table that gives a good insight about differences
I tend to use just 2 terms - Fake and Mock.
Mock only when using a mocking framework like Moq for example because it doesn't seem right to refer to it as a Fake when it's being created with new Mock<ISomething>() - while you can technically use a mocking framework to create Stubs or Fakes, it just seems kind of dumb to call it that in this situation - it has to be a Mock.
Fake for everything else. If a Fake can be summarised as an implementation with reduced capabilities, then I think a Stub could also be a Fake (and if not, who cares, everyone knows what I mean, and not once has anyone ever said "I think you'll find that's a Stub")