I'm using more unit tests in my projects and reading all the information I can online and am getting confused by a lot of the terminology. As a result I'm probably using these terms incorrectly in conversation and google searches.
Can somebody outline all the unit testing terms like the "fake" types as well as types of test ( interaction vs. integration )?
When it comes to mocks vs. fakes vs. stubs, there are actually a few different ways that people interpret them. I usually borrow the meanings defined by Martin Fowler:
Stub objects provide a valid response, but it's static -- no matter what input you pass in, you'll always get the same response.
Fake objects act like the real object, but they go about it in a simpler way -- such as a DAO that uses a Map to store data instead of a real database.
Mock objects are used in mock test cases -- they validate that certain methods are called on those objects.
Interaction testing is a general term that refers to unit tests which ensure that the interaction between objects is correct (making sure that expected methods are called). This is opposed to state (or classical) testing, which doesn't care what happens in the methods, so long as the resulting state is correct. These types of testing are compared in Fowler's article that I linked above.
Integration testing really isn't an aspect of unit testing, it is a level above the unit tests. It takes different units and verifies that they work together correctly.
This article from MSN Magazine explains the terms and goes into some detail with examples and some source code.
Basically these are the test doubles:
Dummy - Dummies contain no implementation
Stub - stubs are minimal implementations of interfaces or base classes
Spy - a spy will record which members were invoked
Fake - more complex, a fake may resemble a production implementation
Mock - A mock is usually dynamically created by a mock library and depending on its configuration, a mock can behave like a dummy, a stub, or a spy
Fowler did of course a great work at differentiating Mocks and Stubs but, to me, the XUnit Test Patterns book is the reference and I'd suggest to check Mocks, Fakes, Stubs and Dummies for a comprehensive comparison.
Yeah, I know, this is confusing and that's why I'd suggest checking Mocks and Stubs aren't Spies, then let’s spy and finally Mockito - The New Mock Framework on the Block too.
Regarding the different types of tests, a simplified explanation could be (this is not an exhaustive list):
unit testing: testing a single "unit", a method, in isolation
integration testing: testing the integration of several units (methods, classes, components, layers)
functional testing: testing an end-to-end scenario (from a user perspective)
All these types are useful and not exclusive, they actually just don't have the same intent.
I've read (and heartily recommend) The Art of Unit Testing by Roy Osherove. He uses a simplified set of terminology.
To paraphrase his book ...
Integration Test - any test that reaches outside of the current process or object to interact with something else
Interaction Test - a test on the way that objects work together
State Test - a test on the results produced by an operation
Fake - any stand-in object that's used instead of the real thing
Stub - a stand-in object that provides a dependency required by the code under test
Mock - a stand-in used to check the results of the test
Note that here both Stubs and Mocks can be provided by a Mocking framework - the distiction is as much about how they're used as the technology used.
this is an excellent book: http://xunitpatterns.com/
see: http://xunitpatterns.com/Mocks,%20Fakes,%20Stubs%20and%20Dummies.html
and http://xunitpatterns.com/XUnit%20Terminology%20Crossreference.html
Related
I want to know what Mockito is.
Is it supporting JUnit or is it an environment for writing JUnit test cases?
Can someone please explain to me the differences between JUnit and Mockito?
JUnit is a framework that helps with writing and running your unit tests.
Mockito (or any other mocking tool) is a framework that you specifically use to efficiently write certain kind of tests. At its core, any mocking framework allows you to omit instantiating "real" objects of production classes, instead the mocking framework creates a stub for you. Doing so gives you full control over that mocked object and enables you to verify the interactions taking place, for example.
Having said that, one core aspect in unit testing is the fact that you want to isolate your "class under test" from anything else in the world. In order to do that, you very often have to create "test doubles" that you provide to an object of your "class under test". You could create all those "test doubles" manually; or you use a mocking framework that generates object of a certain class for you using reflection techniques. Interestingly enough, some people advocate to never use mocking frameworks, but honestly I can't imagine doing that.
In other words, you can definitely use JUnit without using a mocking framework. Same is true for the reverse direction, but in reality, there are not many good reasons why you would want to use Mockito for anything else but unit testing.
JUnit is the Java library used to write tests (offers support for running tests and different extra helpers - like setup and teardown methods, test sets etc.).
Mockito is a library that enables writing tests using the mocking approach.
See here a nice article on the mocking vs non mocking tests: http://martinfowler.com/articles/mocksArentStubs.html
Layman's Explanation with Example
JUnit is used to test APIs in source code. To test APIs in JUnit, sometimes we may require data classes. To create those, we can use mockito.
Mock refers to creating duplicate or dummy objects.
We might be using many data classes in the project. For example, think of a Student object. It might have some 10 parameters like id, age, marks, gender etc. But you have one API to return a list of students whose age is greater than 10. So, to test this API we need to create student objects with all 10 parameters right. It is difficult. So, we can mock objects only with the required parameters. In our scenario, we will mock student objects with only age as a parameter.
Student student1=mock(Student.class);
when(student1.getAge()).thenReturn(20);
Student student2=mock(Student.class);
when(student2.getAge()).thenReturn(8);
So,above 2 objects can be added to a list and can be passed to API to test if API returns students who has age greater than 10.
It is my first time to make a Unit testing so i'm trying to find references to how to make a Unit testing of a DAO. Can you guys make a simple example of the setUpBeforeClass, setUp and how to test a method that inserting new data in database using model for this. Just a simple example using easy mock. Thank you for your consideration
The idea of using mock objects to perform your unit testing strikes me as peculiar as all you are doing is testing your mock objects instead of your real ones. If you think you need to use mock objects to emulate database access then your entire architecture is wrong. I personally build all my software using the 3-Tier Architecture where I can have as many objects as I like in the Business layer, but only 1 object in the Data Access layer. Thus if I wanted to exchange real database access with dummy database access where would I make the change? All 200+ objects in my Business layer, or just the 1 object in the Data Access layer? Why should I then implement a mechanism to change every object within my application when all I really need to do is change one?
Controllers are meant to be integration tested, not unit tested. But the testing pyramid prescribes that the unit level is where the focus should be, so people are sucked into that by default.
Assertions should never fail under any circumstance.
If they fail in your tests, it indicates a logical error.
Basically, if your function is doing "assert( 0 )" instead of returning an error code, then the function should be re-written.
Can you make massive changes to a class definition — even throw out the whole thing and replace it with a completely different implementation — without impacting any of the code that uses that class's objects?
As has unfortunately happened with a variety of patterns that originate from rigid languages like Java, Dependency Injection has spread and been advocated as a cross-language best practice on trumped up benefits of flexibility and malleability.
References
Dependency Injection is Evil - Tony Marston
Test Induced Design Damage - David Heinemeier Hansson
TDDing Getters and Setters Considered Harmful
Why Getter and Setter Methods are Evil
Dependency injection is not a virtue in Ruby (DHH)
TDD the RITE Way – JavaScript Scene – Medium
5 Common Misconceptions About TDD & Unit Tests – JavaScript Scene – Medium
Ned Batchelder: tl;dw: Stop mocking, start testing
ACCU :: Mocks are Bad, Layers are Bad
Google Testing Blog: Testing on the Toilet: Don’t Overuse Mocks
I'm curious as to what method people like to use for mocking and why. The two methods that I know of are using hard coded mock objects and a mocking framework. To demonstrate, I'll outline an example using C#.
Suppose we have an IEmployeeRepository interface with a method called GetEmployeeById.
public interface IEmployeeRepository
{
Employee GetEmployeeById(long id);
}
We can easily create a mock of this:
public class MockEmployeeRepository : IEmployeeRepository
{
public Employee GetEmployeeById(long id)
{
Employee employee = new Employee();
employee.FirstName = "First";
employee.LastName = "Last";
...
return employee;
}
}
Then, in our tests we can explicitly tell our services to use the MockEmployeeRepository, either using a setter or dependency injection. I'm new to mocking frameworks so I'm curious as to why we use them, if we can just do the above?
That's not a Mock, it's a Stub. For stubbing, your example is perfectly acceptable.
From Martin Fowler:
Mocks are what we are talking about here: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.
When you're mocking something, you usually call a "Verify" method.
Look at this for the diff between Mocks and Stubs
http://martinfowler.com/articles/mocksArentStubs.html
I think the choice between writing dummy objects by hand or by using a framework depends a lot upon the types of components that you are testing.
If it is part of the contract for the component under test to communicate with its collaborators following a precise protocol, then instrumented dummy objects ("Mocks") are just the thing to use. It is frequently much easier to test such protocols using a mocking framework than by hand-coding. Consider a component that is required to open a repository, perform some reads and writes in a prescribed order, and then close the repository -- even in the face of an exception. A mocking framework would make it easier to set up all of the necessary tests. Applications related to telecommunications and process control (to pick a couple of random examples) are full of components that need to be tested in this fashion.
On the other hand, many components in general business applications have no particular constraints on how they communicate with their collaborators. Consider a component that performs some kind of analysis of, say, university course loads. The component needs to retrieve instructor, student and course information from a repository. But it does not matter what order it retrieves the data: instructor-student-course, student-course-instructor, all-at-once, or whatever. There is no need to test for and enforce a data access pattern. Indeed, it would likely be harmful to test that pattern as it would be demanding a particular implementation unnecessarily. In this context, simple uninstrumented dummy objects ("Stubs") are adequate and a mocking framework is probably overkill.
I should point out that even when stubbing, a framework can still make your life a lot easier. One doesn't always have the luxury of dictating the signatures of one's collaborators. Imagine unit-testing a component that is required to process data retrieved from a thick interface like IDataReader or ResultSet. Hand-stubbing such interfaces is unpleasant at best -- especially if the component under test only actually uses three of the umpteen methods in the interface.
For me, the projects that have required mocking frameworks were almost invariably of a systems-programming nature (e.g. database or web infrastructure projects, or low-level plumbing in a business application). For applications-programming projects, my experience has been that there were few mocks in sight.
Given that we always strive to hide the messy low-level infrastructure details as much as possible, it would seem that we should aim to have the simple stubs far outnumber the mocks.
Some distinguish between mocks and stubs. A mock object may verify that it has been interacted with in the expected way. A mocking framework can make it easy to generate mocks and stubs.
In your example, you've stubbed out a single method in an interface. Consider an interface with n methods, where n can change over time. A hand-stubbed implementation may require more code and more maintenance.
A mocked interface can have different outputs per test - One test you may have a method return null, another test has the method return an object, another test has the method throw an exception. This is all configured in the unit test, whereas your version would require several hand-written objects.
Psuedocode:
//Unit Test One
MockObject.Expect(m => m.GetData()).Return(null);
//Unit Test Two
MockObject.Expect(m => m.GetData()).Return(new MyClass());
//Unit Test Three
MockObject.Expect(m => m.GetData()).ThrowException();
I tend to write stubs and mocks by hand, first. Then if it can be easily expressed using a mock object framework, I rewrite it so that I have less code to maintain.
I have been writing them by hand. I was having trouble using Moq, but then I read TDD: Introduction to Moq, and I think I get what they say about classical vs. mockist approaches now. I'll be giving Moq another try this evening, and I think understanding the "mockist" approach will give me what I need to make Moq work better for me.
I just watched this funny YouTube Video about unit testing (it's Hitler with fake subtitles chewing out his team for not doing good unit tests--skip it if you're humor impaired) where stubs get roundly criticized. But I don't understand what wrong with stubs.
I haven't started using a mocking framework and I haven't started feeling the pain from not using one.
Am I in for a world a hurt sometime down the line, having chosen handwritten stubs and fakes instead of mocks (like Rhinomock etc)? (using Fowler's taxonomy)
What are the considerations for picking between a mock and handwritten stub?
There is nothing wrong with stubs, there is room for stubs, mocks... and spies. All are "test doubles", but with different purposes as explained in Mocks and Stubs aren't Spies:
[...] Before moving on, I'd like to
clarify and define some terms in use
here, which I originally discovered in
Gerard Meszaros' xUnit Patterns
book.
A Dummy Object is a placeholder object passed to the system under test
but never used.
A Test Stub provides the system under test with indirect input
A Test Spy provides a way to verify that the system under test performed
the correct indirect output
A Mock Object provides the system under test with both indirect input
and a way to verify indirect output
[...] And you can let this handy chart
guide your decisions:
PS: Mockito - The New Mock Framework on the Block is worth the read too.
I use the following terminology (introduced by Roy Osherove, the author of the Art of Unit-Testing):
A fake is called a stub if you tell it to fake something in case a method is called with such and such parameters. But if you also verify that such call actually took place or took place exactly N times, then such fake is called a mock. In short. a fake is a stub unless you call Verify() on it and then it's a mock.
Obviously, you will need to use stubs in some cases and mocks in others. So, criticizing stubs roundly is probably wrong and using stubs exclusively is probably wrong as well.
If you haven't started using a mocking framework (alternative term: isolation framework), you should keep an eye on them and reevaluate your options frequently. I went from manual mocks to NMock2 to Moq very quickly. Here's an interesting poll of programmers that shows what they use. Manual mocks/stubs are in the minority, but aren't that uncommon.
Mocks are just a lot easier to throw in. They are a real instance of your class, pre-stubbed with the ability to override the action of any method with minimal boilerplate.
There are lots of little considerations such as: If you don't want to deal with any of the methods, you can either have it act as a no-op or fail the test--your choice--but either way virtually no code.
How much boilerplate do you get when you stub out a class? How do you handle it if your class is final? Do you play tricks to get your stub on the classpath first, or do you use different source?
I recommend just starting with the mocks--It's easier all around.
There is nothing wrong with using stubs instead of mocks.
If you want to get technical, mocks are "smart" objects with expectations that can be verified. Stubs are dummy objects that return preset values. See Mocks Aren't Stubs.
But many people (myself included) prefer to do state testing with stubs rather than behavior testing with mocks. You inject a stub into the class under test, you call a method, then you check the state of the class under test. It tends to make for less brittle tests than asserting that the class's internals called method X of a mock object with argument Y.
I don't think you're in for a world of hurt. If you haven't started feeling the pain, you probably don't need an isolation/mocking framework yet. When and if you do, having handwritten stubs/fakes around isn't going to hurt anything.
If you have a lot of interfaces, or if your interfaces have a lot of methods, an isolation/mocking framework can save a lot of time over hand-coding stubs.
I like Moq a lot; I find it easier to use than Rhino Mocks for creating stubs.
Mocks and stubs is used to achieve a real unit test. You just mock all the dependencies, and unit test your class in isolation.
I'm currently using MOQ for mocking and stubbing.
I've introduced visitors as one of core architecture ideas in one of my apps. I have several visitors that operate on a same stuff. Now, how should I test it? Some tests I'm thinking of are a bit larger then a unit test should be (integration test? whatever) but I still wanna do it. How would you test code like the C++ sample from wiki art on Visitor Pattern
Unit testing isn't really about testing patterns, it is about testing the correct implementation of methods and functions. The visitor pattern is a specific class structure, and for each of the classes (ConcreteVisitor and ConcreteElement, specifically) involved you'll want unit tests.
When you've developed confidence that your class methods are behaving OK, you could use your unit test framework to develop integration tests as well. Do not start integration testing rightaway: you'll find that you develop a lot of integration tests that are actually testing the behavior of a specific class, i.e. unit tests.
Whether you need mock objects or can use 'real' objects is a different matter. This depends a lot on whether the objects behave nice enough for unit test purposes (i.e. they do not pull in a lot of additional dependencies etc.), and whether the objects themselves are unit tested (i.e. you need to be able to trust these objects 100%). The mock vs. real objects issue has been addressed on stackflow before, so search the unittest tags.
make a test visitor object and make it visit things.... test that it visited the right things.
You can create mock objects and have your visitors visit them, and then create mock visitors, and test that the right actions were performed.