I have for long worked on EasyMocking in JUnits. I am pretty much comfortable with this but now I want to know how EasyMocks are different from Jmockits. I tried going through their documentation and I found out that the syntax is a bit different. But yet I could not figure out if there is any difference in their performances.
Can anyone help me figure out what are the points that make either of them better than the other? Is there any special element in JMockit that is not found in the other?
Thanks in advance...
There are many differences between JMockit and EasyMock/jMock/Mockito/PowerMock.
These are the major ones:
Support for integration testing: JMockit supports an out-of-container integration testing approach, similar to what the Spring Test module provides, but also supporting Java EE. The other mocking libraries only support isolated unit testing with mock objects.
A "faking" API (see also in xUnit Patterns), in addition to the mocking API. Each one of the other mocking libraries only have a mocking API.
Full support for "mocking", in addition to "mock objects". Other mocking libraries work with mock objects which they create and which need to be passed in somehow to the code under test. With EasyMock/jMock/Mockito, static methods, constructors, and "new-ed" objects cannot be mocked at all. PowerMock supports full mocking as well, but still focused on mock objects (specifically, new-ed objects must be "replaced" with mock objects through whenNew recording, while with JMockit a test can simply declare a #Mocked field).
Support for mocking/faking of final classes and methods. Only PowerMock also provides this support. Mockito recently added an "inline mock maker" which adds support for finals, but it's not active by default and may not be as reliable.
Support for mocking/faking of unspecificied subclasses and interface implementations (where the test only declares a base type to be mocked/faked).
In the mocking API, expectations on methods with multiple parameters can be recorded/verified with argument matchers (such as anyString, etc.) only for some parameters, while other mocking APIs require such matchers for every single parameter.
Also in the mocking API, expectations can be explicitly verified after the tested code was exercised, just like in Mockito. EasyMock/jMock do not support this verification model.
As for performance, mocking a type with JMockit (done through class redefinition) probably has a higher runtime overhead when compared to creating a mock object with EasyMock/jMock/Mockito (done through subclass definition), and lower when compared with PowerMock (done through class definition on a custom classloader). However, you should only notice the difference in performance if there is a lot of mocking being done, which most likely indicates overuse of mocking.
You can find a blog post about the differences between them (and also mockito!) here: http://www.baeldung.com/mockito-vs-easymock-vs-jmockit
While they all have different syntax and different ways of working, you should be able to achieve whatever you need in regards to mocking with either framework.
Related
I want to know what Mockito is.
Is it supporting JUnit or is it an environment for writing JUnit test cases?
Can someone please explain to me the differences between JUnit and Mockito?
JUnit is a framework that helps with writing and running your unit tests.
Mockito (or any other mocking tool) is a framework that you specifically use to efficiently write certain kind of tests. At its core, any mocking framework allows you to omit instantiating "real" objects of production classes, instead the mocking framework creates a stub for you. Doing so gives you full control over that mocked object and enables you to verify the interactions taking place, for example.
Having said that, one core aspect in unit testing is the fact that you want to isolate your "class under test" from anything else in the world. In order to do that, you very often have to create "test doubles" that you provide to an object of your "class under test". You could create all those "test doubles" manually; or you use a mocking framework that generates object of a certain class for you using reflection techniques. Interestingly enough, some people advocate to never use mocking frameworks, but honestly I can't imagine doing that.
In other words, you can definitely use JUnit without using a mocking framework. Same is true for the reverse direction, but in reality, there are not many good reasons why you would want to use Mockito for anything else but unit testing.
JUnit is the Java library used to write tests (offers support for running tests and different extra helpers - like setup and teardown methods, test sets etc.).
Mockito is a library that enables writing tests using the mocking approach.
See here a nice article on the mocking vs non mocking tests: http://martinfowler.com/articles/mocksArentStubs.html
Layman's Explanation with Example
JUnit is used to test APIs in source code. To test APIs in JUnit, sometimes we may require data classes. To create those, we can use mockito.
Mock refers to creating duplicate or dummy objects.
We might be using many data classes in the project. For example, think of a Student object. It might have some 10 parameters like id, age, marks, gender etc. But you have one API to return a list of students whose age is greater than 10. So, to test this API we need to create student objects with all 10 parameters right. It is difficult. So, we can mock objects only with the required parameters. In our scenario, we will mock student objects with only age as a parameter.
Student student1=mock(Student.class);
when(student1.getAge()).thenReturn(20);
Student student2=mock(Student.class);
when(student2.getAge()).thenReturn(8);
So,above 2 objects can be added to a list and can be passed to API to test if API returns students who has age greater than 10.
I'm curious as to what method people like to use for mocking and why. The two methods that I know of are using hard coded mock objects and a mocking framework. To demonstrate, I'll outline an example using C#.
Suppose we have an IEmployeeRepository interface with a method called GetEmployeeById.
public interface IEmployeeRepository
{
Employee GetEmployeeById(long id);
}
We can easily create a mock of this:
public class MockEmployeeRepository : IEmployeeRepository
{
public Employee GetEmployeeById(long id)
{
Employee employee = new Employee();
employee.FirstName = "First";
employee.LastName = "Last";
...
return employee;
}
}
Then, in our tests we can explicitly tell our services to use the MockEmployeeRepository, either using a setter or dependency injection. I'm new to mocking frameworks so I'm curious as to why we use them, if we can just do the above?
That's not a Mock, it's a Stub. For stubbing, your example is perfectly acceptable.
From Martin Fowler:
Mocks are what we are talking about here: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.
When you're mocking something, you usually call a "Verify" method.
Look at this for the diff between Mocks and Stubs
http://martinfowler.com/articles/mocksArentStubs.html
I think the choice between writing dummy objects by hand or by using a framework depends a lot upon the types of components that you are testing.
If it is part of the contract for the component under test to communicate with its collaborators following a precise protocol, then instrumented dummy objects ("Mocks") are just the thing to use. It is frequently much easier to test such protocols using a mocking framework than by hand-coding. Consider a component that is required to open a repository, perform some reads and writes in a prescribed order, and then close the repository -- even in the face of an exception. A mocking framework would make it easier to set up all of the necessary tests. Applications related to telecommunications and process control (to pick a couple of random examples) are full of components that need to be tested in this fashion.
On the other hand, many components in general business applications have no particular constraints on how they communicate with their collaborators. Consider a component that performs some kind of analysis of, say, university course loads. The component needs to retrieve instructor, student and course information from a repository. But it does not matter what order it retrieves the data: instructor-student-course, student-course-instructor, all-at-once, or whatever. There is no need to test for and enforce a data access pattern. Indeed, it would likely be harmful to test that pattern as it would be demanding a particular implementation unnecessarily. In this context, simple uninstrumented dummy objects ("Stubs") are adequate and a mocking framework is probably overkill.
I should point out that even when stubbing, a framework can still make your life a lot easier. One doesn't always have the luxury of dictating the signatures of one's collaborators. Imagine unit-testing a component that is required to process data retrieved from a thick interface like IDataReader or ResultSet. Hand-stubbing such interfaces is unpleasant at best -- especially if the component under test only actually uses three of the umpteen methods in the interface.
For me, the projects that have required mocking frameworks were almost invariably of a systems-programming nature (e.g. database or web infrastructure projects, or low-level plumbing in a business application). For applications-programming projects, my experience has been that there were few mocks in sight.
Given that we always strive to hide the messy low-level infrastructure details as much as possible, it would seem that we should aim to have the simple stubs far outnumber the mocks.
Some distinguish between mocks and stubs. A mock object may verify that it has been interacted with in the expected way. A mocking framework can make it easy to generate mocks and stubs.
In your example, you've stubbed out a single method in an interface. Consider an interface with n methods, where n can change over time. A hand-stubbed implementation may require more code and more maintenance.
A mocked interface can have different outputs per test - One test you may have a method return null, another test has the method return an object, another test has the method throw an exception. This is all configured in the unit test, whereas your version would require several hand-written objects.
Psuedocode:
//Unit Test One
MockObject.Expect(m => m.GetData()).Return(null);
//Unit Test Two
MockObject.Expect(m => m.GetData()).Return(new MyClass());
//Unit Test Three
MockObject.Expect(m => m.GetData()).ThrowException();
I tend to write stubs and mocks by hand, first. Then if it can be easily expressed using a mock object framework, I rewrite it so that I have less code to maintain.
I have been writing them by hand. I was having trouble using Moq, but then I read TDD: Introduction to Moq, and I think I get what they say about classical vs. mockist approaches now. I'll be giving Moq another try this evening, and I think understanding the "mockist" approach will give me what I need to make Moq work better for me.
I have a project I am trying to learn unit testing and TDD practices with. I'm finding that I'm getting to quite confusing cases where I am spending a long time setting up mocks for a utility class that's used practically everywhere.
From what I've read about unit testing, if I am testing MyClass, I should be mocking any other functionality (such as provided by UtilityClass). Is it acceptable (assuming that UtilityClass itself has a comprehensive set of tests) to just use the UtilityClass rather than setting up mocks for all the different test cases?
Edit: One of the things I am making a lot of setup for.
I am modelling a map, with different objects in different locations. One of the common methods on my utility class is GetDistanceBetween. I am testing methods that have effects on things depending on their individual properties, so for example a test that selects all objects within 5 units of a point and an age over 3 will need several tests (gets old objects in range, ignores old objects out of range, ignores young objects in range, works correctly with multiples of each case) and all of those tests need setup of the GetDistanceBetween method. Multiply that out by every method that uses GetDistanceBetween (almost every one) and the different results that the method should return in different circumstances, and it gets to be a lot of setup.
I can see as I develop this further, there may be more utility class calls, large numbers of objects and a lot of setup on those mock utility classes.
The rule is not "mock everything" but "make tests simple". Mocking should be used if
You can't create an instance with reasonable effort (read: you need a single method call but to create the instance, you need a working database, a DB connection, and five other classes).
Creation of the additional classes is expensive.
The additional classes return unstable values (like the current time or primary keys from a database)
TDD isn't really about testing. Its main benefit is to help you design clean, easy-to-use code that other people can understand and change. If its main benefit was to test then you would be able to write tests after your code, rather than before, with much of the same effect.
If you can, I recommend you stop thinking of them as "unit tests". Instead, think of your tests as examples of how you can use your code, together with descriptions of its behaviour which show why your code is valuable.
As part of that behaviour, your class may want to use some collaborating classes. You can mock these out.
If your utility classes are a core part of your class's behaviour, and your class has no value or its behaviour makes no sense without them, then don't mock them out.
Aaron Digulla's answer is pretty good; I'd rephrase each of his answers according to these principles as:
The behaviour of the collaborating class is complex and independent of the behaviour of the class you're interested in.
Creation of the collaborating class is not a valuable aspect of your class and does not need to be part of your class's responsibility.
The collaborating class provides context which changes the behaviour of your class, and therefore plays into the examples of how you can use it and what kind of behaviour you might expect.
Hope that makes sense! If you liked it, take a look at BDD which uses this kind of vocabulary far more than "test".
In theory you should try to mock all dependencies, but in reality it's never possible. E.g. you are not going to mock the basic classes from the standard library. In your case if the utility class just contains some basic helper methods I think I wouldn't bother to mock it.
If it's more complicated than that or connects to some external resources, you have to mock it. You could consider creating a dedicated mock builder class, that would create you a standard mock (with some standard stubs defined etc), so that you can avoid mocking code duplication in all test classes.
No, it is not acceptable because you are no longer testing the class in isolation which is one of the most important aspects of a unit test. You are testing it with its dependency to this utility even if the utility has its own set of tests. To simplify the creation of mock objects you could use a mock framework. Here are some popular choices:
Rhino Mocks
Moq
NSubstitute
Of course if this utility class is private and can only be used within the scope of the class under test then you don't need to mock it.
Yes, it is acceptable. What's important is to have the UtilityClass thoroughly unit tested and to be able to differentiate if a test is failing because of the Class under test or because of the UtilityClass.
Testing a class in isolation means testing it in a controlled environment, in an environment where one control how the objects behave.
Having to create too many objects in a test setup is a sign that the environment is getting too large and thus is not controlled enough. Time has come to revert to mock objects.
All the previous answers are very good and really match with my point of view about static utility classes and mocking.
You have two types of utilities classes, your own classes you write and the third party utility classes.
As the purpose of an utility class is to provide small set of helper methods, your utility classes or a third party utility classes should be very well tested.
First Case: the first condition to use your own utility class (even if static) without mocking, is to provide a set of valid unit tests for this class.
Second Case: if you use a third party utility library, you should have enough confidence to this library. Most of the time, those libraries are well tested and well maintained. You can use it without mocking its methods.
I just watched this funny YouTube Video about unit testing (it's Hitler with fake subtitles chewing out his team for not doing good unit tests--skip it if you're humor impaired) where stubs get roundly criticized. But I don't understand what wrong with stubs.
I haven't started using a mocking framework and I haven't started feeling the pain from not using one.
Am I in for a world a hurt sometime down the line, having chosen handwritten stubs and fakes instead of mocks (like Rhinomock etc)? (using Fowler's taxonomy)
What are the considerations for picking between a mock and handwritten stub?
There is nothing wrong with stubs, there is room for stubs, mocks... and spies. All are "test doubles", but with different purposes as explained in Mocks and Stubs aren't Spies:
[...] Before moving on, I'd like to
clarify and define some terms in use
here, which I originally discovered in
Gerard Meszaros' xUnit Patterns
book.
A Dummy Object is a placeholder object passed to the system under test
but never used.
A Test Stub provides the system under test with indirect input
A Test Spy provides a way to verify that the system under test performed
the correct indirect output
A Mock Object provides the system under test with both indirect input
and a way to verify indirect output
[...] And you can let this handy chart
guide your decisions:
PS: Mockito - The New Mock Framework on the Block is worth the read too.
I use the following terminology (introduced by Roy Osherove, the author of the Art of Unit-Testing):
A fake is called a stub if you tell it to fake something in case a method is called with such and such parameters. But if you also verify that such call actually took place or took place exactly N times, then such fake is called a mock. In short. a fake is a stub unless you call Verify() on it and then it's a mock.
Obviously, you will need to use stubs in some cases and mocks in others. So, criticizing stubs roundly is probably wrong and using stubs exclusively is probably wrong as well.
If you haven't started using a mocking framework (alternative term: isolation framework), you should keep an eye on them and reevaluate your options frequently. I went from manual mocks to NMock2 to Moq very quickly. Here's an interesting poll of programmers that shows what they use. Manual mocks/stubs are in the minority, but aren't that uncommon.
Mocks are just a lot easier to throw in. They are a real instance of your class, pre-stubbed with the ability to override the action of any method with minimal boilerplate.
There are lots of little considerations such as: If you don't want to deal with any of the methods, you can either have it act as a no-op or fail the test--your choice--but either way virtually no code.
How much boilerplate do you get when you stub out a class? How do you handle it if your class is final? Do you play tricks to get your stub on the classpath first, or do you use different source?
I recommend just starting with the mocks--It's easier all around.
There is nothing wrong with using stubs instead of mocks.
If you want to get technical, mocks are "smart" objects with expectations that can be verified. Stubs are dummy objects that return preset values. See Mocks Aren't Stubs.
But many people (myself included) prefer to do state testing with stubs rather than behavior testing with mocks. You inject a stub into the class under test, you call a method, then you check the state of the class under test. It tends to make for less brittle tests than asserting that the class's internals called method X of a mock object with argument Y.
I don't think you're in for a world of hurt. If you haven't started feeling the pain, you probably don't need an isolation/mocking framework yet. When and if you do, having handwritten stubs/fakes around isn't going to hurt anything.
If you have a lot of interfaces, or if your interfaces have a lot of methods, an isolation/mocking framework can save a lot of time over hand-coding stubs.
I like Moq a lot; I find it easier to use than Rhino Mocks for creating stubs.
Mocks and stubs is used to achieve a real unit test. You just mock all the dependencies, and unit test your class in isolation.
I'm currently using MOQ for mocking and stubbing.
I recently discussed with a colleague about mocking. He said that mocking classes is very bad and should not be done, only in few cases.
He says that only interfaces should be mocked, otherwise it's an architecture fault.
I wonder why this statement (I fully trust him) is so correct? I don't know it and would like to be convinced.
Did I miss the point of mocking (yes, I read Martin Fowler's article)
Mocking is used for protocol testing - it tests how you'll use an API, and how you'll react when the API reacts accordingly.
Ideally (in many cases at least), that API should be specified as an interface rather than a class - an interface defines a protocol, a class defines at least part of an implementation.
On a practical note, mocking frameworks tend to have limitations around mocking classes.
In my experience, mocking is somewhat overused - often you're not really interested in the exact interaction, you really want a stub... but mocking framework can be used to create stubs, and you fall into the trap of creating brittle tests by mocking instead of stubbing. It's a hard balance to get right though.
IMHO, what your colleague means is that you should program to an interface, not an implementation. If you find yourself mocking classes too often, it's a sign you broke the previous principle when designing your architecture.
Mocking classes (in contrast to mocking interfaces) is bad because the mock still has a real class in the background, it is inherited from, and it is possible that real implementation is executed during the test.
When you mock (or stub or whatever) an interface, there is no risk of having code executed you actually wanted to mock.
Mocking classes also forces you to make everything, that could possibly be mocked, to be virtual, which is very intrusive and could lead to bad class design.
If you want to decouple classes, they should not know each other, this is the reason why it makes sense to mock (or stub or whatever) one of them. So implementing against interfaces is recommended anyway, but this is mentioned here by others enough.
I would suggest to stay away from mocking frameworks as far as possible. At the same time, I would recommend to use mock/fake objects for testing, as much as possible. The trick here is that you should create built-in fake objects together with real objects. I explain it more in detail in a blog post I wrote about it: http://www.yegor256.com/2014/09/23/built-in-fake-objects.html
Generally you'd want to mock an interface.
While it is possible to mock a regular class, it tends to influence your class design too much for testability. Concerns like accessibility, whether or not a method is virtual, etc. will all be determined by the ability to mock the class, rather than true OO concerns.
There is one faking library called TypeMock Isolator that allows you to get around these limitations (have cake, eat cake) but it's pretty expensive. Better to design for testability.
The answer, like most questions about practices, is "it depends".
Overuse of mocks can lead to tests that don't really test anything. It can also lead to tests which are virtual re-implementations of the code under test, tightly bound to a specific implementation.
On the other hand, judicious use of mocks and stubs can lead to unit tests which are neatly isolated and test one thing and one thing alone - which is a good thing.
It's all about moderation.
It makes sense to mock classes so tests can be written early in the development lifecycle.
There is a tendency to continue to use mock classes even when concrete implementations become available. There is also the tendency to develop against mock classes (and stubs) necessary early in a project when some parts of the system have not been built.
Once a piece of the system has been built it is necessary to test against it and continue to test against it (for regression). In this case starting with mocks is good but they should be discarded in favour of the implementation as soon as possible. I have seen projects struggle because different teams continue to develop against the behaviour of the mock rather than the implementation (once it is available).
By testing against mocks you are assuming that the mock is characteristic of the system. Often this involves guessing what the mocked component will do. If you have a specification of the system you are mocking then you don't have to guess, but often the 'as-built' system doesn't match the original specification due to practical considerations discovered during construction. Agile development projects assume this will always happen.
You then develop code that works with the mock. When it turns out that the mock does not truly represent the behaviour of the real as-built system (eg. latency issues not seen in the mock, resource and efficiency issues not seen in the mock, concurrency issues, performance issues etc) you then have a bunch of worthless mocking tests you must now maintain.
I consider the use of mocks to be valuable at the start of development but these mocks should not contribute to project coverage. It is best later if the mocks are removed and proper integration tests are created to replace them otherwise your system will not be getting tested for the variety of conditions which your mock did not simulate (or simulates incorrectly relative to the real system).
So, the question is whether or not to use mocks, it is a matter of when to use them and when to remove them.
It depends how often you use (or are forced by bad design) mocks.
If instantiating the object becomes too hard (and it happens more than often), then it is a sign the code may need some serious refactoring or change in design (builder? factory?).
When you mock everything you end up with tests that know everything about your implementation (white box testing). Your tests no longer document how to use the system - they are basically a mirror of its implementation.
And then comes potential code refactoring..
From my experience it's one of the biggest issues related to overmocking. It becomes painful and takes time, lots of it.
Some developers become fearful of refactoring their code knowing how long will it take.
There is also question of purpose - if everything is mocked, are we really testing the production code?
Mocks of course tend to violate DRY principle by duplicating code in two places: once in the production code and once in the tests.
Therefore, as I mentioned before, any change to code has to be made in two places (if tests aren't written well, it can be in more than that..).
Edit: Since you have clarified that your colleague meant mock class is bad but mock interface is not, the answer below is outdated. You should refer to this answer.
I am talking about mock and stub as defined by Martin Fowler, and I assume that's what your colleague meant, too.
Mocking is bad because it can lead to overspecification of tests. Use stub if possible and avoid mock.
Here's the diff between mock and stub (from the above article):
We can then use state verification on
the stub like this.
class OrderStateTester...
public void testOrderSendsMailIfUnfilled() {
Order order = new Order(TALISKER, 51);
MailServiceStub mailer = new MailServiceStub();
order.setMailer(mailer);
order.fill(warehouse);
assertEquals(1, mailer.numberSent());
}
Of course this is a very simple test -
only that a message has been sent.
We've not tested it was send to the
right person, or with the right
contents, but it will do to illustrate
the point.
Using mocks this test would look quite
different.
class OrderInteractionTester...
public void testOrderSendsMailIfUnfilled() {
Order order = new Order(TALISKER, 51);
Mock warehouse = mock(Warehouse.class);
Mock mailer = mock(MailService.class);
order.setMailer((MailService) mailer.proxy());
mailer.expects(once()).method("send");
warehouse.expects(once()).method("hasInventory")
.withAnyArguments()
.will(returnValue(false));
order.fill((Warehouse) warehouse.proxy());
}
}
In order to use state verification on the stub, I need to make some extra methods on the >stub to help with verification. As a result the stub implements MailService but adds extra >test methods.