I'm having a simple test design problem, which I would like to solve once and for all.
I'm pretty used to regular Java design pattern where there is some Manager interface (or facade) and a set of DAO interfaces. Concrete implementation of ManagerImpl is using concrete implementations of DaoImpl.
Right now I'm at the point of implementation, where I don't have a database connected to my project yet, so I guess this is a perfect time to write proper unit tests without DB :-)
I was able to mock some methods of my manager by using mockito, but since the Test Under Method (or so called System Under Test) is using DAO internally, I would have to mock the DAO as well. Unfortunately I cannot do that without setting concrete DAO implementation in my manager, like myManager.setMyDao(mockedDao), but now I would have to pull out this setMyDao method into interface, which of course breaks encapsulation and makes my clean and perfect interfaces look like garbage.
So the question is: How to mock DAO in tests while preserving clean Manager-Dao architecture?
You should unit test concrete implementations, i.e. ManagerImplTest, which will explicitly create ManagerImpl and therefore you have all setMyDao methods available for mocks. Your test will not know about Manager interface.
BTW, it is not always necessary to create interfaces for everything. In most cases only single implementation of managers and daos will exist. If there is no any other strong reason (e.g. Spring AOP proxies or dependency separations), I guess it's better don't create interfaces for everything in sake of simplicity.
You can get rid of set methods altogether and inject factory. This way no extra method is exposed (in fact, no extra method even exists) and problem disappears. Then in test you configure factory mock to return DAO mocks and from that point on it's plain and simple.
Related
I'm curious as to what method people like to use for mocking and why. The two methods that I know of are using hard coded mock objects and a mocking framework. To demonstrate, I'll outline an example using C#.
Suppose we have an IEmployeeRepository interface with a method called GetEmployeeById.
public interface IEmployeeRepository
{
Employee GetEmployeeById(long id);
}
We can easily create a mock of this:
public class MockEmployeeRepository : IEmployeeRepository
{
public Employee GetEmployeeById(long id)
{
Employee employee = new Employee();
employee.FirstName = "First";
employee.LastName = "Last";
...
return employee;
}
}
Then, in our tests we can explicitly tell our services to use the MockEmployeeRepository, either using a setter or dependency injection. I'm new to mocking frameworks so I'm curious as to why we use them, if we can just do the above?
That's not a Mock, it's a Stub. For stubbing, your example is perfectly acceptable.
From Martin Fowler:
Mocks are what we are talking about here: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.
When you're mocking something, you usually call a "Verify" method.
Look at this for the diff between Mocks and Stubs
http://martinfowler.com/articles/mocksArentStubs.html
I think the choice between writing dummy objects by hand or by using a framework depends a lot upon the types of components that you are testing.
If it is part of the contract for the component under test to communicate with its collaborators following a precise protocol, then instrumented dummy objects ("Mocks") are just the thing to use. It is frequently much easier to test such protocols using a mocking framework than by hand-coding. Consider a component that is required to open a repository, perform some reads and writes in a prescribed order, and then close the repository -- even in the face of an exception. A mocking framework would make it easier to set up all of the necessary tests. Applications related to telecommunications and process control (to pick a couple of random examples) are full of components that need to be tested in this fashion.
On the other hand, many components in general business applications have no particular constraints on how they communicate with their collaborators. Consider a component that performs some kind of analysis of, say, university course loads. The component needs to retrieve instructor, student and course information from a repository. But it does not matter what order it retrieves the data: instructor-student-course, student-course-instructor, all-at-once, or whatever. There is no need to test for and enforce a data access pattern. Indeed, it would likely be harmful to test that pattern as it would be demanding a particular implementation unnecessarily. In this context, simple uninstrumented dummy objects ("Stubs") are adequate and a mocking framework is probably overkill.
I should point out that even when stubbing, a framework can still make your life a lot easier. One doesn't always have the luxury of dictating the signatures of one's collaborators. Imagine unit-testing a component that is required to process data retrieved from a thick interface like IDataReader or ResultSet. Hand-stubbing such interfaces is unpleasant at best -- especially if the component under test only actually uses three of the umpteen methods in the interface.
For me, the projects that have required mocking frameworks were almost invariably of a systems-programming nature (e.g. database or web infrastructure projects, or low-level plumbing in a business application). For applications-programming projects, my experience has been that there were few mocks in sight.
Given that we always strive to hide the messy low-level infrastructure details as much as possible, it would seem that we should aim to have the simple stubs far outnumber the mocks.
Some distinguish between mocks and stubs. A mock object may verify that it has been interacted with in the expected way. A mocking framework can make it easy to generate mocks and stubs.
In your example, you've stubbed out a single method in an interface. Consider an interface with n methods, where n can change over time. A hand-stubbed implementation may require more code and more maintenance.
A mocked interface can have different outputs per test - One test you may have a method return null, another test has the method return an object, another test has the method throw an exception. This is all configured in the unit test, whereas your version would require several hand-written objects.
Psuedocode:
//Unit Test One
MockObject.Expect(m => m.GetData()).Return(null);
//Unit Test Two
MockObject.Expect(m => m.GetData()).Return(new MyClass());
//Unit Test Three
MockObject.Expect(m => m.GetData()).ThrowException();
I tend to write stubs and mocks by hand, first. Then if it can be easily expressed using a mock object framework, I rewrite it so that I have less code to maintain.
I have been writing them by hand. I was having trouble using Moq, but then I read TDD: Introduction to Moq, and I think I get what they say about classical vs. mockist approaches now. I'll be giving Moq another try this evening, and I think understanding the "mockist" approach will give me what I need to make Moq work better for me.
I'd like to brush my brain to avoid confusions. In few words, what can be said about Mocking process in TDD
What's the GREAT idea behind MOCKING?
Mocking frameworks are meant to be used only to avoid accessing DB during tests or they can be used for something else?
For new comers (like me), are all the frameworks equal or I need to choose one for this or that reason?
In addition to eliminating databases and other slow or ancillary concerns from the unit being tested, mocking allows you to start writing tests for a class without having to implement any collaborating classes.
As you design some piece of functionality, you'll realize that you need some other class or service, in order to stick to the single responsibility principle, but then you'll have to implement those to get the first one working, which in turn will demonstrate the need for still more classes.
If you can mock or stub those dependencies, then you can create the interfaces upon which that first class will rely, without actually having to implement anything outside of that class -- just return canned results from stubs of the interfaces.
This is an essential component to a test-first approach.
The GREAT idea: LIMIT THE SCOPE OF YOUR TESTS. By removing dependencies you remove the risk of test failures because of dependencies. That way you can focus on the correctness of the code that USES those dependencies.
Mocking DB's is very common but you can mock any dependency with an interface. In a recent project we mocked a web service, for example. You might even want to mock another business object just to make sure that you aren't relying on the correctness of the logic in that object.
I'd choose whichever one seems easiest to use. Moq is really nice.
I suggest you start here:
Mocks are not Stubs
It probably is the article that got me thinking the right way about Mocks. Sure the mocked object is usually heavy (otherwise it may not be worth mocking) but it doesn't have to be heavy in the sense that has some strong reliance on an external system like a database. It can be just a complex piece that you need to isolate to effectively be testing only your class and not the dependency.
During a recent interview I was asked why one would want to create mock objects. My answer went something like, "Take a database--if you're writing test code, you may not want that test hooked up live to the production database where actual operations will be performed."
Judging by response, my answer clearly was not what the interviewer was looking for. What's a better answer?
I'd summarize like this:
Isolation - You can test only a method, independently on what it calls. Your test becomes a real unit test (most important IMHO)
Decrease test development time - it is usually faster to use a mock then to create a whole class just for help you test
It lets you test even when you don't have implemented all dependencies - You don't even need to create, for instance, your repository class, and you'll be able to test a class that would use this repository
Keeps you away from external resources - helps in the sense you don't need to access databases, or to call web services, or to read files, or to send emails, or to charge a credit card, and so on...
In an interview, I'd recommend including that mocking is even better when developers use dependency injection, once it allows you to have more control, and build tests more easily.
When unit testing, each test is designed to test a single object. However most objects in a system will have other objects that they interact with. Mock Objects are dummy implementations of these other objects, used to isolate the object under test.
The benefit of this is that any unit tests that fail generally isolate the problem to the object under test. In some cases the problem will be with the mock object, but those problems should be simpler to identify and fix.
It might be an idea to write some simple unit tests for the mock objects as well.
They are commonly used to create a mock data access layer so that unit tests can be run in isolation from the data store.
Other uses might be to mock the user interface when testing the controller object in the MVC pattern. This allows better automated testing of UI components that can somewhat simulate user interaction.
An example:
public interface IPersonDAO
{
Person FindById(int id);
int Count();
}
public class MockPersonDAO : IPersonDAO
{
// public so the set of people can be loaded by the unit test
public Dictionary<int, Person> _DataStore;
public MockPersonDAO()
{
_DataStore = new Dictionary<int, Person>();
}
public Person FindById(int id)
{
return _DataStore[id];
}
public int Count()
{
return _DataStore.Count;
}
}
Just to add on to the fine answers here, mock objects are used in top-down or bottom-up structural programming (OOP too). They are there to provide data to upper-level modules (GUI, logic processing) or to act as out mock output.
Consider top-down approach: you develop a GUI first, but a GUI ought to have data. So you create a mock database which just return a std::vector<> of data. You have defined the 'contract' of the relationship. Who cares what goes on inside the database object - as long as my GUI list get a std::vector<> I'm happy. This can go to provide mock user login information, whatever you need to get the GUI working.
Consider a bottom-up approach. You wrote a parser which reads in delimited text files. How do you know if it is working? You write a mock 'data-sink' for those object and route the data there to verify (though usually) that the data are read correctly. The module on the next level up may require 2 data sources, but you have only wrote one.
And while defining the mock objects, you have also define the contract of how the relationship. This is often used in test-driven programming. You write the test cases, use the mock objects to get it working, and often than not, the mock object's interface becomes the final interface (which is why at some point you may want to separate out the mock object's interface into pure abstract class).
Hope this helps
Mock objects/functions can also be useful when working in a team. If you're working on a part of the code base that depends on a different part of the code base that some else is responsible for - which is still being written or hasn't been written yet - a mock object/function is useful in giving you an expected output so that you can carry on with you're work without being held up waiting for the other person to finish their part.
Here are the a few situations where mocking is indispensable:
When you are testing GUI interaction
When you are testing Web App
When you are testing the code that interacts with hardware
When you are testing legacy apps
I will go a different direction here. Stubbing/Faking does all of the things mentioned above, but perhaps the interviewers were thinking of mocks as a fake object that causes the test to pass or fail. I am basing this on the xUnit terminology. This could have lead to some discussion about state testing verses behavior / interaction testing.
The answer they may have been looking for is: That a mock object is different than a stub. A stub emulates a dependency for the method under test. A stub shouldn't cause a test to fail. A mock does this and also checks how and when it is called. Mocks cause a test to pass or fail based on underlying behavior. This has the benefit of being less reliant on data during a test, but tying it more closely to the implementation of the method.
Of course this is speculation, it is more likely they just wanted you to describe the benefits of stubbing and DI.
To take a slightly different approach (as I think mocks have been nicely covered above):
"Take a database--if you're writing test code, you may not want that test hooked up live to the production database where actual operations will be performed."
IMO a bad way of stating the example use. You would never "hook it up to the prod database" during testing with or without mocks. Every developer should have a developer-local database to test against. And then you would move on test environments database then maybe UAT and finally prod. You are not mocking to avoid using the live database, you are mocking in order that classes that are not directly dependent on a database do not require you to set up a database.
Finally (and I accept I might get some comments on this) IMO the developer local database is a valid thing to hit during unit tests. You should only be hitting it while testing code that directly interacts with the database and using mocks when you are testing code that indirectly access the database.
I have written my units tests, and where external resources are needed it is dealt with by using fakes.
All is good so far. Now i' am faced with the other test phases, mainly integration where i want to repeat the unit test methods against real external resources e.g The Database.
So, What are the recommendations for structuring test projects for Unit Vs Integration testing? I understand some people prefer separate assemblies for unit and Integration?
How would one share common test code between the two assemblies? Should i create a thrid assembly which contains all the Abstract Test Classes and let the unit and integration inherit? I am looking for maximum re-usability...
I hear alot of noise about Dependency Injection (StructureMap), How could one utilise such a tool in the given Unit + Integration test setup?
can anyone share some wisdom? Thanks
I don't think you should physically separate the two. A good solution is to put the Microsoft.TeamFoundation.PowerTools.Tasks.CategoryAttribute above your tests to identify regular and integration tests. When running tests (even with MSBuild) you can decide to run only the tests you're interested in.
Alternatively you can put them in seperate namespaces.
For code that will be executed in setup & teardown phases, the base class approach would work well. For integration tests, you can extract the functionality of your unit tests into well-parameterized non-test methods (preferably placed in another namespace) and call these "common" methods from both unit and integration tests. Putting unit tests, integration tests and common methods into separate namespaces would suffice, there would be no need for extra assemblies.
One approach would be to create a separate file with helper methods that would be used across multiple testing contexts, and then include that file in both your unit tests and your functional tests. For the parts that vary, you could use dependency injection - for example, by passing in different factories. In the unit tests, the factory could build a fake object, and in the functional tests it could insert a real object in to your test database.
Whether you split the tests into two projects or keep them in one might depend on the number of classes/tests you have. Too many classes in a single project would make it difficult to dig through. If you do split them out, helper/common methods could be thrown into a third assembly, or you could make them public in the unit test assembly, and let the integration assembly reference that one. Make things only as complex as you have to.
On our project we have both integration and unit tests together but in separate folders. Our project layout is such that we have separate assemblies for the main sections (Domain, Services, etc). Each assembly has a matching test assembly. Test assemblies are allowed to reference other test assemblies.
This means Services.Test can reference Domain.Test which makes sense to us because Services references Domain in the actual code.
In terms of reusable pieces we have
Builders - These provide a fluent interface for creating the most important/complex objects in our domain. These live in the main test folder for our domain. Our domain test assembly is referenced by all other test assemblies.
Mothers - These insert data into the database. They give back an Id for the inserted row which can be used to load the object if required. These live in the main test folder for our services.
Helpers - These are guys that do small things throughout our testing. For instance we prefer to allow access to collections via IEnuermable so we have a CollectionHelper.AssertCountIsEqualTo<_T>(int count, IEnumerable<_T> collection, string message) which wraps the IEnumerable in a List and asserts a count. Our Helpers all live in a common test which every other test references.
As for an IoC container if you can use one on your project they can be a huge help not only in testing (via auto mocking) but also in general development. With the overheard of registering everything with the contain though it might be a bit much for just testing.
After some experimenting this is how you can re-use test methods:
public abstract class TestBase
{
[TestMethod]
public void BaseTestMethod()
{
Assert.IsTrue(true);
}
}
[TestClass]
public class UnitTest : TestBase
{
}
[TestClass]
public class IntegrationTest : TestBase
{
}
The unit and integration test class will pick up the base class test methods and run them as two separate tests.
You should be able to create a parametised constructor on the base class to inject your mocks or resources.
I think this method can olny be used with classed within the same assembly. So it looks like the single assembly approach will have to do for now.
Thanks for the tips ppl!
If the only difference between many of your unit tests and the corresponding integration tests is that the latter use "real" ressources rather than fakes (mocks), one approach is the following:
Make a flag is_unit_test available to your test class from the outside
In the class setup, make fake or real resources available depending on the flag. For instance if you need to use a DB API that is either real (an instance of class DBreal) or fake (an instance of class DBfake), your initialization may look like if is_unit_test then this.dbapi = new DBfake else this.dbapi = new DBreal. (DBreal and DBfake need to conform to the same interface, let's call it DBapi.)
From the point of view of your test methods, step 2 amounts to (manual) Dependency Injection: The method does not know what class actually implements its dependency (the DB API). Rather, the dependency is injected into the method from the outside.
Where your test cases require the DB API, they use this.dbapi
Now you execute one and the same test class with the flag set for unit testing and without the flag set for integration testing. (How to make the flag available depends on your unit testing framework.)
Obviously, the same approach can be used if you need more than one resource in a test class.
Some people will find the explicit if in step 2 ugly. To make it more "elegant", you could employ an Inversion of Control (IoC) container (in Java for instance Spring or PicoContainer) to semi-automate the Dependency Injection instead. The initialization would then look like this.dbapi = myContainer.create(DBapi).
In simple cases, an IoC container will only complicate things, because configuring the container is not trivial, involves learning, opens the possibility of a new class of mistakes, and involves additional files.
In more complex cases however, the container makes things easier, because if the creation of your resources requires still other resources, the container will take care of their initialization as well and complexity would indeed go down. But unless you really get there, I suggest to KISS.
Unless you have an important reason for separate assemblies, they violate KISS. I suggest to wait for that reason first.
(Note that some people may tell you that Dependency Injection is only done at the class level.
I consider this unwarranted dogmatism. Injection simply means that a caller does not know the exact class it is calling, no matter how it obtained the object. It often becomes more useful when applied at the class level, but depending on your test framework this may make things overly complicated in the above case. Note that some test frameworks have their own injection capabilities, though.)
I understand the need to test a class that has logic (for instance, one that can calculate discounts), where you can test the actual class.
But I just started writing unit tests for a project that will act as a repository (get objects from a database). I find myself writing a 'fake' repository that implements an ISomethingRepository interface. It uses a Dictionary<Guid, Something> for storage internally. It implements the Add(Something) and GetById(Guid) methods of the interface.
Why am I writing this? Nothing I'm writing will actually be used by the software when it's deployed, right? I don't really see the value of this exercise.
I also got the advice to use a mock object that I can setup in advance to meet certain expectations. That seems even more pointless to me: of course the test will succeed, I have mocked/faked it to succeed! And I'm still not sure the actual software will perform as it should when connecting to the database...
confused...
Can someone point me in the right direction to help me understand this?
Thank you!
You are not testing your mock object but some other class that is interacting with it. So you could for example test that a controller forwards a save method call to your fake repository. There is something wrong if you are "testing your fake objects"
Don't test the mock class. Do test the production class using the mock class.
The whole point of the test support class is to have something that you can predict its behavior. If you need to test the test support class in order to predict its behavior, there is a problem.
In the fake database article you linked in a comment, the author needs to unit test his fake database because it is his product (at least in the context of the article).
Edit: updated terms to be more consistent.
Mock - created by mocking framework
Fake - created manually, might actually function some.
Test Support - Mocks, Fakes, Stubs, and all the rest. Not production.
The purpose of the mock/stub object is not to be tested instead of the unit you're trying to test, it's to allow you to test that unit without needing other classes.
It's basically so that you can test classes one at a time without having to test all the classes they're also dependent on.
You should not be testing the mock class.
What you normally do is: you create mock classes for all the classes that the class you are testing interact with.
Let's say you are testing a class called Bicycle which takes in the constructor objects of classes Wheel, Saddle, HandleBar,etc.
And then within the class Bike you you want to test test its method GetWeight which probably iterates through each part and calls property/method Weight of them and then returns the total.
What you do:
you write a mock class for each part
(Wheel, saddle etc) which simply
implements the Weight bit
then you pass those mock classes to the Bicycle
test the GetWeight method on the Bicycle class
It that way you can focus on testing the GetWeight on the Bicycle class, in a manner that is independent on other classes (say they are not implemented yet, not deterministic etc.)
Who watches the watchers?
It is interesting for example if the mock implementation throws specific Exceptions for the corner cases, so you know that the classes that use or depend the IRepositorySomething can handle the exceptions that are thrown in real life. Some of these exceptions you can't generate easily with a test database.
You do not test the Mock object with a unit test, but you use it to test classes that depend on it.
Instead of writing a fake class by yourself, you can use a tool (like Rhino or Typemock) to mock it. It is much easier than writing all the mocks yourself. And like others said, there's no need to test fake code, which is no code if you use the tool.
I have actually found two uses for the mock classes that we use in repository implementation testing.
The first is to test the services that use an implementation of the "ISomethingRepository" equivalent that you mention. However, our repository implementations are created by a factory. This means that we do write tests against the "ISomethingRepository", but not against the "MockSomethingRepository" directly. By testing against the interface, we can easily assert that the code coverage for our tests cover 100% of the interface. Code reviews provide simple verification that new interface members are tested. Even if the developers are running against the mock that the factory returns, the build server has a different configuration that tests against the concrete implementation that the factory returns within the nightly builds. It provides the best of both worlds, in terms of test coverage and local performance.
The second use is one that I am surprised that no one else has mentioned. My team is responsible for the middle tier. Our web developers are responsible for the front end of the web products. By building out mock repository implementations, there is not the artificial obstacle of waiting for the database to be modeled and implemented prior to the front-end work starting. Views can be written that will be built off of the mock to provide a minimal amount of "real" data to meet the expectations of the web developers, as well. For example, data can be provided to contain minimum and maximum length string data to verify that neither break their implementation, etc.
Since the factories we use are wired as to which "ISomethingRepository" to return, we have local testing configurations, build testing configurations, production configurations, etc. We purposely are trying to make sure that no team on the project has unreasonable wait times because of another team's implementation time. The largest chunk of wait time is still provided by the development team, but we are able to crank out our domain objects, repositories, and services at a faster pace than the front-end development.
Of course, YMMV. ;-)
You write "fake" class called Stub or Mock object because you want to test an implementation in a simple way without testing the real concrete class. The purpose is to simplify the testing by testing only the Interface (or abstract class).
In your example, you are testing something that has a dictionnary. It might be fill up in real by the database or have a lot of logic behind it. In your "fake" object, you can simplify everything by having all data constant. This way you test only the behavior of the interface and not how the concrete object is built.
There is generally no need to run classical unit tests in the Data Access Layer.
Perhaps you can write integrational style unit test for your Data Acess Classes, that is, Integration Test (= integrating your Data Access Layer Code with the DB) using the features of Unit Testing Frameworks.
For example, in a Spring project you can use Spring Testcontext to start your Spring context inside a unit test, and then connect to a real database and test that the queries returns correct results. You need probably an own database for unit tests, or perhaps you can connect them with a developer DB.
Have a look at the following article for a good explanation of this:
https://web.archive.org/web/20110316193229/http://msdn.microsoft.com/en-us/magazine/cc163358.aspx
Basically, if you write a fake object and it turns out to be fairly complex, it is sometimes worth it to unit test the fake to make sure it works as expected.
Since a repository can be complex, writing unit tests for it often makes sense.