What's the advantage of using mock objects in comparison to a static test database that has known data and using transactions to make sure nothing changes when testing against the database.
You can do both. Use mock objects to test you BLL logic and then use a test database to test your DAL logic. That way if something breaks you can easily see where the problem lies by which test fails.
Firstly, using a mock will be much quicker than connecting to an external database. However the main reason is that mocks behave the same each time the test is run, which you can't guarentee with an external service like a database, which means unit tests don't fail randomly. You can also easily simulate any types of failure you want to handle with a mock object.
It's also a good idea to run integration tests against a real database however to test things like configuration and performance.
If you can ensure that the static test db doesn't change during testing then I think static test db is better than mock objects. But, then it depends on what you want to test and the complexity of the code being tested. I would go with mock objects as they are simpler to maintain compared with a db.
Imagine you're about to write a class which doesn't exist. Maybe it's a controller. It's not something which talks straight to the database.
You have a good idea of how it ought to behave, and you know what it should be responsible for, and what it should delegate to some other service (using Single Responsibility Principle). So you write interfaces to represent the roles of the helper classes it's going to use.
Then you write an example of how you might use your class that you're about to create. If you like, you can call the example a unit test. You mock out the interactions with the helper classes. When you run the example, it fails, because you haven't written the code yet. You can now write the code to make it pass, using the interfaces of the helper classes - which you also haven't written yet.
Then you do the same with the helper classes, mocking out their helpers.
Eventually you'll reach a class which talks to the database. Or maybe it talks to a web service. Or perhaps the data is static, or in memory. (It doesn't matter to the original class, because your class is decoupled from this).
At this point you'll want an example to describe the behaviour of this class, too, and if it's a database connector, you'll need a database for the example.
Writing code this way produces code that's easy to use and understand, with nothing extra that isn't needed by something further up the stack. This is usually more robust than code that's easier to write. That's why we sometimes use mocks - because writing tests first using mocks helps produce good, maintainable, decoupled design.
Usually you would use both of these approaches.
For unit tests you would use mocked objects. This way you can test your system in a much more fine-grained way, because you can mock every object - not only the ones that wrap the database connection. In general it is good practise to unit test every class in separation of all the dependencies. The benefits are - its is possible to test all the error handling on all levels and make sure you test all the code paths, when you get a test failure you can immediately find the cause, etc.
For integration and end-to-end tests you test big parts of the system or the whole thing. Then you would connect to the database (because this connection is part of the test). Obviously you have to ensure that the database is in a known state, etc., etc.
You will end up having much more unit tests than integration tests, so its actually quite important to make them very quick - yet another advantage of using mock objects.
The setup for testing with a db can be arduous. If you find yourself spending more time in setting up the db, just to test some functional aspect of your business logic, then you might want to use a mock/fake.
Related
I have a more general question:
Assuming I have a web application, for example using the Struts2 Framework.
Therefore it becomes quite complicated to write Unit tests for functions, as you have to mock every aspect of the Framework.
The Database+Connection, The Session, a LDAP-Connection or what ever else is needed, which I do not have written on my own
It would be much easier to write the unit Tests so, that they run in a WebInterface inside the Base-Application, as all these things then already would exist.
The question:
Would you guys still consider this as unit testing?
Some thoughts..
The question is very general. My suggestion is that you still want to write some sort of Unit Tests for number of reasons. Firstly you can run them as an automated test suite so if something breaks you know quickly. Secondly you get a better designed system - Your objects are loosely coupled. You get more confident on the code you write.
If you have a framework harder to test,
a. Try abstracting away some dependencies, so they code can be injected without interfering with real instances.
b. Use a testing framework that can break any tightly coupled harder dependencies.
Harder to provide a comprehensive answer, but this is the general direction, which I would suggest.
You should consider what you really want to test first. A framework, for its definition, will use the classes you provide to do some "magic". Do you want to test that has already been tested "magic" or the business core of the app you programmed?.
Also, something you should consider, is where to stop testing. You probably don't want to test the connection to the database (considering what you wrote) so just mock it.
Take in consideration that you will have to test just one functionality at the time, don't think of having, for example, the database connection and the ldap in the same test, it wouldn't be unit testing.
Take a look at this tutorial also :http://tutorials.jenkov.com/java-unit-testing/index.html
I'm currently broadening my Unit Testing by utilising Mock objects (nSubsitute in this particular case). However I'm wondering what the current wisdom when creating a Mock objects. For instance, I'm working with an object that contains various routines to grab and process data - no biggie here but it will be utilised in a fair number of tests.
Should I create a shared function that returns the Mock Object with all the appropriate methods and behaviours mocked for pretty much most of the Testing project and call that object into my Unit Tests? Or shall I Mock the object into every Unit Test, only mocking the behaviour I need for that test (although there will be times I'll be mocking the same behaviour more than one occasion).
Thoughts or advice is gratefully received...
I'm not sure if there is an agreed "current wisdom" on this, but here's my 2 cents.
First, as #codebox pointed out, re-creating your mocks for each unit test is a good idea, as you want your unit tests to run independently of each other. Doing otherwise can result in tests that pass when run together but fail when run in isolation (or vis versa). Creating mocks required for tests is commonly done in test setup ([SetUp] in NUnit, constructor in XUnit), so each test will get a newly created mock.
In terms of configuring these mocks, it depends on the situation and how you test. My preference is to configure them in each test with the minimum amount of configuration necessary. This is a good way of communicating exactly what that test requires of its dependencies. There is nothing wrong with some duplication in these cases.
If a number of tests require the same configuration, I would consider using a scenario-based test fixture (link disclaimer: shameless self-promotion). A scenario could be something like When_the_service_is_unavailable, and the setup for that scenario could configure the mocked service to throw an exception or return an error code. Each test then makes assertions based on that common configuration/scenario (e.g. should display error message, should send email to admin etc).
Another option if you have lots of duplicated bits of configuration is to use a Test Data Builder. This gives you reusable ways of configuring a number of different aspects of your mock or other any other test data.
Finally, if you're finding a large amount of configuration is required it might be worth considering changing the interface of the test dependency to be less "chatty". By looking for a valid abstraction that reduces the number of calls required by the class under test you'll have less to configure in your tests, and have a nice encapsulation of the responsibilities on which that class depends.
It is worth experimenting with a few different approaches and seeing what works for you. Any removal of duplication needs to be balanced with keeping each test case independent, simple, maintainable and reliable. If you find you have a large number of tests fail for small changes, or that you can't figure out the configuration an individual tests needs, or if tests fail depending on the order in which they are run, then you'll want to refine your approach.
I would create new mocks for each test - if you re-use them you may get unexpected behaviour where the state of the mock from earlier tests affects the outcome of later tests.
It's hard to provide a general answer without looking at a specific case.
I'd stick with the same approach as I do everywhere else: first look at the tests as independent beings, then look for similarities and extract the common part out.
Your goal here is to follow DRY, so that your tests are maintainable in case the requirements change.
So...
If it's obvious that every test in a group is going to use the same mock behaviour, provide it in your common set-up
If each of them is significantly different, as in: the content of the mock constitutes a significant part of what you're testing and the test/mock relationship looks like 1:1, then it's reasonable to keep them close to the tests
If the mocks differ between them, but only to some degree, you still want to avoid redundancy. A common SetUp won't help you, but you may want to introduce an utility like PrepareMock(args...) that will cover different cases. This will make your actual test methods free of repetitive set-up, but still let you introduce any degree of difference between them.
The tests look nice when you extract all similarities upwards (to a SetUp or helper methods) so that the only thing that remains in test methods is what's different between them.
During a recent interview I was asked why one would want to create mock objects. My answer went something like, "Take a database--if you're writing test code, you may not want that test hooked up live to the production database where actual operations will be performed."
Judging by response, my answer clearly was not what the interviewer was looking for. What's a better answer?
I'd summarize like this:
Isolation - You can test only a method, independently on what it calls. Your test becomes a real unit test (most important IMHO)
Decrease test development time - it is usually faster to use a mock then to create a whole class just for help you test
It lets you test even when you don't have implemented all dependencies - You don't even need to create, for instance, your repository class, and you'll be able to test a class that would use this repository
Keeps you away from external resources - helps in the sense you don't need to access databases, or to call web services, or to read files, or to send emails, or to charge a credit card, and so on...
In an interview, I'd recommend including that mocking is even better when developers use dependency injection, once it allows you to have more control, and build tests more easily.
When unit testing, each test is designed to test a single object. However most objects in a system will have other objects that they interact with. Mock Objects are dummy implementations of these other objects, used to isolate the object under test.
The benefit of this is that any unit tests that fail generally isolate the problem to the object under test. In some cases the problem will be with the mock object, but those problems should be simpler to identify and fix.
It might be an idea to write some simple unit tests for the mock objects as well.
They are commonly used to create a mock data access layer so that unit tests can be run in isolation from the data store.
Other uses might be to mock the user interface when testing the controller object in the MVC pattern. This allows better automated testing of UI components that can somewhat simulate user interaction.
An example:
public interface IPersonDAO
{
Person FindById(int id);
int Count();
}
public class MockPersonDAO : IPersonDAO
{
// public so the set of people can be loaded by the unit test
public Dictionary<int, Person> _DataStore;
public MockPersonDAO()
{
_DataStore = new Dictionary<int, Person>();
}
public Person FindById(int id)
{
return _DataStore[id];
}
public int Count()
{
return _DataStore.Count;
}
}
Just to add on to the fine answers here, mock objects are used in top-down or bottom-up structural programming (OOP too). They are there to provide data to upper-level modules (GUI, logic processing) or to act as out mock output.
Consider top-down approach: you develop a GUI first, but a GUI ought to have data. So you create a mock database which just return a std::vector<> of data. You have defined the 'contract' of the relationship. Who cares what goes on inside the database object - as long as my GUI list get a std::vector<> I'm happy. This can go to provide mock user login information, whatever you need to get the GUI working.
Consider a bottom-up approach. You wrote a parser which reads in delimited text files. How do you know if it is working? You write a mock 'data-sink' for those object and route the data there to verify (though usually) that the data are read correctly. The module on the next level up may require 2 data sources, but you have only wrote one.
And while defining the mock objects, you have also define the contract of how the relationship. This is often used in test-driven programming. You write the test cases, use the mock objects to get it working, and often than not, the mock object's interface becomes the final interface (which is why at some point you may want to separate out the mock object's interface into pure abstract class).
Hope this helps
Mock objects/functions can also be useful when working in a team. If you're working on a part of the code base that depends on a different part of the code base that some else is responsible for - which is still being written or hasn't been written yet - a mock object/function is useful in giving you an expected output so that you can carry on with you're work without being held up waiting for the other person to finish their part.
Here are the a few situations where mocking is indispensable:
When you are testing GUI interaction
When you are testing Web App
When you are testing the code that interacts with hardware
When you are testing legacy apps
I will go a different direction here. Stubbing/Faking does all of the things mentioned above, but perhaps the interviewers were thinking of mocks as a fake object that causes the test to pass or fail. I am basing this on the xUnit terminology. This could have lead to some discussion about state testing verses behavior / interaction testing.
The answer they may have been looking for is: That a mock object is different than a stub. A stub emulates a dependency for the method under test. A stub shouldn't cause a test to fail. A mock does this and also checks how and when it is called. Mocks cause a test to pass or fail based on underlying behavior. This has the benefit of being less reliant on data during a test, but tying it more closely to the implementation of the method.
Of course this is speculation, it is more likely they just wanted you to describe the benefits of stubbing and DI.
To take a slightly different approach (as I think mocks have been nicely covered above):
"Take a database--if you're writing test code, you may not want that test hooked up live to the production database where actual operations will be performed."
IMO a bad way of stating the example use. You would never "hook it up to the prod database" during testing with or without mocks. Every developer should have a developer-local database to test against. And then you would move on test environments database then maybe UAT and finally prod. You are not mocking to avoid using the live database, you are mocking in order that classes that are not directly dependent on a database do not require you to set up a database.
Finally (and I accept I might get some comments on this) IMO the developer local database is a valid thing to hit during unit tests. You should only be hitting it while testing code that directly interacts with the database and using mocks when you are testing code that indirectly access the database.
How do you decide what to choose:
use mock objects for a test OR
create a test object/ object graph using an IoC framework and run test on that data
It depends what you are trying to test. Unit tests with collaborators mocked out are great because
They are really, really fast
They are small and easy to understand
They don't have dependencies on the wider world which makes them easy to run
They provide excellent defect localisation
However, pure unit tests cannot tell you if you have configured your objects correctly in your IoC container, if the database connection string works etc. You need a test which runs up your IoC container and really reaches out to the Db to prove these things.
If you write as many of your tests as pure, standalone unit tests as possible then your build will stay fast. This is crucial, as a slow bulid gets run less. Even so, don't forget to add a sprinkling of wired tests to prove that your application'hangs together'.
For example, we have a (single) test for every service in our container that proves that we can request it from the IoC container. This proves we're wired up, from then on it is unit tests all the way. We have lots of pure unit tests.
The whole lot is then wrapped in some application level functional tests to prove that the app itself does what the user wants.
The thing to bear in mind is the time cost of each test type. Moving from pure unit -> wired -> functional tests costs an order of magnitude of execution time and complexity when they break.
I am very happy with using IoC for much of my app, and especially I appreciate that test-datasources can be injected for testing.
For more problematic backend connections (currently a single ESB call) or functions that need complicated state I mock.
For unit tests: If an object is not the object tested, mock or stub it.
In that way, you can directly control it so it returns the data that you want.
If you create a test object/object-graph, you have to set it up so that it provides the data that you want. That is probably a lot more work than you want.
For integration tests, of course you'd test a whole object graph at a time.
If you need to write a lot of initialization code - a mocking framework would probably help you write better, easy to understand Unit Tests.
There is no need to re write code that a mocking framework can save you.
I understand the need to test a class that has logic (for instance, one that can calculate discounts), where you can test the actual class.
But I just started writing unit tests for a project that will act as a repository (get objects from a database). I find myself writing a 'fake' repository that implements an ISomethingRepository interface. It uses a Dictionary<Guid, Something> for storage internally. It implements the Add(Something) and GetById(Guid) methods of the interface.
Why am I writing this? Nothing I'm writing will actually be used by the software when it's deployed, right? I don't really see the value of this exercise.
I also got the advice to use a mock object that I can setup in advance to meet certain expectations. That seems even more pointless to me: of course the test will succeed, I have mocked/faked it to succeed! And I'm still not sure the actual software will perform as it should when connecting to the database...
confused...
Can someone point me in the right direction to help me understand this?
Thank you!
You are not testing your mock object but some other class that is interacting with it. So you could for example test that a controller forwards a save method call to your fake repository. There is something wrong if you are "testing your fake objects"
Don't test the mock class. Do test the production class using the mock class.
The whole point of the test support class is to have something that you can predict its behavior. If you need to test the test support class in order to predict its behavior, there is a problem.
In the fake database article you linked in a comment, the author needs to unit test his fake database because it is his product (at least in the context of the article).
Edit: updated terms to be more consistent.
Mock - created by mocking framework
Fake - created manually, might actually function some.
Test Support - Mocks, Fakes, Stubs, and all the rest. Not production.
The purpose of the mock/stub object is not to be tested instead of the unit you're trying to test, it's to allow you to test that unit without needing other classes.
It's basically so that you can test classes one at a time without having to test all the classes they're also dependent on.
You should not be testing the mock class.
What you normally do is: you create mock classes for all the classes that the class you are testing interact with.
Let's say you are testing a class called Bicycle which takes in the constructor objects of classes Wheel, Saddle, HandleBar,etc.
And then within the class Bike you you want to test test its method GetWeight which probably iterates through each part and calls property/method Weight of them and then returns the total.
What you do:
you write a mock class for each part
(Wheel, saddle etc) which simply
implements the Weight bit
then you pass those mock classes to the Bicycle
test the GetWeight method on the Bicycle class
It that way you can focus on testing the GetWeight on the Bicycle class, in a manner that is independent on other classes (say they are not implemented yet, not deterministic etc.)
Who watches the watchers?
It is interesting for example if the mock implementation throws specific Exceptions for the corner cases, so you know that the classes that use or depend the IRepositorySomething can handle the exceptions that are thrown in real life. Some of these exceptions you can't generate easily with a test database.
You do not test the Mock object with a unit test, but you use it to test classes that depend on it.
Instead of writing a fake class by yourself, you can use a tool (like Rhino or Typemock) to mock it. It is much easier than writing all the mocks yourself. And like others said, there's no need to test fake code, which is no code if you use the tool.
I have actually found two uses for the mock classes that we use in repository implementation testing.
The first is to test the services that use an implementation of the "ISomethingRepository" equivalent that you mention. However, our repository implementations are created by a factory. This means that we do write tests against the "ISomethingRepository", but not against the "MockSomethingRepository" directly. By testing against the interface, we can easily assert that the code coverage for our tests cover 100% of the interface. Code reviews provide simple verification that new interface members are tested. Even if the developers are running against the mock that the factory returns, the build server has a different configuration that tests against the concrete implementation that the factory returns within the nightly builds. It provides the best of both worlds, in terms of test coverage and local performance.
The second use is one that I am surprised that no one else has mentioned. My team is responsible for the middle tier. Our web developers are responsible for the front end of the web products. By building out mock repository implementations, there is not the artificial obstacle of waiting for the database to be modeled and implemented prior to the front-end work starting. Views can be written that will be built off of the mock to provide a minimal amount of "real" data to meet the expectations of the web developers, as well. For example, data can be provided to contain minimum and maximum length string data to verify that neither break their implementation, etc.
Since the factories we use are wired as to which "ISomethingRepository" to return, we have local testing configurations, build testing configurations, production configurations, etc. We purposely are trying to make sure that no team on the project has unreasonable wait times because of another team's implementation time. The largest chunk of wait time is still provided by the development team, but we are able to crank out our domain objects, repositories, and services at a faster pace than the front-end development.
Of course, YMMV. ;-)
You write "fake" class called Stub or Mock object because you want to test an implementation in a simple way without testing the real concrete class. The purpose is to simplify the testing by testing only the Interface (or abstract class).
In your example, you are testing something that has a dictionnary. It might be fill up in real by the database or have a lot of logic behind it. In your "fake" object, you can simplify everything by having all data constant. This way you test only the behavior of the interface and not how the concrete object is built.
There is generally no need to run classical unit tests in the Data Access Layer.
Perhaps you can write integrational style unit test for your Data Acess Classes, that is, Integration Test (= integrating your Data Access Layer Code with the DB) using the features of Unit Testing Frameworks.
For example, in a Spring project you can use Spring Testcontext to start your Spring context inside a unit test, and then connect to a real database and test that the queries returns correct results. You need probably an own database for unit tests, or perhaps you can connect them with a developer DB.
Have a look at the following article for a good explanation of this:
https://web.archive.org/web/20110316193229/http://msdn.microsoft.com/en-us/magazine/cc163358.aspx
Basically, if you write a fake object and it turns out to be fairly complex, it is sometimes worth it to unit test the fake to make sure it works as expected.
Since a repository can be complex, writing unit tests for it often makes sense.