I'm working on a personal project (meaning clean source code, no legacy dependencies) and attempting to follow best practices regarding unit testing, dependency management, etc.
My company's codebase is littered with code like this:
public Response chargeCard(CreditCard card, Money amount) {
if(Config.isInUnitTests()) {
throw new IllegalStateException("Cannot make credit card API calls in unit tests!");
}
return CreditCardPOS.connect().charge(card, amount);
}
The goal here is to actively prevent dangerous code / code with external dependencies from being executed during testing. I like the notion of failing fast if unit tests do something bad, but I don't like this implementation for a few reasons:
It leaves hidden dependencies to the static Config class scattered throughout our codebase.
It changes the control flow between testing and live behavior, meaning we're not necessarily testing the same code.
It adds an external dependency to a configuration file, or some other state-holding utility.
It looks ugly :)
A fair number of the places where we use this in my company's codebase could be avoided by better dependency awareness, which I'm attempting to do, but there are still some places where I'm still struggling to see away around implementing an isInUnitTests() method.
Using the credit card example above, I can avoid the isInUnitTests() check on every charge by properly wrapping this up inside a mock-able CardCharger class, or something similar, but while cleaner I feel like I've only moved the problem up one level - how do I prevent a unit test from constructing a real instance of CardCharger without putting a check in the constructor / factory method that creates it?
Is isInUnitTests() a code smell?
If so, how can I still enforce that unit tests don't reach external dependencies?
If not, what's the best way to implement such a method, and what are good practices for when to use / avoid it?
To clarify, I'm trying to prevent unit tests from accessing unacceptable resources, like the database or network. I'm all for test-friendly patterns like dependency injection, but good patterns are useless if they can be violated by a careless developer (i.e. me) - it seems important to me to fail-fast in cases where unit tests do things they aren't supposed to, but I'm not sure the best way to do that.
Is isInUnitTests() a code smell?
Yes, definitively, you have many ways to avoid coupling your code to unit tests. There is no valid reason to have something like that.
If so, how can I still enforce that unit tests don't reach external dependencies?
Your tests must only validate a unit of code and create mock or stubs for external dependencies.
Your code seems to be Java, which it's plenty of mature mock frameworks. Research a little about existing ones and pick the one that likes you more.
EDIT
how do I prevent a unit test from constructing a real instance of HTTPRequest without putting a check in the constructor / factory method that creates it
You're supposed to use a dependency injector to resolve your instance dependencies so you will never need to use a if to test if you're on a test or not since on your "real" code you inject full functional dependencies and on your test you inject mock or stubs.
Take a more serious example, like a credit card charger (a classic unit testing example) - it would seem like a very poor design if it were even possible for test code to access the real card charger without triggering a deluge of exceptions. I don't think it's enough in a case like that to trust the developer will always do the right thing
Again, you're supposed to inject external dependencies as a credit card charger so on your test you would inject a fake one. If some developer don't know this, I think the first thing your company needs is training for that developer and some pair programming to guide the process.
In any case, kind of I get your point and let me tell you a similar situation I experienced.
There was an application that sent a bunch of emails after some processing. These emails were not to be send on any other environment except live, otherwise it would be a big problem.
Before I started working on this application, it happened several times that developers "forgot" about this rule and emails were sent on test environments, causing a lot of problems. It's not a good strategy to depends on human memory to avoid this kind of problem.
What we did to avoid this to happen again was adding a config setting to indicate if send real emails or not. The problem was broader than executing things or not on unit tests as you can see.
Nothing replace communication though, a developer could set the incorrect value for this setting on his development environment. You're never 100% safe.
So in short:
First line of defense is communication and training.
You second line of defense should be dependency injection.
If you feel that this is not enough, you could add a third line of defense: a configuration setting to avoid executing real logic on test/development environments. Nothing wrong about that. But please don't name it IsInUnitTest as the problem is broader than that (you want also o avoid this logic to be executed on developer machine)
The AngularJS Unit Testing Documentation actually does a really fantastic job of describing the different ways that developers handle unit testing.
Of the four different methods they outline, the one they recommend is one that involves using dependency injection so that your production code flow is the same as your testing flow. The only differences are that the objects you pass into your code would be different. Note that Claudio used the term "Stub" to refer to objects you pass into a method to act as placeholders for those you'd use in production:
public Document getPage(String url, AbstractJsoup jsoup) {
return jsoup.connect(url).get();
}
My Java is a little rusty, so consider that you may need to actually do some funky stuff to make this work, like check the type of the jsoup object or cast it to your test Jsoup object or your production Jsoup object, which might just defeat the entire purpose of using dependency injection. Thus, if you're talking about a dynamic language like JavaScript or some other loosely-typed functional programming language, dependency injection would keep things really clean.
But in Java, unless you control the dependencies and can use a design pattern, like the strategy pattern, to pass in different concrete implementations, doing what you're doing may be easier, especially since you don't control the Jsoup code. However, you might check to see if they have available stubs you can use as placeholders, as some library developers do write stubs.
If you don't own the code, another option could be to use a Factory class to obtain the desired objects, depending on a flag you set when first instantiating it. This seems like a less desirable solution, since you're still sort of setting a global flag in that Factory object that may have an effect on things you might not be trying to test. The advantage of dependency injection for unit testing is that you must explicitly pass in a test stub when testing and explicitly pass in the production object when you wish the method to do what it was written to do. Once the method completes its execution, the test is over, and any production process that invokes it will automatically run it in production mode because it will inject production objects.
I've never seen a system where steps were taken to actively prevent code running under unit tests from accessing external resources. The problem just never came up. You have my deepest sympathy for working somewhere where it has.
Is there a way you could control the classpath used for unit tests, to make the libraries needed to access external resources unavailable? If there is no JSoup and no JDBC driver on the classpath, tests for code which attempts to use them will fail. You won't be able to exclude JDK-level classes like Socket and URLConnection this way, but it might still be useful.
If you were running tests using Gradle, that would be fairly straightforward. If you are using Maven, maybe not. I don't think there's any way to have different classpaths for compilation and testing in Eclipse.
Well, you can achieve the same clean way using Abstract Factory Pattern though (maybe not suitable to call it Abstract Factory Pattern).
Example in C#:
public class CardChargerWrapper{
public CardChargerWrapper(
NullCardCharger nullCharger
, TestUnitConfig config){
// assign local value
this.charger = new CardCharger();
}
CardCharger charger;
NullCardCharger nullCharger;
TestUnitConfig config;
public Response Charge(CreditCard card, Money amount){
if(!config.IsUnitTest()) { return charger.connect().charge(card, amount); }
else { return NullCardCharger.charge(card, amount); }
}
}
EDIT: Changed CardChargerWrapper to use hard-coded instance of CardCharger instead of injecting it.
Note: You can change NullCardCharger to something like MockCardCharger or OfflineCardCharger for logging purpose.
Note again: You can change the CardChargerWrapper's constructor to fit. Example, instead of constructor injecting the NullCardCharger, you can make it property injected. Same with TestUnitConfig.
EDIT: Regarding calling IsUnitTest() a good idea or not:
It really depends on your business perspective and how you are doing testings. As many people said, a code that has not yet been tested is not trusted for their correctness. It cannot be reliabled. On side note, I prefer IsChargeRealCard() will be more suitable than IsUnitTest().
Say that we take out the unit test in our context, at least you will still need to do integration testing in test environment. You probably want to test something like:
I want to test the credit card validation (is it real or not, etc).
I want to test the payment method and see whether the card being charged. As a process, but not as a real credit card payment.
For 2nd point, I think the best thing to do is to create a mock credit card charger to log the transaction. This is, to ensure that the charging is correct. And it will be conducted in both test and dev server.
So, How can the CardChargerWrapper help such situation?
Now with CardChargerWrapper, you can:
Switch the NullCardCharger to any mock card chargers to enhance your unit testing.
All class using CardChargerWrapper can be ensured that they are checking IsUnitTest first before charging real card.
Your developer need to use CardChargerWrapper instead of CardCharger, preventing development error for unit tests.
During code review, you can find whether CardCharger being used in other class instead of CardChargerWrapper. This is to ensure no leak of code.
I'm unsure, but seems like you can hide references of your main project to your real CardCharger. This will protect your code further.
If [isInUnitTest() is an antipattern], how can I still enforce that unit tests don't reach external dependencies?
I now have a solution I'm fairly satisfied with, which will ensure external dependencies cannot be used in test environments without explicitly enabling them. All classes which depend on external resources (HTTP requests, databases, credit card processors, etc.) take as one of their arguments a configuration class which contains the necessary settings to initialize these objects. In a real environment, a real Config object is passed in containing the data they need. In a test environment, a mock is passed in, and without explicitly configuring the mock, the object will fail to construct/connect.
For instance, I have an Http connection utility:
public class HttpOp {
public HttpOp(Config conf) {
if(!conf.isHttpAllowed()) {
throw new UnsupportedOperationException("Cannot execute HTTP requests in "+
getClass()+" if HTTP is disabled. Did you mean to mock this class?");
}
}
....
}
In a unit test, if you attempted to run code that constructs an HttpOp object, you'd raise an exception as the mocked Config will not return true unless explicitly set to do so. In a functional test where this is desired, you can do so explicitly:
#Test
public void connect() {
State.Http httpState = mock(State.Http.class);
when(httpState.isEnabled()).thenReturn(true);
RemoteDataProcessor rdp = new RemoteDataProcessor(new HttpOp(httpState));
...
}
Of course, this still depends on Config being properly mocked in test environments, but now we have exactly one danger point to look for and reviewers can quickly verify the Config object is mocked and trust that only explicitly enabled utilities will be accessible. Similarly, there's now only one gotcha new team-members need to be told ("always mock Config in tests") and they can now be confident they won't accidentally charge a credit card or send emails to clients.
Update
This was an idea I had shortly after posting this question, however I'm now convinced it's not a good plan. Leaving it here for posterity, but see my newer answer for what I ended up doing.
I'm far from certain this is the right thing to do, but one thought I had, which at least addresses my first and third objections, comes from Determine if code is running as part of a unit test:
You could avoid storing external state about whether you're in a unit test or not by directly examining the execution stack, like so:
/**
* Determines at runtime and caches per-thread if we're in unit tests.
*/
public static class IsInUnitTest extends ThreadLocal<Boolean> {
private static final ImmutableSet<String> TEST_PACKAGES =
ImmutableSet.of("org.testng");
#Override
protected Boolean initialValue() {
for(StackTraceElement ste : Thread.currentThread().getStackTrace()) {
for(String pkg : TEST_PACKAGES) {
if(ste.getClassName().startsWith(pkg)) {
return true;
}
}
}
return false;
}
}
The primary advantage here is we don't store any state; we simply check the stack trace - if the trace contains a test-framework package, we're in a unit test, otherwise we're not. It's not perfect - in particular it could potentially lead to false positives if you use the same testing framework for integration or other more permissive tests - but avoiding external state seems like at least a slight win.
Curious what others think of this idea.
Related
The most experienced developer in my current team has set a few hard rules based on best practices we should follow. Among them is "You never ever mock domain object". I asked him why we couldn't but he never had time to give me a proper answer. Now he's away for a week, and I reached a peak distrust to that rule, here's my situation.
My domain object has an Update method with several parameters, one being an interface for a calculator. It then updates a few fields, runs the calculators and assign its results to some other of its fields.
The proper unit test for the Update method itself is reasonably long.
Now I have some piece of code which do a few things then call Update on such a domain object.
I would normally mock the objects, and just check the Update method is being called with the proper arguments. But now, I have to test it's being properly called, by checking its fields and mocking the calculator then same way I did when unit testing the Update method itself. And I would have to do that everywhere the Update methods gets called.
How fun will it be when the Update method change a bit, and every of those tests suddenly break and need refactoring... I feels this rule is just like shooting myself in the foot..
So I need to know, why You never ever mock domain object"?
You never ever mock domain object
This is more of a heuristic than a best practice. And I can imagine that it make sense in some projects.
This heuristic can help to have better tests. Good tests are easy to write, easy to read and they will break as soon as something goes wrong. If you mock your domain model, you will probably also have to mock it's behavior. And if you have some complex behavior in your model, mocking it will be time consuming. The test will also be more fragile (mistakes can happen in predicting the domain model output).
The other alternative is to do sociable unit testing. That means the unit to be tested will not be a single class in isolation but a class and some of its dependencies. In other words, you just use a concrete object of your domain model in the test.
I would normally mock the objects, and just check the Update method is being called with the proper arguments. But now, I have to test it's being properly called, by checking its fields and mocking the calculator then same way I did when unit testing the Update method itself.
Checking that the update method is being called doesn't necessarily mean the success of the test. This is probably not what you want to check to see if your service works correctly. If you pass a concrete class of your model(including the calculators) you won't have to do any useless checks or repeat what has been done in other tests.
One of the differences between this solution and testing classes in isolation is that when there is bug in your domain model, many other tests will fail. But this is totally fine in my opinion since it will have to be fixed anyway.
You never ever mock domain object
There might be different reasons to avoid mocking your domain objects:
Mocking libraries, the same as ORMs, can affect design of your domain objects. They usually require interfaces or virtual methods in C#.
A preference to test against real instances. Very often people use mocking libraries because it is an easy way to create a fake or a stub. Very often you can build an instance of your domain object and use it in your test. Most likely a fact that method Update was called can be easily checked via state change.
I would normally mock the objects, and just check the Update method is being called with the proper arguments. But now, I have to test it's being properly called, by checking its fields and mocking the calculator then same way I did when unit testing the Update method itself.
How fun will it be when the Update method change a bit, and every of those tests suddenly break and need refactoring...
It was already mentioned in the comment that probably Update method has to many responsibilities and that's why is used in lots of use cases.
There are different tastes when it comes to defining a system under test (sut). If you tend to test classes then you often find yourself using mocking libraries. If you tend to test scenarios or behaviors then most likely you will only care about state changes.
You can probably explore a different design for your domain object: Try to split Update method into explicit domain specific methods so each client will trigger a different behavior. Given this approach a "reasonably long" test for the Update method can even become obsolete since it will be tested by other tests.
While Update() is probably not as tailored a method as you would expect from a rich domain model - which may have an impact on test fragility - I agree that you should take "never do X" advice with a grain of salt.
If you understand all the implications and know that the integration test version will be less maintainable, definitely use mocks. You, and not blanket statements or cargo cults, are the ultimate judge to what's better for your system in your context.
I don't think this was asked before, although it's really hard to search for a term like unit test ioc container and not find a question about how to implement IoC in order to perform unit tests.
I would like to have unit tests against the IoC container itself basically because sometimes I have issues with the container (like you could with any other part of an application), and it's pretty troublesome to test the resolution of dependencies merely debugging.
If I could introduce unit tests for these cases, I think it would save me a lot of trouble.
Update
Is something like this, not an Unit Test? Is it an integration test?
[TestClass]
public class IoC
{
private IWindsorContainer _container;
[TestInitialize]
public void TestInit()
{
_container = new WindsorContainer();
_container.Install(new WindsorInstaller());
}
[TestMethod]
public void ContainerShouldResolve_MembershipProvider()
{
ContainerShouldResolve<IMembershipProvider>();
}
public void ContainerShouldResolve<T>()
{
T result = _container.Resolve<T>();
Assert.IsInstanceOfType(result, typeof(T));
}
}
The only real "not self-contained" reference is a connection string I had to wire into app.config. Also: when trying to resolve PerWebRequest lifestyle components I had to add the related httpModule, too.
By the way: by doing this I found out the source of my issue in little time compared to what it was taking me to debug it through using the web application.
This falls more into integration testing category. Your components registration might do all kinds of external systems calls (database, file system, network, services ...) when resolved, and this is where unit testing ends.
One approach you can do to this kind of (integration) testing, is to simply resolve your application root type. This might not be complete (especially when your application does large bits of lazy loading), but is often good enough to spot missing bits.
Edit #2 (in response to OP edit)
Of course, it might be possible to do root-resolve test without actually touching any external systems mentioned, but there still might be lot of dependencies wiring and setting up going on real objects (as in, not fakes/stubs). This is many potential reasons to fail (I'd say even too many for unit test) and is up to developer to judge what testing category this falls into.
Doing component-resolve test (resolve that relates to much smaller scope) might be fine for unit test, but once again - it's for developer to judge and it largely depends on what the objects do upon resolve. Most modern containers usually offer more than simple storage and this should be taken into account.
What I'm trying to say is, if you have say DaoModule and you want to verify it can be resolved from container, sure, this might be an unit test. But if resolving your DaoModule setups connection and queries DB to see if it is in valid state, you probably need to reconsider where such test would go.
Edit #1 (in response to OP comment under question)
I want to set up a test where I configure the container, throw an abstract type (or interface) at it, and have it correctly resolve to an expected type.
Basically, you want to verify whether your container works? Don't do that. Simply pick a container that is tested and you can assume it can do its job. If you feel the need to unit test your 3rd party components, you really should be choosing different ones.
If you really whant to write unittests for infrastructure-code like IoC to check that the IoC implementation is correct you can look into the unittests sourcecode of the IoC.
For example DotNet-s autofac IoC Unittests
The other answers and comments on your question (and I in the first run also) thought that you want to write an autoamted test that makes shure that your components where correctly wired by the IoC. Most of us call this kind of test an integrationtest and not a unittest.
I would like to start doing more unit testing in my applications, but it seems to me that most of the stuff I do is just no suitable to be unit tested. I know how unit tests are supposed to work in textbook examples, but in real world applications they do not seem of much use.
Some applications I write have very simple logic and complex interactions with things that are outside my control. For instance I would like to write a daemon which reacts to signals sent by some applications, and changes some user settings in the OS. I can see three difficulties:
first I have to be able to talk with the applications and be notified of their events;
then I need to interact with OS whenever I receive a signal, in order to change the appropriate user settings;
finally all of this should work as a daemon.
All these things are potentially delicate: I will have to browse possibly complex APIs and I may introduce bugs, say by misinterpreting some parameters. What can unit testing do for me? I can mock both the external application and the OS, and check that given a signal from the application, I will call the appropriate API method on the OS. This is... well, the trivial part of the application.
Actually most of the things I do involve interaction with databases, the filesystem or other applications, and these are the most delicate parts.
For another example look at my build tool PHPmake. I would like to refactor it, as it is not very well-written, but I fear to do this as I have no tests. So I would like to add some. The point is that the things which may be broken by refactoring may not be caught by unit tests:
One of things to do is deciding which things are to be built and which one are already up to date, and this depends on the time of last modification of the files. This time is actually changed by external processes, when some build command is fired.
I want to be sure that the output of external processes is displayed correctly. Sometimes the buikd commands require some input, and that should be also managed correctly. But I do not know a priori which processes will be ran - it may be anything.
Some logic is involved in pattern matching, and this may seem to be testable part. But the functions which do the pattern matching use (ni addition to their own logic) the PHP function glob, which works with the filesystem. If I just mock a tree in place of the actual filesystem, glob will not work.
I could go on with more examples, but the point is the following. Unless I have some delicate algorithms, most of what I do involves interaction with external resources, and this is not suitable for unit testing. More that this, often this interaction is actually the non-trivial part. Still many people see unit testing as a basic tool. What am I missing? How can I learn be a better tester?
I think you open a number of issues in your question.
Firstly, when your application integrates with external environments such as OS, other threads, etc. then you have to separate (1) the logic that is tied in with the external enviroment and (2) your business-code.. that is, the stuff your application does. This is no different to how you would separate GUI and SERVER in an application (or web application).
Secondly, you ask if you should test simple logic. I'd say, it depends. Often simple fetch/store functionality is nice to have tests for. It's like the foundation of your application.. hence its important. Other business stuff built upon your foundation that is very simple, you may easily find yourself both feeling that you are wasting your time, and mostly you are :-)
Thirdly, refactory an existing program and testing it in its existing state may be a problem. If your PHP program produces a set of files on the basis of some input, well, maybe thats your entry point to tests are. Sure the tests may be high-level, but it's an easy way to ensure that after the refactoring, your program produces the same output. Hence, aim for higher-level tests in that situation in the start phase of your refactoring efforts.
I'd like to recommend some literature, but I can only come up with one title. "Working Effectively with Legacy Code" By Micheal Feathers. It's a good start. Another would be "xUnit Test Patterns: Refactoring Test Code" by Gerard Meszaros (although that book is much more sloppy and FULL of copy paste text).
As regards your issue about existing code bases that aren't currently covered by tests in which you would like to start refactoring, I would suggest reading:
Working Effectively with Legacy Code By Micheal Feathers.
That book gives you techniques on how to deal with the issues you might be facing with PHPMake. It provides ways to introduce seams for testing, where there previously weren't any.
Additionally, with code that touches say the file systems, you can abstract the file system calls behind a thin wrapper, using the Adapter Pattern. The unit tests would be against a fake implementation of the abstract interface that the wrapping class implements.
At some point you get to a low enough level where a unit of code can't be isolated for unit testing as these depend on library or API calls (such as in the production implementation of the wrapper). Once this happens integration tests are really the only automated developer tests you can write.
I recommend this google tech-talk on unit testing.
The video boils down to
write your code so that it knows as little about how it will be used as possible. The less assumptions your code makes, the easier it is to test. Avoid complex logic in constructors, the use of singletons, static class members, and so on.
isolate your code from the external world (comms, databases, real time), and make sure that your code only talks to your isolation layer. Otherwise, writing tests will be a nightmare in terms of 'fake environment' setup.
unit tests should test stories; that is what we really understand and care for; given a class with a method foo(), testFoo() is uninformative. They actually recommend test names like itShouldCloseConnectionEvenWhenExceptionThrown(). Ideally, your stories should cover enough functionality that you can rebuild the spec from the stories.
NOTE: the video and this post use Java as an example; however, the main points stand for any language.
"Unit tests" tests one unit of your code. No external tools should be involved. This seems to be complicated for your first app (without knowing to much about it ;)) but the phpMake is unit-testable - I'm sure ... because ant, gradle and maven are unit-testable too ;)!
But of course you can test your first application automated too. There are several different layers one could test an application.
So the task for you is to find an automated way to test your app - be it integration testing or whatever.
E.g. you could write shell scripts, which asserts some output! With that you make sure your application behaves correctly ...
Tests of interactions with external resources are integration tests, not unit tests.
Tests of your code to see how it would behave if particular external interactions had occurred can be unit tests. These should be done by writing your code to use dependency injection, and then, in the unit test, injecting mock objects as dependencies.
For example, consider a piece of code that adds the results of a call to one service to the results of a call to another service:
public int AddResults(IService1 svc1, IService2 svc2, int parameter)
{
return svc1.Call(parameter) + svc2.Call(parameter);
}
You can test this by passing in mock objects for the two services:
private class Service1Returns1 : IService1
{
public int Call(int parameter){return 1;}
}
private class Service2Returns1 : IService2
{
public int Call(int parameter){return 1;}
}
public void Test1And1()
{
Assert.AreEqual(2, AddResults(new Service1Returns1(), new Service2Returns1(), 0));
}
First of all, if unit testing doesn't seem like it would be much use in your applications, why do you even want to start doing more of it? What is motivating you to care about it? It is definitely a waste of time if a) you do everything perfect the first time and nothing ever changes or b) you decide it's a waste of time and do it poorly.
If you do think that you really want to do unit testing, the answer to your questions are all the same: encapsulation. In your daemon example, you could create a ApplcationEventObeservationProxy with a very narrow interface that just implements pass through methods. The purpose of this class is to do nothing but completely encapsulate the rest of your code from the third-party event observing library (nothing means nothing -- no logic here). Do the same thing for OS settings. Then you can completely unit test the class that does actions based on events. I'd recommend have a separate class for the daemon-ness that just wraps your main class -- it will make the testing easier.
There are a couple of benefits to this approach outside of unit testing. One is that if you encapsulate the code that interacts directly with the OS, it's easier to switch it out. This kind of code is particularly prone to breakage outside of your control (i.e., MS patchsets). You will also probably want to support more than one OS, and if the OS specific logic is not tangled with the rest of your logic, it will be easier. The other benefit is that you'll be forced to realize that there is more business logic in your app than you think. :)
Finally, don't forget that unit testing is a foundation for a good product, but not the only ingredient. Having a set of tests that explore and verify the OS API calls you'll be using is a good strategy for the "hard" parts of this problem. You should also have end to end tests that ensure the events in your applications cause the OS setting changes.
As other answers suggested Working Effectively with Legacy Code By Micheal Feathers is a good read. If you have to deal with legacy code, and you want to make sure that the systems interaction work as expected, try writing integration tests first. And then it is more appropriate to write Unit Tests to test the behaviour of methods that are valued from the requirements point of view. You Tests serve a whole different purpose than the integration tests. Unit Tests are more likely to improve the design of your system than testing how everything hangs to gather.
During a recent interview I was asked why one would want to create mock objects. My answer went something like, "Take a database--if you're writing test code, you may not want that test hooked up live to the production database where actual operations will be performed."
Judging by response, my answer clearly was not what the interviewer was looking for. What's a better answer?
I'd summarize like this:
Isolation - You can test only a method, independently on what it calls. Your test becomes a real unit test (most important IMHO)
Decrease test development time - it is usually faster to use a mock then to create a whole class just for help you test
It lets you test even when you don't have implemented all dependencies - You don't even need to create, for instance, your repository class, and you'll be able to test a class that would use this repository
Keeps you away from external resources - helps in the sense you don't need to access databases, or to call web services, or to read files, or to send emails, or to charge a credit card, and so on...
In an interview, I'd recommend including that mocking is even better when developers use dependency injection, once it allows you to have more control, and build tests more easily.
When unit testing, each test is designed to test a single object. However most objects in a system will have other objects that they interact with. Mock Objects are dummy implementations of these other objects, used to isolate the object under test.
The benefit of this is that any unit tests that fail generally isolate the problem to the object under test. In some cases the problem will be with the mock object, but those problems should be simpler to identify and fix.
It might be an idea to write some simple unit tests for the mock objects as well.
They are commonly used to create a mock data access layer so that unit tests can be run in isolation from the data store.
Other uses might be to mock the user interface when testing the controller object in the MVC pattern. This allows better automated testing of UI components that can somewhat simulate user interaction.
An example:
public interface IPersonDAO
{
Person FindById(int id);
int Count();
}
public class MockPersonDAO : IPersonDAO
{
// public so the set of people can be loaded by the unit test
public Dictionary<int, Person> _DataStore;
public MockPersonDAO()
{
_DataStore = new Dictionary<int, Person>();
}
public Person FindById(int id)
{
return _DataStore[id];
}
public int Count()
{
return _DataStore.Count;
}
}
Just to add on to the fine answers here, mock objects are used in top-down or bottom-up structural programming (OOP too). They are there to provide data to upper-level modules (GUI, logic processing) or to act as out mock output.
Consider top-down approach: you develop a GUI first, but a GUI ought to have data. So you create a mock database which just return a std::vector<> of data. You have defined the 'contract' of the relationship. Who cares what goes on inside the database object - as long as my GUI list get a std::vector<> I'm happy. This can go to provide mock user login information, whatever you need to get the GUI working.
Consider a bottom-up approach. You wrote a parser which reads in delimited text files. How do you know if it is working? You write a mock 'data-sink' for those object and route the data there to verify (though usually) that the data are read correctly. The module on the next level up may require 2 data sources, but you have only wrote one.
And while defining the mock objects, you have also define the contract of how the relationship. This is often used in test-driven programming. You write the test cases, use the mock objects to get it working, and often than not, the mock object's interface becomes the final interface (which is why at some point you may want to separate out the mock object's interface into pure abstract class).
Hope this helps
Mock objects/functions can also be useful when working in a team. If you're working on a part of the code base that depends on a different part of the code base that some else is responsible for - which is still being written or hasn't been written yet - a mock object/function is useful in giving you an expected output so that you can carry on with you're work without being held up waiting for the other person to finish their part.
Here are the a few situations where mocking is indispensable:
When you are testing GUI interaction
When you are testing Web App
When you are testing the code that interacts with hardware
When you are testing legacy apps
I will go a different direction here. Stubbing/Faking does all of the things mentioned above, but perhaps the interviewers were thinking of mocks as a fake object that causes the test to pass or fail. I am basing this on the xUnit terminology. This could have lead to some discussion about state testing verses behavior / interaction testing.
The answer they may have been looking for is: That a mock object is different than a stub. A stub emulates a dependency for the method under test. A stub shouldn't cause a test to fail. A mock does this and also checks how and when it is called. Mocks cause a test to pass or fail based on underlying behavior. This has the benefit of being less reliant on data during a test, but tying it more closely to the implementation of the method.
Of course this is speculation, it is more likely they just wanted you to describe the benefits of stubbing and DI.
To take a slightly different approach (as I think mocks have been nicely covered above):
"Take a database--if you're writing test code, you may not want that test hooked up live to the production database where actual operations will be performed."
IMO a bad way of stating the example use. You would never "hook it up to the prod database" during testing with or without mocks. Every developer should have a developer-local database to test against. And then you would move on test environments database then maybe UAT and finally prod. You are not mocking to avoid using the live database, you are mocking in order that classes that are not directly dependent on a database do not require you to set up a database.
Finally (and I accept I might get some comments on this) IMO the developer local database is a valid thing to hit during unit tests. You should only be hitting it while testing code that directly interacts with the database and using mocks when you are testing code that indirectly access the database.
A part of my project interacts with iTunes using COM. The goal of the test in question is to verify that my object asks the iTunes API to export all of the album artwork for a collection of tracks to a file.
I have successfully written a test which can prove my code is doing this, however to accomplish that I have had to stub out a chunk of the iTunes implementation, while thiis is to be expected in a unit test I am concerned at the ratio of stub setup code vs. code doing actual testing
My questions:
Is the fact that there is more stub setup code then acting code indicitive of another underlying problem in my code>
There is a lot of setup code and I don't believe repeating it per test is a good idea. What is the best way to refactor this code so that this setup code is seperate from, but available to other tests that need to utilise the stubs.
This seams like the kind of question that might have been asked before, so I applagise in advance if I have created a duplicate
For reference, here is the complete unit test that I am concerned about
[Fact]
public void Add_AddTrackCollection_AsksiTunesToExportArtForEachTrackInCollectionToAFile()
{
var trackCollection = MockRepository.GenerateStub<IITTrackCollection>(null);
var track = MockRepository.GenerateStub<IITTrack>(null);
var artworkCollection = MockRepository.GenerateStub<IITArtworkCollection>(null);
var artwork = MockRepository.GenerateMock<IITArtwork>(null);
var artworkCache = new ArtworkCache();
trackCollection.Stub<IITTrackCollection, int>(collection => {return collection.Count; }).Return(5);
trackCollection.Stub<IITTrackCollection, IITTrack>(collection => { return trackCollection[0]; }).IgnoreArguments().Return(track);
track.Stub<IITTrack, IITArtworkCollection>(stub => { return stub.Artwork; }).Return(artworkCollection);
artworkCollection.Stub<IITArtworkCollection, int>(collection => { return collection.Count; }).Return(1);
artworkCollection.Stub<IITArtworkCollection, IITArtwork>(collection => { return artworkCollection[0]; }).IgnoreArguments().Return(artwork);
artwork.Expect<IITArtwork>(stub => { stub.SaveArtworkToFile(null); }).IgnoreArguments().Repeat.Times(trackCollection.Count-1);
artwork.Replay();
artworkCache.Add(trackCollection);
artwork.VerifyAllExpectations();
//refactor all the iTunes fake-out that isn't specific to this test into its own method and call that from ctor.
The need for a lot of setup code indicates that your code under test interacts deeply with a complex interface; that's not necessarily a problem (it's not, per se, a "code smell") though it may suggest that eventually you might put a facade in front of that complex interface and interact with that (at that time testing the facade may require some setup, but your "substantial" application code will be simpler AND simpler to test), at least if you can identify any repetitive patterns of interaction you may be using.
Widespread unit-test frameworks such as junit, unittest, &c, recognize the need for repetitive setup code and thereby define "test case" classes with a setUp method (and a corresponding tearDown one in case you need to undo something there). If you have otherwise disparate groups of tests that yet need SOME common setup code, but also some setup that's per-group but different from other groups, you can share the "common setup code" by inheriting from an appropriate abstract base class, calling a global function, or as otherwise appropriate in your favorite language.
To address both of your questions:
In both common unit test patterns (Arrange/Act/Assert, and Four-Phase Test), there will almost always be more Arrange code than Act code, since by definition, Act should only contain a single statement. However, it is still a good idea to attempt to minimize the Arrange code as much as possible.
There are more than one way to refactor Setup code into reusable code. As Alex writes in his answer, many unit testing frameworks support setup methods. This is called Implicit Fixture Setup, and in my opinion something that should be avoided, since it does not communicate intent very well. Instead, I prefer explicit setup methods, usually encapsulated into a Fixture Object.
In general, the need for complex Setup code should always cause you to consider if you can model your API differently. This is not always the case, but when it is, you often end up with a better and more concise API than you started out with. That's one of the advantages of TDD.
For the cases where setup is just complex because input is complex, I will recommend AutoFixture, which is a general-purpose Test Data Builder.
Many of the patterns I have used in this answer are described in xUnit Test Patterns, which is an excellent book.