When I write tests for certain types of objects, such as UI elements like Forms or UserControls, I often find myself altering my TDD pattern; instead of going test-first, I define and lay out the form's controls, to provide a "skeleton", then start writing behavioral tests (databinding/"unbinding", display-mode behavior, etc). In doing so, I find myself dealing with non-public members. I also run into the same concern with other behavioral methods; I may want to focus on and exercise logic in some private helper called by another method, before worrying about the usage in the other method and its behavior.
To me, making everything public (and sometimes virtual) just to be able to unit-test everything is a smell; I don't want other objects being able to call a helper, or directly access a textbox; but I need to know that the helper does its job and the textbox gets its value when the form loads.
The solution I arrived at some time ago is to create a "test proxy" for the actual object under test. The proxy derives from the object under test, and doesn't hide or override any behavior, but it does provide internally-visible getters, setters and/or methods that make calls to non-public members of the object under test, allowing me to tell the object to perform certain actions of which I can then view the results, without requiring the test to also depend on proper integration within the object or making the method or some other member of interest public in production code just for testing purposes.
Advantages I see:
Members' visibility is not determined by whether you want a unit test or not.
Finer control over what you can do with the object in a test allows for more flexible and extensible testing.
Disadvantages I see:
Class count increases, with an extra level to develop just for testing purposes.
Care must be taken not to somehow end up using the test proxy in production code (making the constructor or the entire class internal generally does the trick)
Not a "pure" unit test as you are, at some level, dependent on integration between the proxy and actual object under test.
The question is, is this a valid way to architect unit tests, or does the fact that I have to do this indicate a problem with the code or the testing strategy?
My first reaction to this pattern is that you're de-emphasizing the 'D's in TDD. Your code is tested but those tests are not driving your design so the code you end up with has a different structure than it would if you had written tests first. A structure with more inaccessible private state than necessary. In general I'll argue that if you can't test your class' behavior using the public interfaces then you are either writing a test which doesn't make sense (testing implementation details) or you have a poorly designed public interface.
However if you're working with view classes this becomes a bit more complicated since you have "public" inputs and outputs via your view which you want to test but which are not necessarily exposed to the code using this view component. In that case I think it makes sense to give your tests access to that user interface; either by exposing those normally private attributes to the test (your proxy is one option and others may be available depending on the language you are using) or by writing some form of functional test which can drive the UI (again tools available depend on your environment).
I would say it indicates a problem with your testing strategy or code. The reason is because your violating encapsulation which is going to couple your tests to the implementation rather than the interface. This will add to the overall work you do because refactors (for example) may no longer be free.
That being said I think there a good reasons to violate encapsulation, and they revolve around functions with side effects (which is often the case with UI programming). In many cases you need to assure that functions are being called in a particular order or that they are called at all. I think there are ways to mitigate how much you violate encapsulation.
If I'm writing tests for side effects I will usually separate them out into their own test. I will also stub/mock out the side effect functions and assert they are called according to my requirements (ie, order, timing, called or not, arguments, etc). This frees me from knowing the implementation details, but still allows me to assert that particular functions were called properly.
In certain languages it can be difficult to mock out the objects, or methods. In those cases I will use dependency injection to pass the object or functions with side effects. That way when testing I can pass my mocks for verification.
MSTest uses this method to test private methods. Visual Studio puts all the tests in a separate test project and creates an "Accessor" class. That class is a subclass where all the private members are made public. Since this subclass is in the test project it isn't available in the assembly under test. I think this is a viable pattern for testing private methods and could be manually implemented in a TDD environment if you aren't using Visual Studio and MSTest.
Related
I've recently started practising TDD and unit testing, with my main primers being the excellent GOOSGBT and a perusal of TDD-tagged questions here on SO.
Occasionally, the process I use creates a "controller" class - generally, a class which is a facade over a fairly complex subsystem where, as the number of features implemented in the subsystem grows, responsibilities are continually driven out into helper classes until the original class has essentially no responsibilities beyond making correct calls to a small set of collaborator classes and shunting the returned information (if any) to its other collaborator classes.
Originally, the tests for the soon-to-be controller classes were written at the level of intention of end-users of the class: "If I make this call, what should be the observable effects that I, as an end-user of the class, actually care about?". But as more and more responsibilities and tests for edge-cases were driven out into helper classes (which are replaced by Test Doubles in the tests for the controller class), these tests began to seem really ... vague and non-specific: they were invariably "happy-path" tests that didn't really seem to get to the heart of the matter. It's hard to explain what I mean, but reading the tests back left me with a kind of "So what? Why did you choose this particular happy-path test over any other? What is the significance? If someone breaks the code, will this test pinpoint the exact reason why the code is now broken?" As time went by, I was more and more strongly inclined to instead write the tests in terms of how the classes' collaborators were supposed to be used together: "the class should call this method on this collaborator, and pass the result to this other collaborator" which gave a much more focussed, descriptive and clearly-motivated set of tests (and generally, the resulting set of tests is small).
This obviously has its flaws: the tests are now strongly coupled to the specifics of the implementation of the controller class, rather than the much more flexible "what would an end-user of this class see that he would care about?". But really, the tests are already quite coupled to it by virtue of the fact that they must configure all of the Test Double collaborators to behave in the exact way required by the implementation to give the correct results from an end-user of the classes' point of view.
So my question is: do fellow TDD'ers find that a minority of classes do little but marshall their (small) set of collaborators around? Do you find keeping the tests for such classes to be written from an end-user of the classes' point of view to be imprecise and unsatisfactory and if so, is it acceptable to write tests for such classes explicitly in terms of how it calls and transfers data between their collaborators?
Hope it's reasonably clear what I'm driving at, here! :)
As a concrete example: one practise project I was working on was a TV listings downloader/ viewer (if you've ever seen "Digiguide", you'll know the kind of thing I mean), and I was implementing a core part of the app - the part that actually updates the listings over the net and integrates the newly downloaded listings into the current set of known TV programs. The interface to this (surprisingly complex when all requirements are taken on board) functionality was a class called ListingsUpdater, which had a method called "updateListings".
Now, end-users of ListingsUpdater only really care about a few things: after listingsUpdate has been called, is the database of TV listings now correct, and were the changes made to the database (adding TV programs, changing them if broadcast changes occurred etc) described to the provided change listeners? When the implementation was a very, very simple "fake it till you make it" type of deal, this worked fine: but as I progressively drove the implementation towards one that would work in the real-world, the "real work" got driven further and further away from ListingsUpdater, until it mainly just marshalled a few collaborators: a ListingsRequestPreparer for assessing the current state of the listings and building HTTP requests for a ListingsDownloader, and a ListingsIntegrator which unpacked the newly downloaded listings and incorporated them (it too delegating to collaborators) into the listings database. Now, note that in order to fulfil the contract of ListingsUpdater from a user's point of view, I must, in the test, instruct its ListingsIntegrator Test Double to populate the (fake) database with the correct data(!), which seems bizarre. It seems much more sensible to drop the "from the end-user of ListingsUpdater's point of view" tests and instead add a test that says "when the ListingsDownloader has downloaded the new listings ensure they are handed over to the ListingsIntegrator".
This obviously has its flaws: the tests are now strongly coupled to the specifics of the implementation of the controller class, rather than the much more flexible "what would an end-user of this class see that he would care about?". But really, the tests are already quite coupled to it by virtue of the fact that they must configure all of the Test Double collaborators to behave in the exact way required by the implementation to give the correct results from an end-user of the classes' point of view.
I'll repeat what I said in answer to another question:
I need to create either a mock a stub or a dummy object [a test double] for each dependency
This is commonly stated. But I think it is wrong. If a Car is associated with an Engine object, why not use a real Engine object when unit testing your Car class?
But, someone will declare, if you do that you are not unit testing your code; your test depends on both the Car class and the Engine class: two units, so an integration test rather than a unit test. But do those people mock the String class too? Or HashSet<String>? Of course not. The line between unit and integration testing is not so clear.
More philosophically, you can not create good mock objects [test doubles] in many cases. The reason is that, for most methods, the manner in which an object delegates to associated objects is undefined. Whether it does delegate, and how, is left by the contract as an implementation detail. The only requirement is that, on delegating, the method satisfies the preconditions of its delegate. In such a situation, only a fully functional (non-mock) delegate will do. If the real object checks its preconditions, failure to satisfy a precondition on delegating will cause a test failure. And debugging that test failure will be easy.
And I'll add in response to
they were invariably "happy-path" tests that didn't really seem to get to the heart of the matter
This is a more general testing problem, not specific to TDD or unit testing: how to you select a good set of test-cases, given that comprehensive testing is impossible? I rely on equivalence partitioning. When I start work on some code, I use equivalence partitioning to select the set of test-cases I want the code to pass, then work on each in turn in a TDD manner, but if passing one of the test-cases does not require a code change (because early work has created code that also satisfies that test case) I still add the test-case to my test-suite. My test suite therefore has better coverage of potential error paths.
Is there a reason why you shouldn't create a partial fake of an object or just fake one method on the object that you are testing of it for the sake of testing another method? This might be helpful to save you from making an entire new mock object, or when there is an external dependency in the method you are faking which you can't reasonably get rid of and would like to keep out of all the other unit tests?
The objects you want to do this for are trying to do too many things. In particular, if you have an external dependency, you would normally create an object to isolate that dependency. The Façade pattern is one example of this. If your objects weren't designed with testability in mind you may have to do some refactoring. Take a look at Michael Feathers' PDF on working with legacy code(PDF). He also has a book by the same title that goes into much more detail.
It is a very bad idea to mock/fake part of a class to test another.
Doing this, you are not testing what the real code does in the conditions under test leading to unreliable test results.
It also increases the maintenance burden of the faked part of the class. If this is in effect for the whole test program, the fake implementation also makes other tests on the faked method harder.
You need to ask yourself why you need to fake out the part under test.
If it is because the method is accessing a file or database, then you should define an interface and pass an instance of that interface to the class constructor or method. This allows you to test different scenarios in the same test application.
If it is because you are using singletons, you should rethink your design to make it more testable: removing singletons will remove implicit dependencies and maintenance nightmares.
If you are using static methods/free-standing functions to access data in a registry or settings file, you should really move that out of the function under test and pass the data as a parameter or provide a settings provider interface. This will make the code more flexible and robust.
If it is to break a dependency for the purpose of testing (e.g. faking out a vector method to test a method in a matrix class) then you should not be faking that -- you should treat the code under test as what is defined by the class under test by its public interface: methods; pre-conditions, post-conditions, invariants, documentation, parameters and exception specifications.
You can use knowledge of the implementation details to test special edge cases, but trigger those through the main API, not by faking an implementation detail.
For example, suppose you faked std::vector::at() but the implementation switched to use operator[] instead. Your test would break or silently pass.
If the method you want to fake is virtual (as in, not static and not final), then you can subclass your object in your test, override the method in the subclass, and exercise the subclass in the test. No mock-object libraries required.
(Ideally you should consider refactoring, this is not a great long-term solution. But it is a way to get legacy code under test so you can start the refactoring process more easily.)
The Extract and Override technique described in Chapter 3 of Roy Osherove's The Art of Unit Testing does seem to be a way to fake part of the class under test (pp. 71-77). Osherove does not address the concerns raised in some of the other answers to this question.
In addition, Michael Feathers discusses this in Working Effectively with Legacy Code. He terms the resulting class a testing subclass (227) and the technique Subclass and Override Method (401). Now, granted, Feathers is not giving an exposition of pristine techniques that are recommended on new code. But he still gives it serious treatment as a potentially helpful technique.
I also asked my former computer professor about this. He is well-read and currently works full-time in the software industry, where he has advanced rapidly. He said that this technique definitely has a good application, and that there are several dozen classes in the codebase at his company that are under test in this way. He said that, like any technique, it can be overused.
I originally wrote the question when I was new to unit testing and knew next to nothing about dependency injection. Now, after some experience with both, I would add that the need to use this testing technique could be a smell. It may be a sign that need to rework your approach to dependencies. If the method that needs to be faked is one that is inherited from a base class, it may mean that you need to take the adage "favor composition over inheritance" more seriously. You should inject your dependencies rather than inheriting them.
There are some really nice packages for facilitating this kind of stuff. For instance, from the Mockito docs:
//You can mock concrete classes, not only interfaces
LinkedList mockedList = mock(LinkedList.class);
//stubbing
when(mockedList.get(0)).thenReturn("first");
does some real magic that's hard to believe at first. When you call
String firstMember = mockedList.get(0);
you'll get back "first", because of what you said in the "when" statement.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
The community reviewed whether to reopen this question 1 year ago and left it closed:
Original close reason(s) were not resolved
Improve this question
In TDD development, the first thing you typically do is to create your interface and then begin writing your unit tests against that interface. As you progress through the TDD process you would end-up creating a class that implements the interface and then at some point your unit test would pass.
Now my question is about the private and protected methods that I might have to write in my class in support of the methods/properties exposed by the interface:
Should the private methods in the class have their own unit tests?
Should the protected methods in the class have their own unit tests?
My thoughts:
Especially because I am coding to interfaces, I shouldn't worry about protected/private methods as they are black boxes.
Because I am using interfaces, I am writing unit tests to validate that the contract defined is properly implemented by the different classes implementing the interface, so again I shouldnt worry about the private/protected methods and they should be exercised via unit tests that call the methods/properties defined by the interface.
If my code-coverage does not show that the protected/private methods are being hit, then I don't have the right unit-tests or I have code thats not being used and should be removed.
No, I don't think of testing private or protected methods. The private and protected methods of a class aren't part of the public interface, so they don't expose public behavior. Generally these methods are created by refactorings you apply after you've made your test turn green.
So these private methods are tested implicitly by the tests that assert the behavior of your public interface.
On a more philosophical note, remember that you're testing behavior, not methods. So if you think of the set of things that the class under test can do, as long as you can test and assert that the class behaves as expected, whether there are private (and protected) methods that are used internally by the class to implement that behavior is irrelevant. Those methods are implementation details of the public behavior.
I disagree with most of the posters.
The most important rule is: WORKING CODE TRUMPS THEORETICAL RULES about public/protected/private.
Your code should be thoroughly tested. If you can get there by writing tests for the public methods, that sufficiently exercise the protected/private methods, that's great.
If you can't, then either refactor so that you can, or bend the protected/private rules.
There's a great story about a psychologist who gave children a test. He gave each child two wooden boards with a rope attached to each end, and asked them to cross a room w/o touching their feet to the floor, as fast as possible. All the kids used the boards like little skis, one foot on each board, holding them by the ropes, and sliding across the floor. Then he gave them the same task, but using only ONE board. They pivoted/"walked" across the floor, one foot on each end of the single board -- and they were FASTER!
Just because Java (or whatever language) has a feature (private/protected/public) does not necessarily mean you are writing better code because you use it!
Now, there will always be ways to optimize/minimize this conflict. In most languages, you can make a method protected (instead of public), and put the test class in the same package (or whatever), and the method will be available for test. There are annotations that can help, as described by other posters. You can use reflection to get at the private methods (yuck).
The context also matters. If you're writing an API for use by external people, public/private is more important. If it's an internal project -- who really cares?
But at the end of the day, think about how many bugs have been caused by lack of testing. Then compare to how many bugs have been caused by "overly visible" methods. That answer should drive your decision.
You wrote:
In TDD development, the first thing
you typically do is to create your
interface and then begin writing your
unit tests against that interface. As
you progress through the TDD process
you would end-up creating a class that
implements the interface and then at
some point your unit test would pass.
Please let me rephrase this in BDD language:
When describing why a class is valuable and how it behaves, the first thing you typically
do is to create an example of how to use the class, often via its interface*. As you add
desired behavior you end up creating a class which provides that value, and then at some
point your example works.
*May be an actual Interface or simply the accessible API of the class, eg: Ruby
doesn't have interfaces.
This is why you don't test private methods - because a test is an example of how to use the class, and you can't actually use them. Something you can do if you want to is delegate the responsibilities in the private methods to a collaborating class, then mock / stub that helper.
With protected methods, you're saying that a class which extends your class should have some particular behavior and provide some value. You could then use extensions of your class to demonstrate that behavior. For instance, if you were writing an ordered collection class, you might want to demonstrate that two extensions with the same contents demonstrated equality.
Hope this helps!
When you're writing the unit tests for your class, you shouldn't necessarily care whether or not the functionality of the class is implemented directly in the method on the public interface or if it is implemented in a series of private methods. So yes, you should be testing your private methods, but you shouldn't need to call them directly from your test code in order to do so (directly testing the private methods tightly couples your implementation to your tests and makes refactoring unnecessarily hard).
Protected methods form a different contract between your class and its future children, so you should really be testing it to a similar extent as your public interface to ensure that the contract is well defined and exercised.
No! Only test interfaces.
One of the big benefits of TDD is assuring that the interface works no matter how you've chosen to implement the private methods.
Completing what others said above, I would say that protected methods are part of an interface of some kind: it simply happens to be the one exposed to inheritance instead of composition, which is what everyone tends to think about when considering interfaces.
Marking a method as protected instead of private means it is expected to be used by third party code, so some sort of contract needs to be defined and tested, as happens with normal interfaces defined by public methods, which are open both for inheritance and composition.
There's two reasons for writing tests:
Asserting expected behavior
Preventing regression of behavior
The take on (1) Asserting expected behavior:
When you're asserting expected behavior, you want to make sure the code works as you think it should. This is effectively an automated way of doing your routine manual verification that any dev would perform when implementing any kind of code:
Did what I just write works?
Does this loop actually end?
Is it looping in the order I think it is?
Would this work for a null input?
Those are questions we all answer in our heads, and normally, we'd try to execute the code in our heads too, make sure it looks like it does work. For these cases, it is often useful to have the computer answer them in a definitive manner. So we write a unit test that assert it does. This gives us confidence in our code, helps us find defects early, and can even help actually implementing the code.
It's a good idea to do this wherever you feel is necessary. Any code that is a little tricky to understand, or is non trivial. Even trivial code could benefit from it. It's all about your own confidence. How often to do it and how far to go will depend on your own satisfaction. Stop when you can confidently answer Yes to: Are you sure this works?
For this kind of testing, you don't care about visibility, interfaces, or any of that, you only care about having working code. So yes, you would test private and protected methods if you felt they needed to be tested for you to answer Yes to the question.
The take on (2) Preventing regression of behavior:
Once you've got working code, you need to have a mechanism in place to protect this code from future damage. If nobody was to ever touch your source and your config ever again, you wouldn't need this, but in most cases, you or others will be touching the source and the configs of your software. This internal fiddling is highly likely to break your working code.
Mechanisms exist in most languages already as a way to protect against this damage. The visibility features are one mechanism. A private method is isolated, and hidden. Encapsulation is another mechanism, where you compartmentalize things, so that changing other compartments doesn't affect others.
The general mechanism for this is called: coding to boundary. By creating boundaries between parts of the code, you protect everything inside a boundary from things outside of it. The boundaries become the point of interaction, and the contract by which things interact.
This means that changes to a boundary, either by breaking it's interface, or breaking it's expected behavior, would damage and possibly break other boundaries which relied on it. That's why it's a good idea to have a unit test, that targets those boundaries and assert they don't change in semantic and in behavior.
This is your typical unit test, the one most everybody talks about when mentioning TDD or BDD. The point is to hardened the boundaries and protect them from change. You do not want to test private methods for this, because a private method is not a boundary. Protected methods are a restricted-boundary, and I would protect them. They aren't exposed to the world, but are still exposed to other compartments or "Units".
What to make of this?
As we've seen, there's a good reason to unit test public and protected methods, as to assert our interfaces don't change. And there's also good reason to test private methods, as to assert our implementation works. So should we unit test them all?
Yes and No.
Firstly: Test all methods that you feel you need a definitive proof that it works in most cases as to be able to be confident your code works, no matter the visibility. Then, disable those tests. They've done there job.
Lastly: Write tests for your boundaries. Have a unit test for each point that will be used by other units of your system. Make sure this test assert the semantic contract, method name, number of arguments, etc. And also make sure the test asserts the available behavior of the unit. Your test should demonstrate how to use the unit, and what the unit can do. Keep these tests enabled so that they run on every code push.
NOTE: The reason you disabled the first set of test is to allow refactoring work to occur. An active test is a code coupling. It prevents future modification of the code it's testing. You only want this for your interfaces and interaction contracts.
No, you shouldn't test private methods (how would you anyway without using something horrible like reflection). With protected methods it is slightly less obvious in C# you can make things protected internal and I think it is OK to do that to test derived classes that implement all of their functionality through template pattern methods.
But, in general, if you think that your public methods are doing too much then it is time to refactor your classes into more atomic classes and then test those clases.
I too agree with #kwbeam's answer about not testing private methods. However, an important point I'd like to highlight - protected methods ARE part of a class' exported API and hence MUST be tested.
Protected methods may not be publicly accessible but you definitely are providing a way for sub-classes to use/override them. Something outside the class can access them and hence you need to ensure that those protected members behave in an expected manner. So don't test private methods, but do test public and protected methods.
If you believe you have a private method which contains critical logic, I'd try to extract it out into a separate object, isolate it and provide a way to test its behavior.
Hope it helps!
If you are aiming high code coverage (I suggest you should), you should test your all methods regardless of they are private or protected.
Protected is a kind of different discussion point, but in summary, it should not be there at all. Either it breaks encapsulation on deployed code, or it forces you to inherit from that class, just to unit test it, even sometimes you do not need to inherit.
Just hiding a method to client (making private) does not give it to have privilege not to be audited. Therefore, they can be tested by public methods as mentioned before.
I agree with everyone else: The answer to your question is 'no'.
Indeed you are entirely correct with your approach and your thoughts, especially about code coverage.
I would also add that the question (and the answer 'no') also applies to public methods that you might introduce to classes.
If you add methods (public/protected or private) because they make a failing test pass, then you've more or less achieved the goal of TDD.
If you add methods (public/protected or private) because you just decide to, violating TDD, then your code coverage should catch these and you should be able to improve your process.
Also, for C++ (and I should think only for C++) I implement interfaces using private methods only, to indicate that the class should only be used via the interface it implements. It stops me mistakenly calling new methods added to my implementation from my tests
A good design means splitting the application into multiple testable units. After doing this, some units are exposed to the public API, but some others may not be. Also, the interaction points between exposed units and these "internal" units are also not a part of the pubic API.
I think once we have the identifiable unit, is would benefit from the unit tests, regardless if exposed via public API or not.
Simple answer - NO.
Explanation :
why should test a private function ? It will automatically be tested anyway (and must be tested) when you test the feature/method which is using it - the private function.
I have a project I am trying to learn unit testing and TDD practices with. I'm finding that I'm getting to quite confusing cases where I am spending a long time setting up mocks for a utility class that's used practically everywhere.
From what I've read about unit testing, if I am testing MyClass, I should be mocking any other functionality (such as provided by UtilityClass). Is it acceptable (assuming that UtilityClass itself has a comprehensive set of tests) to just use the UtilityClass rather than setting up mocks for all the different test cases?
Edit: One of the things I am making a lot of setup for.
I am modelling a map, with different objects in different locations. One of the common methods on my utility class is GetDistanceBetween. I am testing methods that have effects on things depending on their individual properties, so for example a test that selects all objects within 5 units of a point and an age over 3 will need several tests (gets old objects in range, ignores old objects out of range, ignores young objects in range, works correctly with multiples of each case) and all of those tests need setup of the GetDistanceBetween method. Multiply that out by every method that uses GetDistanceBetween (almost every one) and the different results that the method should return in different circumstances, and it gets to be a lot of setup.
I can see as I develop this further, there may be more utility class calls, large numbers of objects and a lot of setup on those mock utility classes.
The rule is not "mock everything" but "make tests simple". Mocking should be used if
You can't create an instance with reasonable effort (read: you need a single method call but to create the instance, you need a working database, a DB connection, and five other classes).
Creation of the additional classes is expensive.
The additional classes return unstable values (like the current time or primary keys from a database)
TDD isn't really about testing. Its main benefit is to help you design clean, easy-to-use code that other people can understand and change. If its main benefit was to test then you would be able to write tests after your code, rather than before, with much of the same effect.
If you can, I recommend you stop thinking of them as "unit tests". Instead, think of your tests as examples of how you can use your code, together with descriptions of its behaviour which show why your code is valuable.
As part of that behaviour, your class may want to use some collaborating classes. You can mock these out.
If your utility classes are a core part of your class's behaviour, and your class has no value or its behaviour makes no sense without them, then don't mock them out.
Aaron Digulla's answer is pretty good; I'd rephrase each of his answers according to these principles as:
The behaviour of the collaborating class is complex and independent of the behaviour of the class you're interested in.
Creation of the collaborating class is not a valuable aspect of your class and does not need to be part of your class's responsibility.
The collaborating class provides context which changes the behaviour of your class, and therefore plays into the examples of how you can use it and what kind of behaviour you might expect.
Hope that makes sense! If you liked it, take a look at BDD which uses this kind of vocabulary far more than "test".
In theory you should try to mock all dependencies, but in reality it's never possible. E.g. you are not going to mock the basic classes from the standard library. In your case if the utility class just contains some basic helper methods I think I wouldn't bother to mock it.
If it's more complicated than that or connects to some external resources, you have to mock it. You could consider creating a dedicated mock builder class, that would create you a standard mock (with some standard stubs defined etc), so that you can avoid mocking code duplication in all test classes.
No, it is not acceptable because you are no longer testing the class in isolation which is one of the most important aspects of a unit test. You are testing it with its dependency to this utility even if the utility has its own set of tests. To simplify the creation of mock objects you could use a mock framework. Here are some popular choices:
Rhino Mocks
Moq
NSubstitute
Of course if this utility class is private and can only be used within the scope of the class under test then you don't need to mock it.
Yes, it is acceptable. What's important is to have the UtilityClass thoroughly unit tested and to be able to differentiate if a test is failing because of the Class under test or because of the UtilityClass.
Testing a class in isolation means testing it in a controlled environment, in an environment where one control how the objects behave.
Having to create too many objects in a test setup is a sign that the environment is getting too large and thus is not controlled enough. Time has come to revert to mock objects.
All the previous answers are very good and really match with my point of view about static utility classes and mocking.
You have two types of utilities classes, your own classes you write and the third party utility classes.
As the purpose of an utility class is to provide small set of helper methods, your utility classes or a third party utility classes should be very well tested.
First Case: the first condition to use your own utility class (even if static) without mocking, is to provide a set of valid unit tests for this class.
Second Case: if you use a third party utility library, you should have enough confidence to this library. Most of the time, those libraries are well tested and well maintained. You can use it without mocking its methods.
I'm still in the learning stages regarding unit-testing and in particular regarding mocking (I'm using the PascalMock and DUnit frameworks). One thing I now stumbled over was that I couldn't find a way around hard-coding implementation details of the tested class/interface into my unit test and that just feels wrong...
For example: I want to test a class that implements a very simple interface for reading and writing application settings (basically name/value pairs). The interface that is presented to the consumer is completely agnostic to where and how the values are actually stored (e.g. registry, INI-file, XML, database, etc.). Naturally, the access layer is implemented by yet a different class that gets injected into the tested class on construction. I created a mock object for this access layer and I am now able to fully test the interface-implementing class without actually reading or writing anything to any registry/INI-file/whatever.
However, in order to ensure the mock behaves exactly like the real thing when accessed by the tested class, my unit tests have to set up the mock object by very explicitly defining expected method calls and the return values expected by the tested class. This means that if I should ever have to make changes to the interface of the access layer or to the way that the tested class uses that layer I will also have to change the unit tests for the class that internally uses that interface even though the interface of the class I'm actually testing hasn't changed at all. Is this something I will just have to live with when using mocks or is there a better way to design the class-dependencies that would avoid this?
to ensure the mock behaves exactly like the real thing when accessed by the tested class, my unit tests have to set up the mock object by very explicitly defining expected method calls and the return values expected by the tested class.
Correct.
changes to the interface of the access layer or to the way that the tested class uses that layer I will also have to change the unit tests
Correct.
even though the interface of the class I'm actually testing hasn't changed at all.
"Actually testing"? You mean the exposed interface class? That's fine.
The way the "tested" (interface) class uses the access layer means you've changed the internal interface to the access layer. Interface changes (even internal ones) require test changes and may lead to breakage if you've done something wrong.
Nothing wrong with this. Indeed, the whole point is that any change to the access layer must require changes to the mocks to assure that the change "works".
Testing is not supposed to be "robust". It's supposed to be brittle. If you make a change that alters internal behavior, then things can break. If your tests were too robust they wouldn't test anything -- they'd just work. And that's wrong.
Tests should only work for the exact right reason.
Is this something I will just have to
live with when using mocks or is there
a better way to design the
class-dependencies that would avoid
this?
A lot of times mocks (particularly sensitive frameworks like JMock) force you to account for details that don't relate directly to the behavior you're trying to test, and sometimes this can even be helpful by exposing suspect code that is doing too much and has too many calls/dependencies.
However in your case, if I read your description right, it sounds like you really don't have a problem. If you design the read/write layer correctly and with an appropriate level of abstraction, you shouldn't have to change it.
This means that if I should ever have
to make changes to the interface of
the access layer or to the way that
the tested class uses that layer I
will also have to change the unit
tests for the class that internally
uses that interface even though the
interface of the class I'm actually
testing hasn't changed at all.
Isn't the point of writing the abstracted access layer to avoid this? In general, following the Open/Closed principle, an interface of this sort shouldn't change and shouldn't break the contract with the class that consumes it, and by extension it won't break your unit tests either. Now if you change the order of the method calls, or have to make new calls to the abstracted layer, then, yes, particularly with some frameworks, your mock expectations will break. This is just part of the cost of using mocks, and it's perfectly acceptable. But the interface itself should, in general, remain stable.
Just to put some names to your example,
RegistryBasedDictionary implements the Role (interface) Dictionary.
RegistryBasedDictionary has a dependency on the Role RegistryAccessor, implemented by RegistryWinAPIWrapper.
You are currently interested in testing RegistryBasedDictionary. The unit tests would inject a mock dependency for the RegistryAccessor Role and would test the expected interaction with the dependencies.
The trick here to avoid unnecessary test-maintenance is to "Specify precisely what should happen.. and no more." (From the GOOS book (must-read for mock flavored TDD), so if order of dependency method calls doesn't matter, don't specify it in the test. That leaves you free to change the order of calls in the implementation.)
Design the Roles such that the they do not contain any leaks from the actual implementations - keep the Roles implementation-agnostic.
The only reason to change RegistryBasedDictionary tests, would be a change in the behavior of RegistryBasedDictionary and not in any of its dependencies. So if its interaction with its dependencies or the roles/contracts change, the tests would need to be updated. That is the price of interaction-based tests you need to pay, for the flexibility to test in isolation. However in practice, it doesn't that happen that often.