I've used the self-shunt unit testing pattern a few times over the years. As I was explaining it to someone recently they argued that it violated the SRP. The argument is that the test class can now be changed for one of two reasons: when the test changes, or when the method signature on the interface that the test is implementing changes. After thinking about it for a while, it seems like this is a correct assessment, but I wanted to get other peoples' opinions. Thoughts?
Reference:
http://www.objectmentor.com/resources/articles/SelfShunPtrn.pdf
My take on this is that the test class technically violates the SRP, but it doesn't violate the spirit of SRP. The alternative to self-shunting is to have a mock class separate from the test class.
With the separate mock class you might think that it's all self contained and satisfies the SRP, however the semantic coupling to the mock class's attributes is still there. So, really, we didn't achieve any meaningful separation.
Taking the example out of the PDF:
public class ScannerTest extends TestCase implements Display
{
public ScannerTest (String name) {
super (name);
}
public void testScan () {
// pass self as a display
Scanner scanner = new Scanner (this);
// scan calls displayItem on its display
scanner.scan ();
assertEquals (new Item (“Cornflakes”), lastItem);
}
// impl. of Display.displayItem ()
void displayItem (Item item) {
lastItem = item;
}
private Item lastItem;
}
Now we make a Mock:
public class DisplayMock implements Display
{
// impl. of Display.displayItem ()
void displayItem (Item item) {
lastItem = item;
}
public Item getItem() {
return lastItem;
}
private Item lastItem;
}
public class ScannerTest extends TestCase
{
public ScannerTest (String name) {
super (name);
}
public void testScan () {
// pass self as a display
DisplayMock dispMock = new DisplayMock();
Scanner scanner = new Scanner (dispMock );
// scan calls displayItem on its display
scanner.scan ();
assertEquals (new Item (“Cornflakes”), dispMock.GetItem());
}
}
In practical terms (IMHO) the higher coupling of TestClass to DisplayMock is a greater evil than the violation of the SRP for TestClass. Besides, with the use mocking frameworks, this problem goes away completely.
EDIT I have just encountered a brief mention of self-shunt pattern in Robert C. Martin's excellent book Agile Principles, Patterns, and Practices in C#. Here is the snippet out of the book:
We can accomplish this by using an abstract interface for the database. One implementation of this abstract interface uses the real database. Another implementation is test code written to simulate the behavior of the database and to check that the database calls are being made correctly. Figure 29-5 shows the structure. The PayrollTest module tests the PayrollModule by making calls to it and also implements the Database interface so that it can trap the calls that Payroll makes to the database. This allows PayrollTest to ensure that Payroll is behaving properly. It also allows PayrollTest to simulate many kinds of database failures and problems that are otherwise difficult to create. This is a testing pattern known as SELF-SHUNT, also sometimes known as mocking or spoofing.
So, the guy who coined the SRP (which is talked about in great detail in the same book) has no qualms using self-shunt pattern. In light of that, I'd say you are pretty safe from the OOP (Objected Orientated Police) when using this pattern.
In my opinion it is a violation, but a very minor one.
You test class is now a test class and a dependency for whatever you are testing.
However, is that a bad thing? For a couple of simple tests, probably not. As your number of test cases grows, you'll probably want to refactor and use a mock class to separate some of the concerns. (As the link you pasted says, self-shunt is a stepping stone to mocking). But if the number of test cases remains static and low, what's the problem?
I think a little pragmatism is required. Does it violate the SRP? Yes, but I'm guessing probably not as much as some of the code in the system you're testing. Do you need to do anything about it? No, not as long as the code is clear and maintainable, which for me is always the bottom line. SRP is a guideline, not a rule.
If the interface being implemented or shunted is changed, it's relatively likely that the test suite will have to change as well. So I don't really see it as a violation of SRP.
I prefer to have a little more control over the mocks/stubs that I'm creating. When I tried using the self shunt pattern I ended up making my test class more complex. By creating the mocks as locals within a test method I ended up having cleaner code.
FWIW unless you're using something powerful like C# (or python or an equivalent) your test code will change when you change an interface.
Related
I want to use Google test to test my class.
Lets assume I have a state machine implementation and the current state is private
so I have a method SetNextState that looks like that:
void setNextState
{
switch(m_currentState) //m_currentState is a private member
{
case INIT_STATE:
{
if some conditions occurred m_currentState=GO_STATE
}
......
}
}
so I have several cases and each define the behavior to move from certain state to another.
My question:
How do I perform tests on that method assuming the state is relevant only to this class so there is no output
How do I set its value to be, for example "GO_STATE" to test the GO_STATE case
and how do i check the m_currentState at the end of the test
Im trying to avoid putting friends etc. in my UUT code since I want it to be as original as possible
You don't. You do the same thing that your actual program will do, which is provide an input, then examine the result; you say there's no output, but there must be some effect, otherwise the class is pointless!
Failing that, you could make the test a "friend" of the class so that it can inspect its internals, or add an immutable getter for the current state (and who really cares if your class's users get to see that?) but neither option is really in the spirit of the thing.
In my experience, you'll occasionally realise that you're not really unit testing any more but instead functional testing, and Google Test may not be the right tool for that job. If your class is as big as it sounds, that could be the case here. Conversely, you could help yourself by splitting the class into smaller chunks, then unit testing those. Depends what you're going for, really.
Lightness Races in Orbit is correct. However, if sometimes you feel like it's useful to test the private member functions of your class, it often means that your class could be split in multiple smaller pieces.
If you don't think those smaller components are useful to the clients of your library, you can simply hide them in a detail:: namespace and then create unit tests as usual. This will allow you to test the internal behavior of your classes without polluting your public API.
After much considerations I decided to wrap my UUT with a helper which provides set and get to the relevant private members.and use it in the test procedure before calling the tested API
Original code
===============
class UUT //That's the actual class I want to test
{
protected:
int m_protectedMember;
public:
void methodToTest()
{
//Do something with m_protectedMember use its value as input
//and set it as output
}
};
In the tester
==============
class UUTHelper: public UUT
{
public:
int getProtectedMember() { return m_protectedMember; }
void setProtectedMember(int value) { m_protectedMember = value; }
};
The pros:
My test code is very simple and I easily create complicated scenarios .
I test the real code without any "friends" or any other manipulations.
The cons:
As written in the discussion, not the best "good practice", touching private members
Thank you all :)
Consider you have the following method:
public Foo ParseMe(string filepath)
{
// break up filename
// validate filename & extension
// retrieve info from file if it's a certain type
// some other general things you could do, etc
var myInfo = GetFooInfo(filename);
// create new object based on this data returned AND data in this method
}
Currently I have unit tests for GetFooInfo, but I think I also need to build unit tests for ParseMe. In a situation like this where you have a two methods that return two different properties - and a change in either of them could break something - should unit tests be created for both to determine the output is as expected?
I like to err on the side of caution and be more wary about things breaking and ensuring that maintenance later on down the road is easier, but I feel very skeptical about adding very similar tests in the test project. Would this be bad practice or is there any way to do this more efficiently?
I'm marking this as language agnostic, but just in case it matters I am using C# and NUnit - Also, I saw a post similar to this in title only, but the question is different. Sorry if this has already been asked.
ParseMe looks sufficiently non-trivial to require a unit test. To answer your precise question, if "you have a two methods that return two different properties - and a change in either of them could break something" you should absolutely unit test them.
Even if the bulk of the work is in GetFooInfo, at minimum you should test that it's actually called. I know nothing about NUnit, but I know in other frameworks (like RSpec) you can write tests like GetFooInfo.should be_called(:once).
It is not a bad practice to test a method that is calling another method. In fact, it is a good practice. If you have a method calling another method, it is probably performing additional functionality, which should be tested.
If you find yourself unit testing a method that calls a method that is also being unit tested, then you are probably experiencing code reuse, which is a good thing.
I agree with #tsm - absolutely test both methods (assuming both are public).
This may be a smell that the method or class is doing too much - violating the Single Responsibility Principle. Consider doing an Extract Class refactoring and decoupling the two classes (possibly with Dependency Injection). That way you could test both pieces of functionality independently. (That said, I'd only do that if the functionality was sufficiently complex to warrant it. It's a judgment call.)
Here's an example in C#:
public interface IFooFileInfoProvider
{
FooInfo GetFooInfo(string filename);
}
public class Parser
{
private readonly IFooFileInfoProvider _fooFileInfoProvider;
public Parser(IFooFileInfoProvider fooFileInfoProvider)
{
// Add a null check
_fooFileInfoProvider = fooFileInfoProvider;
}
public Foo ParseMe(string filepath)
{
string filename = Path.GetFileName(filepath);
var myInfo = _fooFileInfoProvider.GetFooInfo(filename);
return new Foo(myInfo);
}
}
public class FooFileInfoProvider : IFooFileInfoProvider
{
public FooInfo GetFooInfo(string filename)
{
// Do I/O
return new FooInfo(); // parameters...
}
}
Many developers, me included, take a programming by contract approach. That requires you to consider each method as a black box. If the method delegates to another method to accomplish its task does not matter, when you are testing the method. But you should also test all large or complicated parts of your program as units. So whether you need to unit test the GetFooInfo depends on how complicated that method is.
My colleagues and I are currently introducing unit tests to our legacy Java EE5 codebase. We use mostly JUnit and Mockito. In the process of writing tests, we have noticed that several methods in our EJBs were hard to test because they did a lot of things at once.
I'm fairly new to the whole testing business, and so I'm looking for insight in how to better structure the code or the tests. My goal is to write good tests without a headache.
This is an example of one of our methods and its logical steps in a service that manages a message queue:
consumeMessages
acknowledgePreviouslyDownloadedMessages
getNewUnreadMessages
addExtraMessages (depending on somewhat complex conditions)
markMessagesAsDownloaded
serializeMessageObjects
The top-level method is currently exposed in the interface, while all sub-methods are private. As far as I understand it, it would be bad practice to just start testing private methods, as only the public interface should matter.
My first reaction was to just make all the sub-methods public and test them in isolation, then in the top-level method just make sure that it calls the sub-methods. But then a colleague mentioned that it might not be a good idea to expose all those low-level methods at the same level as the other one, as it might cause confusion and other developers might start using when they should be using the top-level one. I can't fault his argument.
So here I am.
How do you reconcile exposing easily testable low-level methods versus avoiding to clutter the interfaces? In our case, the EJB interfaces.
I've read in other unit test questions that one should use dependency injection or follow the single responsibility principle, but I'm having trouble applying it in practice. Would anyone have pointers on how to apply that kind of pattern to the example method above?
Would you recommend other general OO patterns or Java EE patterns?
At first glance, I would say that we probably need to introduce a new class, which would 1) expose public methods that can be unit tested but 2) not be exposed in the public interface of your API.
As an example, let's imagine that you are designing an API for a car. To implement the API, you will need an engine (with complex behavior). You want to fully test your engine, but you don't want to expose details to the clients of the car API (all I know about my car is how to push the start button and how to switch the radio channel).
In that case, what I would do is something like that:
public class Engine {
public void doActionOnEngine() {}
public void doOtherActionOnEngine() {}
}
public class Car {
private Engine engine;
// the setter is used for dependency injection
public void setEngine(Engine engine) {
this.engine = engine;
}
// notice that there is no getter for engine
public void doActionOnCar() {
engine.doActionOnEngine();
}
public void doOtherActionOnCar() {
engine.doActionOnEngine();
engine.doOtherActionOnEngine(),
}
}
For the people using the Car API, there is no way to access the engine directly, so there is no risk to do harm. On the other hand, it is possible to fully unit test the engine.
Dependency Injection (DI) and Single Responsibility Principle (SRP) are highly related.
SRP is basicly stating that each class should only do one thing and delegate all other matters to separate classes. For instance, your serializeMessageObjects method should be extracted into its own class -- let's call it MessageObjectSerializer.
DI means injecting (passing) the MessageObjectSerializer object as an argument to your MessageQueue object -- either in the constructor or in the call to the consumeMessages method. You can use DI frameworks to do this for, but I recommend to do it manually, to get the concept.
Now, if you create an interface for the MessageObjectSerializer, you can pass that to the MessageQueue, and then you get the full value of the pattern, as you can create mocks/stubs for easy testing. Suddenly, consumeMessages doesn't have to pay attention to how serializeMessageObjects behaves.
Below, I have tried to illustrate the pattern. Note, that when you want to test consumeMessages, you don't have to use the the MessageObjectSerializer object. You can make a mock or stub, that does exactly what you want it to do, and pass it instead of the concrete class. This really makes testing so much easier. Please, forgive syntax errors. I did not have access to Visual Studio, so it is written in a text editor.
// THE MAIN CLASS
public class MyMessageQueue()
{
IMessageObjectSerializer _serializer;
//Constructor that takes the gets the serialization logic injected
public MyMessageQueue(IMessageObjectSerializer serializer)
{
_serializer = serializer;
//Also a lot of other injection
}
//Your main method. Now it calls an external object to serialize
public void consumeMessages()
{
//Do all the other stuff
_serializer.serializeMessageObjects()
}
}
//THE SERIALIZER CLASS
Public class MessageObjectSerializer : IMessageObjectSerializer
{
public List<MessageObject> serializeMessageObjects()
{
//DO THE SERILIZATION LOGIC HERE
}
}
//THE INTERFACE FOR THE SERIALIZER
Public interface MessageObjectSerializer
{
List<MessageObject> serializeMessageObjects();
}
EDIT: Sorry, my example is in C#. I hope you can use it anyway :-)
Well, as you have noticed, it's very hard to unit test a concrete, high-level program. You have also identified the two most common issues:
Usually the program is configured to use specific resources, such as a specific file, IP address, hostname etc. To counter this, you need to refactor the program to use dependency injection. This is usually done by adding parameters to the constructor that replace the ahrdcoded values.
It's also very hard to test large classes and methods. This is usually due to the combinatorical explosion in the number of tests required to test a complex piece of logic. To counter this, you will usually refactor first to get lots more (but shorter) methods, then trying to make the code more generic and testable by extracting several classes from your original class that each have a single entry method (public) and several utility methods (private). This is essentially the single responsibility principle.
Now you can start working your way "up" by testing the new classes. This will be a lot easier, as the combinatoricals are much easier to handle at this point.
At some point along the way you will probably find that you can simplify your code greatly by using these design patterns: Command, Composite, Adaptor, Factory, Builder and Facade. These are the most common patterns that cut down on clutter.
Some parts of the old program will probably be largely untestable, either because they are just too crufty, or because it's not worth the trouble. Here you can settle for a simple test that just checks that the output from known input has not changed. Essentially a regression test.
I'm writing program arguments parser, just to get better in TDD, and I stuck with the following problem. Say I have my parser defined as follows:
class ArgumentsParser {
public ArgumentsParser(ArgumentsConfiguration configuration) {
this.configuration = configuration;
}
public void parse(String[] programArguments) {
// all the stuff for parsing
}
}
and I imagine to have ArgumentsConfiguration implementation like:
class ArgumentsConfiguration {
private Map<String, Class> map = new HashMap<String, Class>();
public void addArgument(String argName, Class valueClass) {
map.add(argName, valueClass);
}
// get configured arguments methods etc.
}
This is my current stage. For now in test I use:
#Test
public void shouldResultWithOneAvailableArgument() {
ArgumentsConfiguration config = prepareSampleConfiguration();
config.addArgument("mode", Integer.class);
ArgumentsParser parser = new ArgumentsParser(configuration);
parser.parse();
// ....
}
My question is if such way is correct? I mean, is it ok to use real ArgumentsConfiguration in tests? Or should I mock it out? Default (current) implementation is quite simple (just wrapped Map), but I imagine it can be more complicated like fetching configuration from kind of datasource. Then it'd be natural to mock such "expensive" behaviour. But what is preferred way here?
EDIT:
Maybe more clearly: should I mock ArgumentsConfiguration even without writing any implementation (just define its public methods), use mock for testing and deal with real implementation(s) later, or should I use the simplest one in tests, and let them cover this implementation indirectly. But if so, what about testing another Configuration implementation provided later?
Then it'd be natural to mock such "expensive" behaviour.
That's not the point. You're not mocking complex classes.
You're mocking to isolate classes completely.
Complete isolation assures that the tests demonstrate that classes follow their interface and don't have hidden implementation quirks.
Also, complete isolation makes debugging a failed test much, much easier. It's either the test, the class under test or the mocked objects. Ideally, the test and mocks are so simple they don't need any debugging, leaving just the class under test.
The correct answer is that you should mock anything that you're not trying to test directly (e.g.: any dependencies that the object under test has that do not pertain directly to the specific test case).
In this case, because your ArgumentsConfiguration is so simple, I'd recommend using the real implementation until your requirements demand something more complicated. There doesn't seem to be any logic in your ArgumentsConfiguration class, so it's safe to use the real object. If the time comes where the configuration is more complicated, then an approach you should probably take would be not to create a configuration that talks to some data source, but instead generate the ArgumentsConfiguration object from that datasource. Then you could have a test that makes sure it generates the configuration properly from the datasource and you don't need unnecessary abstractions.
Consider the following class:
public class MyIntSet
{
private List<int> _list = new List<int>();
public void Add(int num)
{
if (!_list.Contains(num))
_list.Add(num);
}
public bool Contains(int num)
{
return _list.Contains(num);
}
}
Following the "only test one thing" principle, suppose I want to test the "Add" function.
Consider the following possibility for such a test:
[TestClass]
public class MyIntSetTests
{
[TestMethod]
public void Add_AddOneNumber_SetContainsAddedNumber()
{
MyIntSet set = new MyIntSet();
int num = 0;
set.Add(num);
Assert.IsTrue(set.Contains(num));
}
}
My problem with this solution is that it actually tests 2 methods: Add() and Contains().
Theoretically, there could be a bug in both, that only manifests in scenarios where they are not called one after the other. Of course, Contains() now servers as a thin wrapper for List's Contains() which shouldn't be tested in itself, but what if it changes to something more complex in the future? Perhaps a simple "thin wrap" method should always be kept for testing purposes ?
An alternative approach might suggest mocking out or exposing (possibly using InternalsVisibleTo or PrivateObject) the private _list member and have the test inspect it directly, but that could potentially create test maintainability problems if someday the internal list is replaced by some other collection (maybe C5).
Is there a better way to do this?
Are any of my arguments against the above implementations flawed?
Thanks in advance,
JC
Your test seems perfectly OK to me. You may have misunderstood a principle of unit testing.
A single test should (ideally) only test one thing, that is true, but that does not mean that it should test only one method; rather it should only test one behaviour (an invariant, adherence to a certain business rule, etc.) .
Your test tests the behaviour "if you add to a new set, it is no longer empty", which is a single behaviour :-).
To address your other points:
Theoretically, there could be a bug in both, that only manifests in scenarios where they are not called one after the other.
True, but that just means you need more tests :-). For example, add two numbers, then call Contains, or call Contains without Add.
An alternative approach might suggest mocking out or exposing (possibly using InternalsVisibleTo) the private _list member and have the test inspect it directly, but that could potentially create test maintainability problems[...]
Very true, so don't do this. A unit test should always be against the public interface of the unit under test. That's why it's called a unit test, and not a "messing around inside a unit"-test ;-).
There are two possibilities.
You've exposed a flaw in your design. You should carefully consider if the actions that your Add method is executing is clear to the consumer. If you don't want people adding duplicates to the list, why even have a Contains() method? The user is going to be confused when it's not added to the list and no error is thrown. Even worse, they might duplicate the functionality by writing the exact same code before they call .Add() on their list collection. Perhaps it should be removed, and replaced with an indexer? It's not clear from your list class that it's not meant to hold duplicates.
The design is fine, and your public methods should rely on each other. This is normal, and there is no reason you can't test both methods. The more test cases you have, theoretically the better.
As an example, say you have a functions that just calls down into other layers, which may already be unit tested. That doesn't mean you don't write unit tests for the function even if it's simply a wrapper.
In practice, your current test is fine. For something this simple it's very unlikely that bugs in add() and contains() would mutually conspire to hide each other. In cases where you are really concerned about testing add() and add() alone, one solution is to make your _list variable available to your unit test code.
[TestClass]
public void Add_AddOneNumber_SetContainsAddedNumber() {
MyIntSet set = new MyIntSet();
set.add(0);
Assert.IsTrue(set._list.Contains(0));
}
Doing this has two drawbacks. One: it requires access to the private _list variable, which is a little complex in C# (I recommend the reflection technique). Two: it makes your test code dependent on the actual implementation of your Set implementation, which means you'll have to modify the test if you ever change the implementation. I'd never do this for something as simple as a collections class, but in some cases it may be useful.