To be able to unit test my C++ code I usually pass the constructor of the class under test one or several objects that can be either "production code" or fake/mock objects (let's call these injection objects). I have done this either by
Creating an interface that both the "production code" class and the fake/mock class inherits.
Making the class under test a template class that takes the types of the injection objects as template parameters, and instances of the injection objects as parameters to the constructor.
Some random thoughts:
Until we have concepts (C++0x), only documentation and parameter naming will hint what to provide the class under test (when using templates).
It is not always possible to create interfaces for legacy code
The interface is basically only created to be able to do dependency injection
In the same way: templating the class under test is done only to enable dependency injection
What are your thoughts? Are there other solutions to this problem?
With C++, there's another option - you give your mock classes exact same names as the real classes, and when linking your unit tests, just link them with mock object/library files instead of real ones.
I think interface option is better, but one doesn't have to create common base class just for test. You can inherit your mock class from production class and override necessary methods. You'll have to make the methods virtual though, but that's how tools like mockpp work and they also allow automate this process a little bit.
Templates will have slightly less performance penalties for runtime (less indirections, less calls, more inline optimizations), but will make you suffer a very high penalty for compilation times...
I think that for this purpose, interfaces are better (until we have concepts in C++0x TR1)... unless if you can't slow down some "bottleneck code". Interfaces are more dynamic and switchable at run-time.
Remember that you can construct your class with default injection objects (the real ones), but you can have factories that inject the mock ones on your tests... you don't even need to subclass.
Don't know if it helps, but you can have template constructors:
struct Class_Under_Test
{
template <typename Injected>
Class_Under_Test()
{
...
// and even specialize them
template <>
Class_Under_Test <A_Specific_Injection_Class>
{
...
Only the one that is actually used will get included.
Related
I am reviewing some code where the developer has some classes ClassA and ClassB. They both inherit from ParentClass so both must implement a number of abstract methods (e.g. return5Values(2))
In ClassA the values are all double the previous value: [2,4,8,16,32]
In ClassB the values are all +1 the previous value [2,3,4,5,6]
There are also other constraints such as raising an error if the parameter is negative etc.
Other tests like getting the 3rd value only, also exist etc.
(Obviously these are just fake examples to get my point across)
Now, instead of writing a lot of similar tests for both ClassA and ClassB, what the developer has done is created ParentClassChildTests which contains a some code something like this:
public void testVariablesAreCorrect() {
returnedValues = clazz.return5Values(2)
# Does a bunch of other things as well
# ...
assertEqual(expectedValues, returnedValues)
}
ClassATests now inherits from ParentClassChildTest and must define expectedValues as a class variable.
The expectedValues are used within a few different tests as well, so they aren't being defined just for this single test.
Now when ClassATests and ClassBTests are run, it also runs all the tests inside ParentClassChildTests.
My question is: Is this a good method to avoid a lot of duplicate tests and ensure everything works as expected in child classes? Are there any major issues this can lead to? Or a better way of handling this?
Whilst this is all Java code, my question isn't about any particular testing framework or language but the idea in general of inheriting from a parent class which also has tests in it.
The situation that it is possible and sensible to re-use tests for different implementations of an interface / base class is not very common. The following aspects limit the applicability:
Derived classes have different dependencies, which may require different mocks to be created and to be set up. In such a case, the test methods can not be identical. Even if classA and classB currently do not have dependencies or the same dependencies with (coincidentially) the same setup, this can change over time or the next class classC will have different dependencies.
Each derived class will implement different algorithms. In your case, return5Values performs different algorithms in ClassA and ClassB. Due to the different algorithms, the behaviour of the SUT for the same set up and the same inputs may be different: For example, each algorithm will run into overflows at different points. Even the call return5Values(2) that allows to use a derived test for classA and classB today, could with a potential future classC lead to an overflow scenario with possible exceptions thrown.
The different algorithms implemented in the derived classes will have different potential bugs and different corner cases. That is, the necessary set up and inputs for the SUT will have to be different to stimulate the respective boundaries. For some implementations, testing the call return5Values(2) may simply not bring any benefit while test for other parameters than 2 are necessary.
If you share test methods between the classes and only provide the parameters, it is not the test method which is associated with the tests' intent - each parameter set has its own intent. The intent/scenario, however, should ideally be part of the output of each individual test.
Given all these problems, inheritance of test methods does not seem to be the best approach for re-use here. Instead, it may be more beneficial to have have some common helper functions that can be used by the different derived classes.
Having class hierarchies in tests creates dependencies between them. A UnitTest serves the purpose of testing a Unit in isolation where Unit refers to a certain class. I'd argue that it is ok to have helpers and utils to avoid duplicating very basic functionality.
As much as possible unit tests should allow for quick and independent changes of a certain Unit. Having a commonly enforced structure for all tests increases the amount of work to be done if the implementation of unrelated parts of the application changes.
When it comes to integration testing there will be shared functionality for setting up the infrastructure. So the answer is a very clear it depends. Generally it is favorable to reduce dependencies between tests as much as possible and having a base test that determines the inner workings of a derived test is detrimental to that goal.
Many websites on unit testing say to extract an interface and code to the interface (which makes sense), but that requires using polymorphism via pointers. Is it possible to accomplish this without pointers so I don't have to modify the production code? I would rather not use pointers and manage memory.
Conditional compilation is allowed.
I am specifically using gmock for my stubs/mocks.
Some things that I've researched are:
Using references
involves writing special copy constructors or making it non-copyable
still have to manage memory with new/delete
not sure if this will cause unforseen problems down the line
Creating via code-generation a collection of pointer-wrapper classes. The interface stays the same with a few added methods for testing.
seems like it would work but require upkeep
example of what I mean down below
Please note that gmock mock objects are not copyable, therefore I cannot constructor inject them. (https://groups.google.com/forum/#!topic/googlemock/GD73UXjQowE/discussion)
Problem Example
class Example
{
public:
Example();
~Example();
private:
// I want to stub out _foo.
Dependency _foo;
};
Pointer Wrapper Class Example
#ifndef UNIT_TEST
Foo _foo;
#else
PtrWrapFoo _foo;
#endif
...
_foo.setImpl(StubFoo *aStubFoo);
...
void PtrWrapFoo::doSomething()
{
_impl->doSomething();
}
In the past I have implemented Dependency in a separate compilation unit and linked against that instead of the original.
This is what Michael Feathers calls a Link Seam.
After careful consideration, I have decided to abandon the idea. It becomes too difficult to try and manage the circumstances in which the dependency should use a particular implementation (e.g. real, mock, fake), given the testing scenario.
All dependencies that need testing now have interfaces, which are memberless. My production code uses pointers for dependencies, which is a a reality I have to live with if I want testable code. I was persuaded to this notion after reading Roy Osherove's book, The Art of Unit Testing. My regular constructors instantiate the real, concrete class. I also have extra constructors/setters that are conditionally compiled for unit tests so I can properly set up dependencies by using stubs/mocks.
I have reduced my need to write extra code by using a tool to extract an interface from a class.
Overall, the new design is good and adequately sidesteps the problem of mocking non-pointer member variables with minimal overhead.
I'm trying to get dependency inversion, or at least understand how to apply it, but the problem I have at the moment is how to deal with dependencies that are pervasive. The classic example of this is trace logging, but in my application I have many services that most if not all code will depend on (trace logging, string manipulation, user message logging etc).
None of the solutions to this would appear to be particularly palatable:
Using constructor dependency injection would mean that most of the constructors would have several, many, standard injected dependencies because most classes explicitly require those dependencies (they are not just passing them down to objects that they construct).
Service locator pattern just drives the dependencies underground, removing them from the constructor but hiding them so that it's not even explicit that the dependencies are required
Singleton services are, well, Singletons, and also serve to hide the dependencies
Lumping all those common services together into a single CommonServices interface and injecting that aswell a) violates the Law of Demeter and b) is really just another name for a Service Locator, albeit a specific rather than a generic one.
Does anyone have any other suggestions for how to structure these kinds of dependencies, or indeed any experience of any of the above solutions?
Note that I don't have a particular DI framework in mind, in fact we're programming in C++ and would be doing any injection manually (if indeed dependencies are injected).
Service locator pattern just drives the dependencies underground,
Singleton services are, well, Singletons, and also serve to hide the
dependencies
This is a good observation. Hiding the dependencies doesn't remove them. Instead you should address the number of dependencies a class needs.
Using constructor dependency injection would mean that most of the
constructors would have several, many, standard injected dependencies
because most classes explicitly require those dependencies
If this is the case, you are probably violating the Single Responsibility Principle. In other words, those classes are probably too big and do too much. Since you are talking about logging and tracing, you should ask yourself if you aren't logging too much. But in general, logging and tracing are cross-cutting concerns and you should not have to add them to many classes in the system. If you correctly apply the SOLID principles, this problem goes away (as explained here).
The Dependency Inversion principle is part of the SOLID Principles and is an important principle for among other things, to promote testability and reuse of the higher-level algorithm.
Background:
As indicated on Uncle Bob's web page, Dependency Inversion is about depend on abstractions, not on concretions.
In practice, what happens is that some places where your class instantiates another class directly, need to be changed such that the implementation of the inner class can be specified by the caller.
For instance, if I have a Model class, I should not hard code it to use a specific database class. If I do that, I cannot use the Model class to use a different database implementation. This might be useful if you have a different database provider, or you may want to replace the database provider with a fake database for testing purposes.
Rather than the Model doing a "new" on the Database class, it will simply use an IDatabase interface that the Database class implements. The Model never refers to a concrete Database class. But then who instantiates the Database class? One solution is Constructor Injection (part of Dependency Injection). For this example, the Model class is given a new constructor that takes an IDatabase instance which it is to use, rather than instantiate one itself.
This solves the original problem of the Model no longer references the concrete Database class and uses the database through the IDatabase abstraction. But it introduces the problem mentioned in the Question, which is that it goes against Law of Demeter. That is, in this case, the caller of Model now has to know about IDatabase, when previously it did not. The Model is now exposing to its clients some detail about how it gets its job done.
Even if you were okay with this, there's another issue that seems to confuse a lot of people, including some trainers. There's as an assumption that any time a class, such as Model, instantiates another class concretely, then it's breaking the Dependency Inversion principle and therefore it is bad. But in practice, you can't follow these types of hard-and-fast rules. There are times when you need to use concrete classes. For instance, if you're going to throw an exception you have to "new it up" (eg. threw new BadArgumentException(...)). Or use classes from the base system such as strings, dictionaries, etc.
There's no simple rule that works in all cases. You have to understand what it is that you're trying to accomplish. If you're after testability, then the fact that the Model classes references the Database class directly is not itself a problem. The problem is the fact that the Model class has no other means of using another Database class. You solve this problem by implementing the Model class such that it uses IDatabase, and allows a client to specify an IDatabase implementation. If one is not specified by the client, the Model can then use a concrete implementation.
This is similar to the design of the many libraries, including C++ Standard Library. For instance, looking at the declaration std::set container:
template < class T, // set::key_type/value_type
class Compare = less<T>, // set::key_compare/value_compare
class Alloc = allocator<T> > // set::allocator_type
> class set;
You can see that it allows you to specify a comparer and an allocator, but most of the time, you take the default, especially the allocator. The STL has many such facets, especially in the IO library where detailed aspects of streaming can be augmented for localization, endianness, locales, etc.
In addition to testability, this allows the reuse of the higher-level algorithm with entirely different implementation of the classes that the algorithm internally uses.
And finally, back to the assertion I made previously with regard to scenarios where you would not want to invert the dependency. That is, there are times when you need to instantiate a concrete class, such as when instantiating the exception class, BadArgumentException. But, if you're after testability, you can also make the argument that you do, in fact, want to invert dependency of this as well. You may want to design the Model class such that all instantiations of exceptions are delegated to a class and invoked through an abstract interface. That way, code that tests the Model class can provide its own exception class whose usage the test can then monitor.
I've had colleagues give me examples where they abstract instantiation of even system calls, such as "getsystemtime" simply so they can test daylight savings and time-zone scenarios through their unit-testing.
Follow the YAGNI principle -- don't add abstractions simply because you think you might need it. If you're practicing test-first development, the right abstractions becomes apparent and only just enough abstraction is implemented to pass the test.
class Base {
public:
void doX() {
doA();
doB();
}
virtual void doA() {/*does A*/}
virtual void doB() {/*does B*/}
};
class LoggedBase public : Base {
public:
LoggedBase(Logger& logger) : l(logger) {}
virtual void doA() {l.log("start A"); Base::doA(); l.log("Stop A");}
virtual void doB() {l.log("start B"); Base::doB(); l.log("Stop B");}
private:
Logger& l;
};
Now you can create the LoggedBase using an abstract factory that knows about the logger. Nobody else has to know about the logger, nor do they need to know about LoggedBase.
class BaseFactory {
public:
virtual Base& makeBase() = 0;
};
class BaseFactoryImp public : BaseFactory {
public:
BaseFactoryImp(Logger& logger) : l(logger) {}
virtual Base& makeBase() {return *(new LoggedBase(l));}
};
The factory implementation is held in a global variable:
BaseFactory* baseFactory;
And is initialized to an instance of BaseFactoryImp by 'main' or some function close to main. Only that function knows about BaseFactoryImp and LoggedBase. Everyone else is blissfully ignorant of them all.
I like the things that templates methods can do for me, since they can be used to dramatically simplify some code I'd otherwise have to write over and over.
However, I have a very hard time testing any classes that depend on collaborators that have public template methods. The real problem is they can't be virtual, so my usual method of subclassing dependencies and stubbing doesn't work.
Presently, my solution is to create a non-templated interface that includes every type that's supported, then use the templates as an implementation detail. This is testable, but is tedious to update for large numbers of types passing through many classes. I suppose I could also redesign to use inheritance-based polymorphism instead, but if my problems with testing could be solved without a system redesign I'd prefer that.
I thought there might be something I could do with using template specialization to change the behaviour in my test library vs. my production library, but I'm not sure if that's effective... or even possible.
So, what's the best way of testing classes that use templated methods in their dependencies?
Google mock framework provides a way to create a mock with template parameters. Then you have to add a templated base class and pass a mock object to the constructor of the class under test.
I wanted to investigate this further, so I took some time to try out the template specialization solution I was thinking of. It's possible to do this, but still a real pain. The big restriction on template specialization is that it must be done before any reference is made to the template. Basically, this means you have to do your mocking for any types used in tests in the same header file as the production code. That's bad enough for simple types, but could be tricky or impossible for classes in other libraries, due to the need to have the type defined where it's specialized.
Therefore, I don't think template specialization is a feasible way to mock out a function for unit tests. Making the class under test a template and using Google mock to generate the mock version seems to be the only reasonable solution short of defining a non-template interface.
I have a code base where many of the classes I implement derive from classes that are provided by other divisions of my company. Working with these other devisions often have the working relationship as though they are third party middle ware vendors.
I'm trying to write test code without modifying these base classes. However, there are issues with creating meaningful test
objects due to the lack of interfaces:
//ACommonClass.h
#include "globalthermonuclearwar.h" //which contains deep #include dependencies...
#include "tictactoe.h" //...and need to exist at compile time to get into test...
class Something //which may or may not inherit from another class similar to this...
{
public:
virtual void fxn1(void); //which often calls into many other classes, similar to this
//...
int data1; //will be the only thing I can test against, but is often meaningless without fxn1 implemented
//...
};
I'd normally extract an interface and work from there, but as these are "Third Party", I can't commit these changes.
Currently, I've created a separate file that holds fake implementations for functions that are defined in the third-party supplied base class headers on a need to know basis, as has been described in the book "Working with Legacy Code".
My plan was to continue to use these definitions and provide alternative test implementations for each third party class that I needed:
//SomethingRequiredImplementations.cpp
#include "ACommonClass.h"
void CGlobalThermoNuclearWar::Simulate(void) {}; // fake this and all other required functions...
// fake implementations for otherwise undefined functions in globalthermonuclearwar.h's #include files...
void Something::fxn1(void) { data1 = blah(); } //test specific functionality.
But before I start doing that I was wondering if any one has tried providing actual objects on a code base similar to mine, which would allow creating new test specific classes to use in place of actual third-party classes.
Note all code bases in question are written in C++.
Mock objects are suitable for this kind of task. They allow you to simulate the existence of other components without needing them to be present. You simply define the expected input and output in your tests.
Google have a good mocking framework for C++.
I'm running into a very similar problem at the moment. I don't want to add a bunch of interfaces that are only there for the purpose of testing, so I can't use any of the existing mock object libraries. To get around this I do the same thing, creating a different file with fake implementations, and having my tests link the fake behaviour, and production code links the real behaviour.
What I wish I could do at this point, is take the internals of another mock framework, and use it inside my fake objects. It would look a little something like this:
Production.h
class ConcreteProductionClass { // regular everyday class
protected:
ConcreteProductionClass(); // I've found the 0 arg constructor useful
public:
void regularFunction(); // regular function that I want to mock
}
Mock.h
class MockProductionClass
: public ConcreteProductionClass
, public ClassThatLetsMeSetExpectations
{
friend class ConcreteProductionClass;
MockTypes membersNeededToSetExpectations;
public:
MockClass() : ConcreteProductionClass() {}
}
ConcreteProductionClass::regularFunction() {
membersNeededToSetExpectations.PassOrFailTheTest();
}
ProductionCode.cpp
void doSomething(ConcreteProductionClass c) {
c.regularFunction();
}
Test.cpp
TEST(myTest) {
MockProductionClass m;
m.SetExpectationsAndReturnValues();
doSomething(m);
ASSERT(m.verify());
}
The most painful part of all this is that the other mock frameworks are so close to this, but don't do it exactly, and the macros are so convoluted that it's not trivial to adapt them. I've begun looking into this on my spare time, but it's not moving along very quickly. Even if I got my method working the way I want, and had the expectation setting code in place, this method still has a couple drawbacks, one of them being that your build commands can get to be kind of long if you have to link against a lot of .o files rather than one .a, but that's manageable. It's also impossible to fall through to the default implementation, since we're not linking it. Anyway, I know this doesn't answer the question, or really even tell you anything you don't already know, but it shows how close the C++ community is to being able to mock classes that don't have a pure virtual interface.
You might want to consider mocking instead of faking as a potential solution. In some cases you may need to write wrapper classes that are mockable if the original classes aren't. I've done this with framework classes in C#/.Net, but not C++ so YMMV.
If I have a class that I need under test that derives from something I can't (or don't want to) run under test I'll:
Make a new logic-only class.
Move the code-i-wanna-test to the logic class.
Use an interface to talk back to the real class to interact with the base class and/or things I can't or won't put in the logic.
Define a test class using that same interface. This test class could have nothing but noops or fancy code that simulates the real classes.
If I have a class that I just need to use in testing, but using the real class is a problem (dependencies or unwanted behaviors):
I'll define a new interface that looks like all of the public methods I need to call.
I'll create a mock version of the object that supports that interface for testing.
I'll create another class that is constructed with a "real" version of that class. It also supports that interface. All interface calls a forwarded to the real object methods.
I'll only do this for methods I actually call - not ALL the public methods. I'll add to these classes as I write more tests.
For example, I wrap MFC's GDI classes like this to test Windows GDI drawing code. Templates can make some of this easier - but we often end up not doing that for various technical reasons (stuff with Windows DLL class exporting...).
I'm sure all this is in Feather's Working with Legacy Code book - and what I'm describing has actual terms. Just don't make me pull the book off the shelf...
One thing you did not indicate in your question is the reason why your classes derive from base classes from the other division. Is the relationship really a IS-A relationshiop ?
Unless your classes needs to be used by a framework, you could consider favoring delegation over inheritance. Then you can use dependency injection to provide your class with a mock of their class in the unit tests.
Otherwise, an idea would be to write a script to extract and create the interface your need from the header they provide, and integrate this to the compilation process so your unit test can ve checked in.