I am a fairly new software developer currently working adding unit tests to an existing C++ project that started years ago. Due to a non-technical reason, I'm not allowed to modify any existing code. The base class of all my modules has a bunch of methods for Setting/Getting data and communicating with other modules.
Since I just want to unit testing each individual module, I want to be able to use canned values for all my inter-module communication methods. I.e. for a method Ping() which checks if another module is active, I want to have it return true or false based on what kind of test I'm doing. I've been looking into Google Test and Google Mock, and it does support mocking non-virtual methods. However the approach described (https://google.github.io/googletest/gmock_cook_book.html#MockingNonVirtualMethods) requires me to "templatize" the original methods to take in either real or mock objects. I can't go and templatize my methods in the base class due to the requirement mentioned earlier, so I need some other way of mocking these virtual methods
Basically, the methods I want to mock are in some base class, the modules I want to unit test and create mocks of are derived classes of that base class. There are intermediate modules in between my base Module class and the modules that I want to test.
I would appreciate any advise!
Thanks,
JW
EDIT: A more concrete examples
My base class is lets say rootModule, the module I want to test is leafModule. There is an intermediate module which inherits from rootModule, leafModule inherits from this intermediate module.
In my leafModule, I want to test the doStuff() method, which calls the non virtual GetStatus(moduleName) defined in the rootModule class. I need to somehow make GetStatus() to return a chosen canned value. Mocking is new to me, so is using mock objects even the right approach?
There are some different ways of replacing non-virtual functions. One is to re-declare them and compile a new test executable for each different set of non-virtual functions you'd like to test. That's hardly scaleable.
A second option is to make them virtual for test. Most compilers allow you to define something on the command-line so compile your code with -DTEST_VIRTUAL=virtual or -DTEST_VIRTUAL to make them either virtual or normal depending on whether or not it's under test or not.
A third option which may be usable is to use a mocking framework that lets you mock non-virtual functions. I'm the author of HippoMocks (disclaimer with regard to neutrality and so on) and we've recently added the ability to mock plain C functions on X86 platforms. This can be extended to non-virtual member functions with a bit of work and would be what you're looking for. Keep in mind that, if your compiler can see both the use and the definition of a function at one time that it may inline it and that the mocking may fail. That holds in particular for functions that are defined in headers.
If regular C function mocking is sufficient for you, you can use it as it is now.
I would write a Perl/Ruby/Python script to read in the original source tree and write out a mocked source tree in a different directory. You don't have to fully parse C++ in order to replace a function definition.
One approach would be to specify different sources for testing. Say your production target uses rootModule.h and rootModule.cpp. Use different sources for your testing target. You can specify a different header by changing your include path, so that #include "rootModule.h" actually loads unittest/rootModule.h. Then mock rootModule to your heart's content.
Related
I am reviewing some code where the developer has some classes ClassA and ClassB. They both inherit from ParentClass so both must implement a number of abstract methods (e.g. return5Values(2))
In ClassA the values are all double the previous value: [2,4,8,16,32]
In ClassB the values are all +1 the previous value [2,3,4,5,6]
There are also other constraints such as raising an error if the parameter is negative etc.
Other tests like getting the 3rd value only, also exist etc.
(Obviously these are just fake examples to get my point across)
Now, instead of writing a lot of similar tests for both ClassA and ClassB, what the developer has done is created ParentClassChildTests which contains a some code something like this:
public void testVariablesAreCorrect() {
returnedValues = clazz.return5Values(2)
# Does a bunch of other things as well
# ...
assertEqual(expectedValues, returnedValues)
}
ClassATests now inherits from ParentClassChildTest and must define expectedValues as a class variable.
The expectedValues are used within a few different tests as well, so they aren't being defined just for this single test.
Now when ClassATests and ClassBTests are run, it also runs all the tests inside ParentClassChildTests.
My question is: Is this a good method to avoid a lot of duplicate tests and ensure everything works as expected in child classes? Are there any major issues this can lead to? Or a better way of handling this?
Whilst this is all Java code, my question isn't about any particular testing framework or language but the idea in general of inheriting from a parent class which also has tests in it.
The situation that it is possible and sensible to re-use tests for different implementations of an interface / base class is not very common. The following aspects limit the applicability:
Derived classes have different dependencies, which may require different mocks to be created and to be set up. In such a case, the test methods can not be identical. Even if classA and classB currently do not have dependencies or the same dependencies with (coincidentially) the same setup, this can change over time or the next class classC will have different dependencies.
Each derived class will implement different algorithms. In your case, return5Values performs different algorithms in ClassA and ClassB. Due to the different algorithms, the behaviour of the SUT for the same set up and the same inputs may be different: For example, each algorithm will run into overflows at different points. Even the call return5Values(2) that allows to use a derived test for classA and classB today, could with a potential future classC lead to an overflow scenario with possible exceptions thrown.
The different algorithms implemented in the derived classes will have different potential bugs and different corner cases. That is, the necessary set up and inputs for the SUT will have to be different to stimulate the respective boundaries. For some implementations, testing the call return5Values(2) may simply not bring any benefit while test for other parameters than 2 are necessary.
If you share test methods between the classes and only provide the parameters, it is not the test method which is associated with the tests' intent - each parameter set has its own intent. The intent/scenario, however, should ideally be part of the output of each individual test.
Given all these problems, inheritance of test methods does not seem to be the best approach for re-use here. Instead, it may be more beneficial to have have some common helper functions that can be used by the different derived classes.
Having class hierarchies in tests creates dependencies between them. A UnitTest serves the purpose of testing a Unit in isolation where Unit refers to a certain class. I'd argue that it is ok to have helpers and utils to avoid duplicating very basic functionality.
As much as possible unit tests should allow for quick and independent changes of a certain Unit. Having a commonly enforced structure for all tests increases the amount of work to be done if the implementation of unrelated parts of the application changes.
When it comes to integration testing there will be shared functionality for setting up the infrastructure. So the answer is a very clear it depends. Generally it is favorable to reduce dependencies between tests as much as possible and having a base test that determines the inner workings of a derived test is detrimental to that goal.
There is an emphasis on using interfaces instead of concrete types in order to make the code easier to test. I wonder though why this wasn't done for the types in the sql package like DB or Rows. In order to mock those dependencies I had to create my own interfaces so that I could write unit tests (not integration tests). Aren't DB facing code supposed to be tested that way?
Exposing interfaces in your public API instead of concrete types increases the risk of breaking other peoples code when you add methods to the interface.
See for example os.File. If os.File was an interface it would be an interface with 17 public methods. Adding an 18th method would break everyone who defined their own types that implemented the os.File interface. In contrast, adding an 18th method to the current os.File struct won't break any methods taking an io.Reader, io.Writer or any other interface that defines a subset of the methods of an os.File. It also won't break test code which mocks these io.Reader and io.Writer interfaces.
So expose an interface in your public API if you want other people to define their own implementations of them. Otherwise expose a concrete type and let people define their own interfaces implemented by your concrete type using only the subset of methods they need.
I have a testing dilemma:
I'm writing a unit test for Unit A. The method I'm about to test is func(B param). In the (non testing) code the only place where the func(..) is called is in class C, it's also the only place in the project where variables of type B can be instanciated (to be sent as parameters to the func(B param), so the instanciation method is private.
Now, I'm not sure how should I create a B instance inside the Unit test.
Of cause I can change private to public in a method declaration of the method which creates B, but I don't feel right about it, since it'll expose the method to the rest of the project.
I can also simply duplicate the B creation method into the Unit test class, but I hate duplicating code.
Is there some known best practice for such cases?
Thanks a lot,
Dima
You would be looking into using a mocking framework like EasyMock or Mokito to create a "test double" object of your class B here.
As a rule of thumb, only public methods should be tested directly. By extension, public code should test all private code. Depending on what language you're using, you can also force less exposed code to be exposed to unit testing frameworks without having to change every method (e.g. InternalsVisibleTo in C#) but I accept that this option won't be available to all languages using all frameworks.
As well as unit testing, another tool in your armoury should be some sort of code coverage tool to ensure that as much of your code is being covered by said tests as possible.
I am using Google Mock 1.6 RC and am trying to Mock a COM Interface. There are close to 50 methods in the COM Interface some of which are inherited from base interfaces. When I create a mock struct that inherits from this interface and mock only the methods I am using, I get the cannot instantiate abstract class error.
I want to know if it is possible to do this in googlemock or not.
It is not possible to do. You have to overload all pure virtual methods from all interfaces (except for the constructor and destructor).
You have to override every method that has been declared as pure virtual in the classes you inherit from, directly or indirectly. There are two reasons not to want override them all:
There are too many of them and you have something better to do with your time than to go over them all.
Compiling a mock class with all of them mocked out is too slow and takes too much memory.
The fix for (1) is to use the gmock_gen.py script in Google Mock's scripts directory. It goes over the class definition and converts method declarations into the MOCK_METHOD statements. If you have problems with (2), you can replace the unnecessary MOCK_METHOD statements with stubs:
MOCK_METHOD1(f, bool(int i));
with
virtual bool f(int i) {
thrown std::exception("The stub for f(int) has been invoked unexpectedly.");
}
Throwing an exception will alert you to a situation where a particular stub has been invoked, meaning you likely need to mock it instead.
Edit: If the original interfaces to mock are written using Microsoft's macros, this thread has a script posted that converts them to C++ acceptable to gmock_gen.py.
I'm not entirely sure whether all methods should be covered in the mock class... In the gmock examples you can see that for example destructors are not mocked. Therefore I presume there is no need to mock the entire class.
Anyway, shouldn't you create mock class rather than mock struct?
However, there is a gmock_gen.py tool in scripts/generator that should do the hard work of mocking large classes for you.
I have a code base where many of the classes I implement derive from classes that are provided by other divisions of my company. Working with these other devisions often have the working relationship as though they are third party middle ware vendors.
I'm trying to write test code without modifying these base classes. However, there are issues with creating meaningful test
objects due to the lack of interfaces:
//ACommonClass.h
#include "globalthermonuclearwar.h" //which contains deep #include dependencies...
#include "tictactoe.h" //...and need to exist at compile time to get into test...
class Something //which may or may not inherit from another class similar to this...
{
public:
virtual void fxn1(void); //which often calls into many other classes, similar to this
//...
int data1; //will be the only thing I can test against, but is often meaningless without fxn1 implemented
//...
};
I'd normally extract an interface and work from there, but as these are "Third Party", I can't commit these changes.
Currently, I've created a separate file that holds fake implementations for functions that are defined in the third-party supplied base class headers on a need to know basis, as has been described in the book "Working with Legacy Code".
My plan was to continue to use these definitions and provide alternative test implementations for each third party class that I needed:
//SomethingRequiredImplementations.cpp
#include "ACommonClass.h"
void CGlobalThermoNuclearWar::Simulate(void) {}; // fake this and all other required functions...
// fake implementations for otherwise undefined functions in globalthermonuclearwar.h's #include files...
void Something::fxn1(void) { data1 = blah(); } //test specific functionality.
But before I start doing that I was wondering if any one has tried providing actual objects on a code base similar to mine, which would allow creating new test specific classes to use in place of actual third-party classes.
Note all code bases in question are written in C++.
Mock objects are suitable for this kind of task. They allow you to simulate the existence of other components without needing them to be present. You simply define the expected input and output in your tests.
Google have a good mocking framework for C++.
I'm running into a very similar problem at the moment. I don't want to add a bunch of interfaces that are only there for the purpose of testing, so I can't use any of the existing mock object libraries. To get around this I do the same thing, creating a different file with fake implementations, and having my tests link the fake behaviour, and production code links the real behaviour.
What I wish I could do at this point, is take the internals of another mock framework, and use it inside my fake objects. It would look a little something like this:
Production.h
class ConcreteProductionClass { // regular everyday class
protected:
ConcreteProductionClass(); // I've found the 0 arg constructor useful
public:
void regularFunction(); // regular function that I want to mock
}
Mock.h
class MockProductionClass
: public ConcreteProductionClass
, public ClassThatLetsMeSetExpectations
{
friend class ConcreteProductionClass;
MockTypes membersNeededToSetExpectations;
public:
MockClass() : ConcreteProductionClass() {}
}
ConcreteProductionClass::regularFunction() {
membersNeededToSetExpectations.PassOrFailTheTest();
}
ProductionCode.cpp
void doSomething(ConcreteProductionClass c) {
c.regularFunction();
}
Test.cpp
TEST(myTest) {
MockProductionClass m;
m.SetExpectationsAndReturnValues();
doSomething(m);
ASSERT(m.verify());
}
The most painful part of all this is that the other mock frameworks are so close to this, but don't do it exactly, and the macros are so convoluted that it's not trivial to adapt them. I've begun looking into this on my spare time, but it's not moving along very quickly. Even if I got my method working the way I want, and had the expectation setting code in place, this method still has a couple drawbacks, one of them being that your build commands can get to be kind of long if you have to link against a lot of .o files rather than one .a, but that's manageable. It's also impossible to fall through to the default implementation, since we're not linking it. Anyway, I know this doesn't answer the question, or really even tell you anything you don't already know, but it shows how close the C++ community is to being able to mock classes that don't have a pure virtual interface.
You might want to consider mocking instead of faking as a potential solution. In some cases you may need to write wrapper classes that are mockable if the original classes aren't. I've done this with framework classes in C#/.Net, but not C++ so YMMV.
If I have a class that I need under test that derives from something I can't (or don't want to) run under test I'll:
Make a new logic-only class.
Move the code-i-wanna-test to the logic class.
Use an interface to talk back to the real class to interact with the base class and/or things I can't or won't put in the logic.
Define a test class using that same interface. This test class could have nothing but noops or fancy code that simulates the real classes.
If I have a class that I just need to use in testing, but using the real class is a problem (dependencies or unwanted behaviors):
I'll define a new interface that looks like all of the public methods I need to call.
I'll create a mock version of the object that supports that interface for testing.
I'll create another class that is constructed with a "real" version of that class. It also supports that interface. All interface calls a forwarded to the real object methods.
I'll only do this for methods I actually call - not ALL the public methods. I'll add to these classes as I write more tests.
For example, I wrap MFC's GDI classes like this to test Windows GDI drawing code. Templates can make some of this easier - but we often end up not doing that for various technical reasons (stuff with Windows DLL class exporting...).
I'm sure all this is in Feather's Working with Legacy Code book - and what I'm describing has actual terms. Just don't make me pull the book off the shelf...
One thing you did not indicate in your question is the reason why your classes derive from base classes from the other division. Is the relationship really a IS-A relationshiop ?
Unless your classes needs to be used by a framework, you could consider favoring delegation over inheritance. Then you can use dependency injection to provide your class with a mock of their class in the unit tests.
Otherwise, an idea would be to write a script to extract and create the interface your need from the header they provide, and integrate this to the compilation process so your unit test can ve checked in.