We have the following problem: a number of classes that we cannot touch but need to unit test them unfortunately the classes are not designed with unit testing in mind so we issues creating mock objects to test the code.
Example:
class SomeOtherClass
{
public:
void foo2() { … }
};
class ClassToTest
{
public:
ClassToTest() {…}
void foo1() { SomeOtherClass A.foo2(); }
};
In the above example we would like to test foo1() but it needs foo2() so we would like to make foo2() belong to a mock object (in real life these functions/classes are vastly more complex and involve interaction with hardware configurations etc thus the need for mock objects/functions). Until now we have done something like this but it is really not optimal because the code seems to have side effects on other unit tests.
class MockSomeOtherClass
{
public:
foo2() { … } // mock function
};
#define SomeOtherClass MockSomeOtherClass
#include “ClassToTest.cpp”
...
Is there a better way to do this without changing the original classes (or with minimal changes)? We use CPPUnit for testing.
EDIT: added tag winapi to more clearly describe out environment.
There is a product called Typemock Isolator++ that appears to address the issues you have raised. I have not tried it yet, so can't comment on how well it works or how easy/difficult it is to use.
Unfortunately, you have to give them your email address to try it. The download is easy enough, but then you are redirected to this page which cheerfully directs you to "Register your software now to get a FREE trial! Please enter your details including a valid email in order to receive your activation key to start using Isolator++."
Related
I want to use Google test to test my class.
Lets assume I have a state machine implementation and the current state is private
so I have a method SetNextState that looks like that:
void setNextState
{
switch(m_currentState) //m_currentState is a private member
{
case INIT_STATE:
{
if some conditions occurred m_currentState=GO_STATE
}
......
}
}
so I have several cases and each define the behavior to move from certain state to another.
My question:
How do I perform tests on that method assuming the state is relevant only to this class so there is no output
How do I set its value to be, for example "GO_STATE" to test the GO_STATE case
and how do i check the m_currentState at the end of the test
Im trying to avoid putting friends etc. in my UUT code since I want it to be as original as possible
You don't. You do the same thing that your actual program will do, which is provide an input, then examine the result; you say there's no output, but there must be some effect, otherwise the class is pointless!
Failing that, you could make the test a "friend" of the class so that it can inspect its internals, or add an immutable getter for the current state (and who really cares if your class's users get to see that?) but neither option is really in the spirit of the thing.
In my experience, you'll occasionally realise that you're not really unit testing any more but instead functional testing, and Google Test may not be the right tool for that job. If your class is as big as it sounds, that could be the case here. Conversely, you could help yourself by splitting the class into smaller chunks, then unit testing those. Depends what you're going for, really.
Lightness Races in Orbit is correct. However, if sometimes you feel like it's useful to test the private member functions of your class, it often means that your class could be split in multiple smaller pieces.
If you don't think those smaller components are useful to the clients of your library, you can simply hide them in a detail:: namespace and then create unit tests as usual. This will allow you to test the internal behavior of your classes without polluting your public API.
After much considerations I decided to wrap my UUT with a helper which provides set and get to the relevant private members.and use it in the test procedure before calling the tested API
Original code
===============
class UUT //That's the actual class I want to test
{
protected:
int m_protectedMember;
public:
void methodToTest()
{
//Do something with m_protectedMember use its value as input
//and set it as output
}
};
In the tester
==============
class UUTHelper: public UUT
{
public:
int getProtectedMember() { return m_protectedMember; }
void setProtectedMember(int value) { m_protectedMember = value; }
};
The pros:
My test code is very simple and I easily create complicated scenarios .
I test the real code without any "friends" or any other manipulations.
The cons:
As written in the discussion, not the best "good practice", touching private members
Thank you all :)
I often face the problem that mock objects need to be brought in a certain state before the "interesting" part of a test can start.
For example, let's say I want to test the following class:
struct ToTest
{
virtual void onEnable();
virtual void doAction();
};
Therefore, I create the following mock class:
struct Mock : ToTest
{
MOCK_METHOD0(onEnable, void());
MOCK_METHOD0(doAction, void());
};
The first test is that onEnable is called when the system that uses a ToTest object is enabled:
TEST(SomeTest, OnEnable)
{
Mock mock;
// register mock somehow
// interesting part of the test
EXPECT_CALL(mock, onEnable());
EnableSystem();
}
So far, so good. The second test is that doAction is called when the system performs an action and is enabled. Therefore, the system should be enabled before the interesting part of the test can start:
TEST(SomeTest, DoActionWhenEnabled)
{
Mock mock;
// register mock somehow
// initialize system
EnableSystem();
// interesting part of the test
EXPECT_CALL(mock, doAction());
DoSomeAction();
}
This works but gives an annoying warning about an uninteresting call to onEnable. There seem to be two common fixes of this problem:
Using NiceMock<Mock> to suppress all such warnings; and
Add an EXPECT_CALL(mock, onEnable()) statement.
I don't want to use the first method since there might be other uninteresting calls that really should not happen. I also don't like the second method since I already tested (in the first test) that onEnable is called when the system is enabled; hence, I don't want to repeat that expectation in all tests that work on enabled systems.
What I would like to be able to do is say that all mock calls up to a certain point should be completely ignored. In this example, I want expectations to be only checked starting from the "interesting part of the test" comment.
Is there a way to accomplish this using Google Mock?
The annoying thing is that the necessary functions are there: gmock/gmock-spec-builders.h defines Mock::AllowUninterestingCalls and others to control the generation of warnings for a specific mock object. Using these functions, it should be possible to temporarily disable warnings about uninteresting calls.
That catch is, however, that these functions are private. The good thing is that class Mock has some template friends (e.g., NiceMock) that can be abused. So I created the following workaround:
namespace testing
{
// HACK: NiceMock<> is a friend of Mock so we specialize it here to a type that
// is never used to be able to temporarily make a mock nice. If this feature
// would just be supported, we wouldn't need this hack...
template<>
struct NiceMock<void>
{
static void allow(const void* mock)
{
Mock::AllowUninterestingCalls(mock);
}
static void warn(const void* mock)
{
Mock::WarnUninterestingCalls(mock);
}
static void fail(const void* mock)
{
Mock::FailUninterestingCalls(mock);
}
};
typedef NiceMock<void> UninterestingCalls;
}
This lets me access the private functions through the UninterestingCalls typedef.
The flexibility you're looking for is not possible in gmock, by design. From the gmock Cookbook (emphasis mine):
[...] you should be very cautious about when to use naggy or strict mocks, as they tend to make tests more brittle and harder to maintain. When you refactor your code without changing its externally visible behavior, ideally you should't need to update any tests. If your code interacts with a naggy mock, however, you may start to get spammed with warnings as the result of your change. Worse, if your code interacts with a strict mock, your tests may start to fail and you'll be forced to fix them. Our general recommendation is to use nice mocks (not yet the default) most of the time, use naggy mocks (the current default) when developing or debugging tests, and use strict mocks only as the last resort.
Unfortunately, this is an issue that we, and many other developers, have encountered. In his book, Modern C++ Programming with Test-Driven Development, Jeff Langr writes (Chapter 5, on Test Doubles):
What about the test design? We split one test into two when we changed from a hand-rolled mock solution to one using Google Mock. If we expressed everything in a single test, that one test could set up the expectations to cover all three significant events. That’s an easy fix, but we’d end up with a cluttered test.
[...]
By using NiceMock, we take on a small risk. If the code later somehow changes to invoke another method on the [...] interface, our tests aren’t going to know about it. You should use NiceMock when you need it, not habitually. Seek to fix your design if you seem to require it often.
You might be better off using a different mock class for your second test.
class MockOnAction : public ToTest {
// This is a non-mocked function that does nothing
virtual void onEnable() {}
// Mocked function
MOCK_METHOD0(doAction, void());
}
In order for this test to work, you can have onEnable do nothing (as shown above). Or it can do something special like calling the base class or doing some other logic.
virtual void onEnable() {
// You could call the base class version of this function
ToTest::onEnable();
// or hardcode some other logic
// isEnabled = true;
}
When designing functions inside a C++ class, I often keep testing in mind. So when I finish all the functionalists of a class, very often several functions that are only used for testing purpose are added in the class as well. Take the following example:
class SimpleClass
{
public:
int a;
int b;
void fun1();
void fun2();
void testIntermediateResult();
private:
void _fun1();
void _fun2();
};
In this function testIntermediateResult is only needed for testing purpose. Is it a good practice just leave the function inside the class or should I do something like that:
class SimpleClass
{
public:
int a;
int b;
void fun1();
void fun2();
#ifdef TESTING
void testIntermediateResult();
#endif
private:
void _fun1();
void _fun2();
};
The philosophy here is when this class has been finished testing and will be given to the clients then TESTING will be defined and this function will not be inside the class. Therefore, my question is: is really needed to remove the testing function when the class is given to the client? Suppose the client will not use the testing function, what's the effect of adding several redundant testing functions inside a class? Thanks.
I'm assuming that your client is not going to be involved in testing the class that you're handing over. I would not want to add testing functions directly into the class itself as there's a possibility that they could disrupt the behaviour of the class. As for the client, I would prefer not to give them something with a test function in it since:
It's not necessary - it's not their job to test it.
They could decide to try to use it - who knows what'll happen then?
It's just simpler if they don't see it. As for using the pre-processor, this could be fraught with problems particularly if you have attributes that need to be guarded in the same way. If any of them are missed or your macro is redefined in the build process by the client then you could get run-time crashes due to class size mismatches etc.
I'd favour having a one-to-one external class to test your deliverable classes. Something like TestSimpleClass which performs the testing. There are a number of advantages to this approach:
It's entirely separate from your code and not built into it so you aren't bloating code or causing any potential issues with it.
It's going to test your class interface the way your client sees it (i.e. black box testing)
Because it's separate, you don't have to give it to your client - they never have to know about it.
If you really want to test the internals of the class, you can always make your test class a friend of your deliverable class. It's only one line extra in the deliverable class and you still don't have to ship your test classes or libraries.
It known that in C++ mocking/faking nonvirtual methods for testing is hard. For example, cookbook of googlemock has two suggestion - both mean to modify original source code in some way (templating and rewriting as interface).
It appear this is very bad problem for C++ code. How can be done best if you can't modify original code that needs to be faked/mocked? Duplicating whole code/class (with it whole base class hierarchy??)
One way that we sometimes use is to split the original .cpp file into at least two parts.
Then the test apparatus can supply its own implementations; effectively using the linker to do the dirty work for us.
This is called the "Link Seam" in some circles.
I followed the Link Seam link from sdg's answer. There I read about different types of seams, but I was most impressed by Preprocessing Seams. This made me think about exploiting further the preprocessor. It turned out that it is possible to mock any external dependency without actually changing the calling code.
To do this, you have to compile the calling source file with a substitute dependency definition.
Here is an example how to do it.
dependency.h
#ifndef DEPENDENCY_H
#define DEPENDENCY_H
class Dependency
{
public:
//...
int foo();
//...
};
#endif // DEPENDENCY_H
caller.cpp
#include "dependency.h"
int bar(Dependency& dependency)
{
return dependency.foo() * 2;
}
test.cpp
#include <assert.h>
// block original definition
#define DEPENDENCY_H
// substitute definition
class Dependency
{
public:
int foo() { return 21; }
};
// include code under test
#include "caller.cpp"
// the test
void test_bar()
{
Dependency mockDependency;
int r = bar(mockDependency);
assert(r == 42);
}
Notice that the mock does not need to implement complete Dependency, just the minimum (used by caller.cpp) so the test can compile and execute.
This way you can mock non-virtual, static, global functions or almost any dependency without changing the productive code.
Another reason I like this approach is that everything related to the test is in one place. You don't have to tweak compiler and linker configurations here and there.
I have applied this technique successfully on a real world project with big fat dependencies.
I have described it in more detail in Include mock.
Code has to be written to be testable, by whatever test techniques you use. If you want to test using mocks, that means some form of dependency injection.
Non-virtual calls with no dependence on a template parameter pose the same problem as final and static methods in Java[*] - the code under test has explicitly said, "I want to call this code, not some unknown bit of code that's dependent in some way on an argument". You, the tester, want it to call different code under test from what it normally calls. If you can't change the code under test then you, the tester, will lose that argument. You might as well ask how to introduce a test version of line 4 of a 10-line function without changing the code under test.
If the class to be mocked is in a different TU from the class under test, you can write a mock with the same name as the original and link that instead. Whether you can generate that mock using your mocking framework in the normal way, I'm not so sure.
If you like, I suppose it's a "very bad problem for C++" that it's possible to write code that's hard to test. It shares this "problem" with a great number of other languages...
[*] My Java knowledge is quite low-power. There may be some clever way of mocking such methods in Java, which aren't applicable to C++. If so, please disregard them in order to see the analogy ;-)
I think it is not possible to do it with standard C++ right now (but lets hope that a powerful compile-time reflection will come to C++ soon...). However, there are a number of options for doing so.
You might have a look at Injector++. It is Windows only right now, but plans to add support for Linux & Mac.
Another option is CppFreeMock, which seems to work with GCC, but has no recent activities.
HippoMocks also provide such ability, but only for free functions. It doesn't support it for class member functions.
I'm not completely sure, but it seems that all the above achieve this with overwriting the target function at runtime so that it jumps to the faked function.
The there is C-Mock, which is an extension to Google Mock allowing you to mock non-virtual functions by redefining them, and relying on the fact that original functions are in dynamic libraries. It is limited to GNU/Linux platform.
Finally, you might also try PowerFake (for which, I'm the author) as introduced here.
It is not a mocking framework (currently) and it provides the possibility for replacing production functions with test ones. I hope to be able to integrate it to one or more mocking frameworks; if not, it'll become one.
Update: It has an integration with FakeIt.
Update 2: Added support for Google Mock
It also overrides the original function during linking (so, it won't work if a function is called in the same translation unit in which it is defined), but uses a different trick than C-Mock as it uses GNU ld's --wrap option. It also needs some changes to your build system for tests, but doesn't affect the main code in any way (except if you are forced to put a function in a separate .cpp file); but support for easily integrating it into CMake projects is provided.
But, it is currently limited to GCC/GNU ld (works also with MinGW).
Update: It supports GCC & Clang compilers, and GNU ld & LLVM lld linkers (or any compatible linker).
#zaharpopov you can use Typemock IsolatorPP to create mocks of non-virtual class and methods without changing your code (or legacy code).
for example if you have a non-virtual class called MyClass:
class MyClass
{
public:
int GetResult() { return -1; }
}
you can mock it with typemock like so:
MyClass* fakeMyClass = FAKE<MyClass>();
WHEN_CALLED(fakeMyClass->GetResult()).Return(10);
By the way the classes or methods that you want to test can also be private as typemock can mock them too, for example:
class MyClass
{
private:
int PrivateMethod() { return -1; }
}
MyClass* myClass = new MyClass();
PRIVATE_WHEN_CALLED(myClass, PrivateMethod).Return(1);
for more information go here.
You very specifically say "if you can't modify original code", which the techniques you mention in your question (and all the other current "answers") do.
Without changing that source, you can still generally (for common OSes/tools) preload an object that defines its own version of the function(s) you wish to intercept. They can even call the original functions afterwards. I provided an example of doing this in (my) question Good Linux TCP/IP monitoring tools that don't need root access?.
That is easier then you think. Just pass the constructed object to the constructor of the class you are testing. In the class store the reference to that object. Then it is easy to use mock classes.
EDIT :
The object that you are passing to the constructor needs an interface, and that class store just the reference to the interface.
struct Abase
{
virtual ~Abase(){}
virtual void foo() = 0;
};
struct Aimp : public Abase
{
virtual ~Aimp(){}
virtual void foo(){/*do stuff*/}
};
struct B
{
B( Aimp &objA ) : obja( objA )
{
}
void boo()
{
objA.foo();
}
Aimp &obja;
};
int main()
{
//...
Aimp realObjA;
B objB( realObjA );
// ...
}
In the test, you can pass the mock object easy.
I used to create an interface for the parts I needed to mock. Then I simply created a stub class that derived from this interface and passed this instance to my classes under test. Yes, it is a lot of hard work, but I found it worth it for some circumstances.
Oh, by interface I mean a struct with only pure virtual methods. Nothing else!
we have code that persists a file and I would like to check the content of the file during testing.
I though that such scenario will be best implemented if I abstract the file persistence operation to the following interface:
public interface IFilePersist
{
void Save(XXX, FileLocation);
}
In unit testing I will inject the mock that will check the content and in production the interface will be actually persist to right location.
Is it an overhead? Is this practice commonly used?
For DB related operations this kind of operation is trivial and always used.
Yes, that's one way to do it, if you want to reduce overhead of creating a tracking mock object you may also do a local override of the method if possible. I don't know what language you're using, but in Java a local override would look like this:
// actual class
public class SomeClass {
public void method() {
System.out.println("Hello!");
}
}
// creating the class in test
SomeClass c = new SomeClass() {
public void method() {
System.out.println("Hi!");
}
};
Now your class actually prints "Hi!" when m() is called because it's overriden in an anonymous inner class whilst the actual production class still keeps printing "Hello!".
Yes, it's good to keep unit tests isolated from file system. First because file system access is slower than memory access. Then one can run into other problems (permissions, missing path, FS full or not mounted ...).
The real persistence, on the file system, shall be tested with integration tests.