I'm writing unit tests for my class and would like to exercise methods that are private. I'm doing something like this:
class MyClass {
Myclass() {}
~MyClass() {}
#ifdef TESTING
public:
#else
private:
#endif
void MyMethod1();
void MyMethod2();
};
If TESTING is defined, I change the class signature to make everything public. In my test code, I just do something like this:
#define TESTING
#include "MyClass.h"
void MyTestMethod()
{
MyClass mc;
mc.MyMethod1(); // Now I can access MyMethod1
}
So that its only public in my test files and not anywhere else. My testing executable sees a header that describes a class that is all public. The code for the class is actually being built somewhere else (where TESTING is not defined), so the scope will differ when the testing project links against the library.
Does this have the potential to break anything? I'm worried that this could change vtables locations or compiler and linker expectations if the header differs from what was actually produced in the object file.
You risk ending up with low quality tests. Tests that are testing some implementation details not relevant to the actual function of the classes. Testing the public interface only drives you towards a good design and test cases that can stand code refactors without breaking.
Ugly, although it's possible but don't do it that way instead create a wrapper class for your tests and use the friend TestClass; keyword in MyClass.
Related
When designing functions inside a C++ class, I often keep testing in mind. So when I finish all the functionalists of a class, very often several functions that are only used for testing purpose are added in the class as well. Take the following example:
class SimpleClass
{
public:
int a;
int b;
void fun1();
void fun2();
void testIntermediateResult();
private:
void _fun1();
void _fun2();
};
In this function testIntermediateResult is only needed for testing purpose. Is it a good practice just leave the function inside the class or should I do something like that:
class SimpleClass
{
public:
int a;
int b;
void fun1();
void fun2();
#ifdef TESTING
void testIntermediateResult();
#endif
private:
void _fun1();
void _fun2();
};
The philosophy here is when this class has been finished testing and will be given to the clients then TESTING will be defined and this function will not be inside the class. Therefore, my question is: is really needed to remove the testing function when the class is given to the client? Suppose the client will not use the testing function, what's the effect of adding several redundant testing functions inside a class? Thanks.
I'm assuming that your client is not going to be involved in testing the class that you're handing over. I would not want to add testing functions directly into the class itself as there's a possibility that they could disrupt the behaviour of the class. As for the client, I would prefer not to give them something with a test function in it since:
It's not necessary - it's not their job to test it.
They could decide to try to use it - who knows what'll happen then?
It's just simpler if they don't see it. As for using the pre-processor, this could be fraught with problems particularly if you have attributes that need to be guarded in the same way. If any of them are missed or your macro is redefined in the build process by the client then you could get run-time crashes due to class size mismatches etc.
I'd favour having a one-to-one external class to test your deliverable classes. Something like TestSimpleClass which performs the testing. There are a number of advantages to this approach:
It's entirely separate from your code and not built into it so you aren't bloating code or causing any potential issues with it.
It's going to test your class interface the way your client sees it (i.e. black box testing)
Because it's separate, you don't have to give it to your client - they never have to know about it.
If you really want to test the internals of the class, you can always make your test class a friend of your deliverable class. It's only one line extra in the deliverable class and you still don't have to ship your test classes or libraries.
In C++ how would one test a class constructor? For example:
Class myClass{
int a;
public:
myClass(); //Constructor
void reset(); //resets a to 0;
};
//Constructor defined to initialize a to 0
myClass::myClass(){
reset();
}
//reset sets a to 0
void myClass::reset(){
a = 0;
}
How would I test the myClass() constructor? I can't access the private data member, so would it be valid to say if reset works then the constructor works?
I am going to rephrase your question to: how do I test properties of my class that are not directly observable from my tests because they are protected or private? Here are several possibilities in the order that I usually think about them:
Move the checking part of the test into the class by using an assert(). If what you want to test is that your constructor has successfully established some complex invariant, then you can put an assert() at the end of the constructor that verifies that invariant. Then all your test has to do is to call the constructor - the assert inside the constructor does the actual checking that the test would otherwise have done. If you habitually add asserts as you write the code then that assert will already be there (if asserts make your debug build too slow, use a profiler to pinpoint those asserts that are actually a problem).
Extract the complex piece of code that you want to test into a second class. You can then test the second class through its public interface. This option is attractive when extracting the second class improves the code even when not considering testability.
Change the design of your code to something more testable in some other way. Again, this is attractive when the change is an improvement even when not considering testability.
Strategic retreat. Testing code in isolation is good all other things being equal, but there is no law saying that all code must be tested in isolation. Sometimes the right answer is to just let that code be tested as a side-effect of testing something else. All your other tests for that class will call the constructor, after all, so if something is wrong with the constructor it will probably cause one of those other tests to fail. It may be harder to cover all paths this way and at the time that you get that failure it will be more trouble to track down the cause of the failure because now you do not know if the problem is the constructor or the other thing being tested. That may or may not be a big deal. Is it truly worth the trouble to test this code in isolation, taking into consideration what else you could be using that time on and the probability that this test will ever actually find a bug? Would a code review be a more useful way to spend that time? Is there something else that it would be more useful to do?
If you truly must test something that cannot be tested with assert() and where there is no acceptable refactoring to improve testability, then there are still some options. I've never had to use these for testing, but here they are:
Use a friend declaration to give your tests access to private fields on your class. Each class now needs to maintain a list of friend declarations for tests that need access. This can be a chore to maintain.
Maintain a separate interface for use by tests. Have a set of methods that give the access that you need and whose name has a prefix like "testOnly". So in your case it would be testOnlyGetA(). You will have to be disciplined in only calling these methods from tests and you now need to maintain a larger interface than before.
Hold your nose and do #define private public in your test file before including the header that you want to test.
Upd. Sorry, misunderstood.
You may add a friend testing function to a class or a class-member testing function (which is better to test implementation details, but sometimes is impossible due to testing framework restrictions)
You could add a boost test case to the class:
Class myClass{
private:
int a;
public:
myClass(); //Constructor
void reset(); //resets a to 0;
void test_a_reset() {
BOOST_CHECK_EQUAL(a, 0);
}
};
Im not sure about how you are doing the testing though. This may not be what you want.
You can declare a test class with the same memory layout of your original class, but all members being public.
void testMyClassConstructor()
{
class myClassPublic
{
public:
int a;
myClassPublic();
void reset();
};
typedef union
{
myClass* original;
myClassPublic* testView;
} u_myClass;
myClass testInstance;
u_myClass testUnion;
testUnion.original = &testInstance;
assert(testUnion.testView->a == 0);
}
This is fine in a white box test case, if you can't change the tested class' interface.
However, remember that any changes to the memory layout of the original class will break the unit test aswell.
I'm writing unit tests for my app, and now I've stumbled on a class in which I should test private methods. This could be result of poor design of particular class, but I must do it. Is there any way in Qt to call private methods, maybe using QMetaObject or something similar ?
For unit testing I am using QTestLib framework.
The proper (read annoying) answer is that you should not be testing private methods, those are implementation details ;-).
OTOH -- have you thought about conditionally declaring them protected/private depending on whether you are in testing or no and then extending? I've used that to get me out of a similar pinch in the past.
#ifdef TESTING
// or maybe even public!
#define ACCESS protected
#else
#define ACCESS private
#endif
/* your class here */
class Foo {
ACCESS
int fooeyness;
}
// or better yet, place this in a different file!
#ifdef TESTING
/*
class which extends the tested class which has public accessors
*/
#endif
Solution 1, quick and easy: make the test class(es) a friend of the public one.
class Foo {
// ...
private:
friend class FooTest;
};
This way your FooTest class can access all members of the public class. However, this way you need to modify the original class every time you want to access private data from a different test, and you leak information about the tests in the public API, and you possibly open up for class naming conflicts (what if there's /another/ FooTest class around?), and so on.
Solution 2, aka properly done: don't put private methods in the public class, but make a private class with public methods.
class Foo {
//
private:
friend class FooPrivate;
FooPrivate *d;
};
FooPrivate gets declared in its own header, which may not be installed, or stay in a include-privates/ subdirectory, or whatever -- i.e. it stays out of the way for normal usage. The public class stays clean this way.
class FooPrivate {
public:
// only public stuff in here;
// and especially this:
static FooPrivate *get(Foo *f) { return f->d; }
};
The test then includes the private header and calls FooPrivate::get(fooObj) to get the private class instance and then happily uses it.
I disagree with the "private members are implementation details" mentality, in my mind that roughly translates to "test only part of your code".
I can related to "units are units" argument, but why not try to cover as much of your code with tests as possible, even inside units? Aka. giving your units a thorough rectal examination.
And with this image in mind, one approach that I have been using frequently that is not mentioned in the other answers is to do the following:
Always declare members you want to test as protected in stead of private in your code.
Subclass your class in the testcode and simply make the selectors you need or simply write the test-code directly as members in that sub-class's implementation.
NOTE: You have to be careful with classes that rely on complex instantiation patterns to ensure that you construct them correctly (read: call the original class' constructor from the subclass ctor).
I have found more convenient way to do this. First, all private methods should be private slots.
Then you create an instance of the class:
Foo a;
Then we can use QMetaObject::invokeMethod to call any slot that method has (public or private). So if we want to call method Test, we can do it like this:
QMetaObject::invokeMethod(&a, "Test", Qt::DirectConnection);
Also, we can have return value and send arguments ... Actually, everything is answered here: http://doc.qt.nokia.com/stable/qmetaobject.html#invokeMethod
I've once read that Unit Test should test the class public interface, and not the protected/private stuff.
Your class should just behave right from outside. If implementation strategy changes, your Unit Test class is still the same.
Suppose I have classes
class Inner {
public:
void doSomething();
};
class Outer {
public:
Outer(Inner *inner); // Dependency injection.
void callInner();
};
Proper unit-testing says I should have tests for Inner. Then, I should have tests for Outer that uses not a real Inner but rather a MockInner so that I would be performing unit-tests on the functionality added by just Outer instead of the full stack Outer/Inner.
To do so, Googletest seems to suggest turning Inner into a pure abstract class (interface) like this:
// Introduced merely for the sake of unit-testing.
struct InnerInterface {
void doSomething() = 0;
};
// Used in production.
class Inner : public InnerInterface {
public:
/* override */ void doSomething();
};
// Used in unit-tests.
class MockInner : public InnerInterface {
public:
/* override */ void doSomething();
};
class Outer {
public:
Outer(Inner *inner); // Dependency injection.
void callInner();
};
So, in production code, I would use Outer(new Inner); while in test, Outer(new MockInner).
OK. Seems nice in theory, but when I started using this idea throughout the code, I find myself creating a pure abstract class for every freaking class. It's a lot of boiler-plate typing, even if you can ignore the slight run-time performance degradable due to the unnecessary virtual dispatch.
An alternative approach is to use templates as in the following:
class Inner {
public:
void doSomething();
};
class MockInner {
public:
void doSomething();
};
template<class I>
class Outer {
public:
Outer(I *inner);
void callInner();
};
// In production, use
Outer<Inner> obj;
// In test, use
Outer<MockInner> test_obj;
This avoids the boiler-plating and the unnecessary virtual dispatch; but now my entire codebase is in the freaking header files, which makes it impossible to hide source implementations (not to mention dealing with frustrating template compilation errors and the long build time).
Are those two methods, virtuals and templates, the only ways to do proper unit-testing? Are there better ways to do proper unit-testing?
By proper unit-testing, I mean each unit-test tests only the functionalities introduced by that unit but not the unit's dependencies also.
I don't think you must mock out every dependency of your tested class in practice. If it is complicated to create, use or sense through, then yes. Also if it directly depends on some unneeded external resource such as a DB, network or filesystem.
But if none of these is an issue, IMO it is OK to just use an instance of it directly. As you already unit tested it, you can be reasonably sure that it works as expected and doesn't interfere with higher-level unit tests.
I personally prefer working unit tests and simple, clean, maintainable design over adhering to some ideal set up by unit test purists.
each unit-test tests only the functionalities introduced by that unit but not the unit's dependencies also.
Using a functionality and testing a functionality are two very different things.
I also think that using an instance on Inner directly is OK.
My problem is mocking external objects that are not part of my code (provided through static libraries or DLLs, sometimes 3rd party).
I am inclined to rewrite a mock DLL or library with the same class names, and then linking differently for test. Modifying the header file of the external dependency to add "virtual"s seems unacceptable to me.
Does anyone have a better solution?
It known that in C++ mocking/faking nonvirtual methods for testing is hard. For example, cookbook of googlemock has two suggestion - both mean to modify original source code in some way (templating and rewriting as interface).
It appear this is very bad problem for C++ code. How can be done best if you can't modify original code that needs to be faked/mocked? Duplicating whole code/class (with it whole base class hierarchy??)
One way that we sometimes use is to split the original .cpp file into at least two parts.
Then the test apparatus can supply its own implementations; effectively using the linker to do the dirty work for us.
This is called the "Link Seam" in some circles.
I followed the Link Seam link from sdg's answer. There I read about different types of seams, but I was most impressed by Preprocessing Seams. This made me think about exploiting further the preprocessor. It turned out that it is possible to mock any external dependency without actually changing the calling code.
To do this, you have to compile the calling source file with a substitute dependency definition.
Here is an example how to do it.
dependency.h
#ifndef DEPENDENCY_H
#define DEPENDENCY_H
class Dependency
{
public:
//...
int foo();
//...
};
#endif // DEPENDENCY_H
caller.cpp
#include "dependency.h"
int bar(Dependency& dependency)
{
return dependency.foo() * 2;
}
test.cpp
#include <assert.h>
// block original definition
#define DEPENDENCY_H
// substitute definition
class Dependency
{
public:
int foo() { return 21; }
};
// include code under test
#include "caller.cpp"
// the test
void test_bar()
{
Dependency mockDependency;
int r = bar(mockDependency);
assert(r == 42);
}
Notice that the mock does not need to implement complete Dependency, just the minimum (used by caller.cpp) so the test can compile and execute.
This way you can mock non-virtual, static, global functions or almost any dependency without changing the productive code.
Another reason I like this approach is that everything related to the test is in one place. You don't have to tweak compiler and linker configurations here and there.
I have applied this technique successfully on a real world project with big fat dependencies.
I have described it in more detail in Include mock.
Code has to be written to be testable, by whatever test techniques you use. If you want to test using mocks, that means some form of dependency injection.
Non-virtual calls with no dependence on a template parameter pose the same problem as final and static methods in Java[*] - the code under test has explicitly said, "I want to call this code, not some unknown bit of code that's dependent in some way on an argument". You, the tester, want it to call different code under test from what it normally calls. If you can't change the code under test then you, the tester, will lose that argument. You might as well ask how to introduce a test version of line 4 of a 10-line function without changing the code under test.
If the class to be mocked is in a different TU from the class under test, you can write a mock with the same name as the original and link that instead. Whether you can generate that mock using your mocking framework in the normal way, I'm not so sure.
If you like, I suppose it's a "very bad problem for C++" that it's possible to write code that's hard to test. It shares this "problem" with a great number of other languages...
[*] My Java knowledge is quite low-power. There may be some clever way of mocking such methods in Java, which aren't applicable to C++. If so, please disregard them in order to see the analogy ;-)
I think it is not possible to do it with standard C++ right now (but lets hope that a powerful compile-time reflection will come to C++ soon...). However, there are a number of options for doing so.
You might have a look at Injector++. It is Windows only right now, but plans to add support for Linux & Mac.
Another option is CppFreeMock, which seems to work with GCC, but has no recent activities.
HippoMocks also provide such ability, but only for free functions. It doesn't support it for class member functions.
I'm not completely sure, but it seems that all the above achieve this with overwriting the target function at runtime so that it jumps to the faked function.
The there is C-Mock, which is an extension to Google Mock allowing you to mock non-virtual functions by redefining them, and relying on the fact that original functions are in dynamic libraries. It is limited to GNU/Linux platform.
Finally, you might also try PowerFake (for which, I'm the author) as introduced here.
It is not a mocking framework (currently) and it provides the possibility for replacing production functions with test ones. I hope to be able to integrate it to one or more mocking frameworks; if not, it'll become one.
Update: It has an integration with FakeIt.
Update 2: Added support for Google Mock
It also overrides the original function during linking (so, it won't work if a function is called in the same translation unit in which it is defined), but uses a different trick than C-Mock as it uses GNU ld's --wrap option. It also needs some changes to your build system for tests, but doesn't affect the main code in any way (except if you are forced to put a function in a separate .cpp file); but support for easily integrating it into CMake projects is provided.
But, it is currently limited to GCC/GNU ld (works also with MinGW).
Update: It supports GCC & Clang compilers, and GNU ld & LLVM lld linkers (or any compatible linker).
#zaharpopov you can use Typemock IsolatorPP to create mocks of non-virtual class and methods without changing your code (or legacy code).
for example if you have a non-virtual class called MyClass:
class MyClass
{
public:
int GetResult() { return -1; }
}
you can mock it with typemock like so:
MyClass* fakeMyClass = FAKE<MyClass>();
WHEN_CALLED(fakeMyClass->GetResult()).Return(10);
By the way the classes or methods that you want to test can also be private as typemock can mock them too, for example:
class MyClass
{
private:
int PrivateMethod() { return -1; }
}
MyClass* myClass = new MyClass();
PRIVATE_WHEN_CALLED(myClass, PrivateMethod).Return(1);
for more information go here.
You very specifically say "if you can't modify original code", which the techniques you mention in your question (and all the other current "answers") do.
Without changing that source, you can still generally (for common OSes/tools) preload an object that defines its own version of the function(s) you wish to intercept. They can even call the original functions afterwards. I provided an example of doing this in (my) question Good Linux TCP/IP monitoring tools that don't need root access?.
That is easier then you think. Just pass the constructed object to the constructor of the class you are testing. In the class store the reference to that object. Then it is easy to use mock classes.
EDIT :
The object that you are passing to the constructor needs an interface, and that class store just the reference to the interface.
struct Abase
{
virtual ~Abase(){}
virtual void foo() = 0;
};
struct Aimp : public Abase
{
virtual ~Aimp(){}
virtual void foo(){/*do stuff*/}
};
struct B
{
B( Aimp &objA ) : obja( objA )
{
}
void boo()
{
objA.foo();
}
Aimp &obja;
};
int main()
{
//...
Aimp realObjA;
B objB( realObjA );
// ...
}
In the test, you can pass the mock object easy.
I used to create an interface for the parts I needed to mock. Then I simply created a stub class that derived from this interface and passed this instance to my classes under test. Yes, it is a lot of hard work, but I found it worth it for some circumstances.
Oh, by interface I mean a struct with only pure virtual methods. Nothing else!