Suppose I have classes
class Inner {
public:
void doSomething();
};
class Outer {
public:
Outer(Inner *inner); // Dependency injection.
void callInner();
};
Proper unit-testing says I should have tests for Inner. Then, I should have tests for Outer that uses not a real Inner but rather a MockInner so that I would be performing unit-tests on the functionality added by just Outer instead of the full stack Outer/Inner.
To do so, Googletest seems to suggest turning Inner into a pure abstract class (interface) like this:
// Introduced merely for the sake of unit-testing.
struct InnerInterface {
void doSomething() = 0;
};
// Used in production.
class Inner : public InnerInterface {
public:
/* override */ void doSomething();
};
// Used in unit-tests.
class MockInner : public InnerInterface {
public:
/* override */ void doSomething();
};
class Outer {
public:
Outer(Inner *inner); // Dependency injection.
void callInner();
};
So, in production code, I would use Outer(new Inner); while in test, Outer(new MockInner).
OK. Seems nice in theory, but when I started using this idea throughout the code, I find myself creating a pure abstract class for every freaking class. It's a lot of boiler-plate typing, even if you can ignore the slight run-time performance degradable due to the unnecessary virtual dispatch.
An alternative approach is to use templates as in the following:
class Inner {
public:
void doSomething();
};
class MockInner {
public:
void doSomething();
};
template<class I>
class Outer {
public:
Outer(I *inner);
void callInner();
};
// In production, use
Outer<Inner> obj;
// In test, use
Outer<MockInner> test_obj;
This avoids the boiler-plating and the unnecessary virtual dispatch; but now my entire codebase is in the freaking header files, which makes it impossible to hide source implementations (not to mention dealing with frustrating template compilation errors and the long build time).
Are those two methods, virtuals and templates, the only ways to do proper unit-testing? Are there better ways to do proper unit-testing?
By proper unit-testing, I mean each unit-test tests only the functionalities introduced by that unit but not the unit's dependencies also.
I don't think you must mock out every dependency of your tested class in practice. If it is complicated to create, use or sense through, then yes. Also if it directly depends on some unneeded external resource such as a DB, network or filesystem.
But if none of these is an issue, IMO it is OK to just use an instance of it directly. As you already unit tested it, you can be reasonably sure that it works as expected and doesn't interfere with higher-level unit tests.
I personally prefer working unit tests and simple, clean, maintainable design over adhering to some ideal set up by unit test purists.
each unit-test tests only the functionalities introduced by that unit but not the unit's dependencies also.
Using a functionality and testing a functionality are two very different things.
I also think that using an instance on Inner directly is OK.
My problem is mocking external objects that are not part of my code (provided through static libraries or DLLs, sometimes 3rd party).
I am inclined to rewrite a mock DLL or library with the same class names, and then linking differently for test. Modifying the header file of the external dependency to add "virtual"s seems unacceptable to me.
Does anyone have a better solution?
Related
I am stuck in a problem and can't seem to find the solution.
I am using VS2005 SP1 for compiling the code.
I have a global function:
A* foo();
I have a mock class
class MockA : public A {
public:
MOCK_METHOD0 (bar, bool());
...
};
In the sources, it is accessed like this: foo()->bar(). I cannot find a way to mock this behavior. And I cannot change the sources, so the solution in google mock cook book is out of question.
Any help or pointers in the right direction will be highly appreciated. :)
No it's not possible, without changing the sources, or bringing your own version of foo() that is linked with the executable code.
From GoogleMock's FAQ it says
My code calls a static/global function. Can I mock it?
You can, but you need to make some changes.
In general, if you find yourself needing to mock a static function, it's a sign that your modules are too tightly coupled (and less flexible, less reusable, less testable, etc). You are probably better off defining a small interface and call the function through that interface, which then can be easily mocked. It's a bit of work initially, but usually pays for itself quickly.
This Google Testing Blog post says it excellently. Check it out.
Also from the Cookbook
Mocking Free Functions
It's possible to use Google Mock to mock a free function (i.e. a C-style function or a static method). You just need to rewrite your code to use an interface (abstract class).
Instead of calling a free function (say, OpenFile) directly, introduce an interface for it and have a concrete subclass that calls the free function:
class FileInterface {
public:
...
virtual bool Open(const char* path, const char* mode) = 0;
};
class File : public FileInterface {
public:
...
virtual bool Open(const char* path, const char* mode) {
return OpenFile(path, mode);
}
};
Your code should talk to FileInterface to open a file. Now it's easy to mock out the function.
This may seem much hassle, but in practice you often have multiple related functions that you can put in the same interface, so the per-function syntactic overhead will be much lower.
If you are concerned about the performance overhead incurred by virtual functions, and profiling confirms your concern, you can combine this with the recipe for mocking non-virtual methods.
As you mentioned in your comment that you actually provide your own version of foo(), you can easily solve this having a global instance of another mock class:
struct IFoo {
virtual A* foo() = 0;
virtual ~IFoo() {}
};
struct FooMock : public IFoo {
FooMock() {}
virtual ~FooMock() {}
MOCK_METHOD0(foo, A*());
};
FooMock fooMock;
// Your foo() implementation
A* foo() {
return fooMock.foo();
}
TEST(...) {
EXPECT_CALL(fooMock,foo())
.Times(1)
.WillOnceReturn(new MockA());
// ...
}
Don't forget to clear all call expectations, after each test case run.
There are 2 options:
If you insist on using gmock, there's an "extension" for global mocking from apriorit: https://github.com/apriorit/gmock-global
It's rather limited, though - or at least I couldn't figure out in 5 minutes how to have side effects on a mocked call.
If you're willing to switch from gmock, then hippomocks has a very neat way of doing what you want.
Here's an example for mocking fopen, fclose and fgets for testing a member function which reads from a file using cstdio (streams are very inefficient):
TEST_CASE("Multi entry") {
std::vector<std::string> files{"Hello.mp3", "World.mp3"};
size_t entry_idx = 0;
MockRepository mocks;
mocks.OnCallFunc(fopen).Return(reinterpret_cast<FILE *>(1));
mocks.OnCallFunc(fgets).Do(
[&](char * buf, int n, FILE * f)->char *{
if (entry_idx < files.size())
{
strcpy(buf, files[entry_idx++].c_str());
return buf;
}
else
return 0;
}
);
mocks.OnCallFunc(fclose).Return(0);
FileExplorer file_explorer;
for (const auto &entry: files)
REQUIRE_THAT(file_explorer.next_file_name(), Equals(entry.c_str()));
REQUIRE_THAT(file_explorer.next_file_name(), Equals(""));
}
Where the function under test looks like this:
string FileExplorer::next_file_name() {
char entry[255];
if (fgets((char *)entry, 255, _sorted_entries_in_dir) == NULL)
return string();
_current_idx++;
if (_current_idx == _line_offsets.size())
_line_offsets.push_back(static_cast<unsigned>(char_traits<char>::length(entry)) + _line_offsets.back());
return string(entry);
}
I'm using catch2 as the testing framework here, but I think hippomocks would work with Google's Testing framework as well (I recommend catch2, by the way, really easy to work with).
Of course, the answer explaining the solution according to GTest/GMock's documentation couldn't be much more correct.
But I would like to add a temporary quick&dirty approach. It should be applicable to cases where you want to get legacy C/C++ code under test as quickly and as non-invasively as possible. (Just to proceed with fixes, refactoring and more proper testing as soon as possible after.)
So, to mock a free function void foo(int) appearing in some code to be tested, within the source file you just make the following adaptions:
#if TESTING
#define foo(param) // to nothing, so calls to that disappear
#endif
// ... code that calls foo stays untouched and could be tested
The macro TESTING, indicating that the code runs under test, doesn't come with GTest/GMock - you need to add it to test targets by yourself.
The possibilities are rather limited, but you might also be able to construct something useful for return types as A* in the question's example.
Unfortunately, also this isn't a solution without changing the code.
If that is really necessary, you could Google for 'link seams'. But my guess is that this could be quite a hassle in practice. And it even might not be possible at all in many/most cases?!
If your free function is in the form of an std::function object, you can mock it using MockFunction. See this answer
What has worked for me is
to define A* foo() in a separate source file foo.cpp in the main project,
not to include foo.cpp in the test project,
include a different source file mock-foo.cpp in the test project that provides the mock implementation of A* foo().
For example, pseudocode for the main project file (e.g. .vcxproj or CMakeLists.txt):
include src/foo.hpp # declare A* foo()
include src/foo.cpp # define A* foo()
and the test project file:
include src/foo.hpp
include test/mock-foo.cpp # define mocked A* foo()
Simple and sweet, but may or may not work in your case.
I'm writing unit tests for my class and would like to exercise methods that are private. I'm doing something like this:
class MyClass {
Myclass() {}
~MyClass() {}
#ifdef TESTING
public:
#else
private:
#endif
void MyMethod1();
void MyMethod2();
};
If TESTING is defined, I change the class signature to make everything public. In my test code, I just do something like this:
#define TESTING
#include "MyClass.h"
void MyTestMethod()
{
MyClass mc;
mc.MyMethod1(); // Now I can access MyMethod1
}
So that its only public in my test files and not anywhere else. My testing executable sees a header that describes a class that is all public. The code for the class is actually being built somewhere else (where TESTING is not defined), so the scope will differ when the testing project links against the library.
Does this have the potential to break anything? I'm worried that this could change vtables locations or compiler and linker expectations if the header differs from what was actually produced in the object file.
You risk ending up with low quality tests. Tests that are testing some implementation details not relevant to the actual function of the classes. Testing the public interface only drives you towards a good design and test cases that can stand code refactors without breaking.
Ugly, although it's possible but don't do it that way instead create a wrapper class for your tests and use the friend TestClass; keyword in MyClass.
When designing functions inside a C++ class, I often keep testing in mind. So when I finish all the functionalists of a class, very often several functions that are only used for testing purpose are added in the class as well. Take the following example:
class SimpleClass
{
public:
int a;
int b;
void fun1();
void fun2();
void testIntermediateResult();
private:
void _fun1();
void _fun2();
};
In this function testIntermediateResult is only needed for testing purpose. Is it a good practice just leave the function inside the class or should I do something like that:
class SimpleClass
{
public:
int a;
int b;
void fun1();
void fun2();
#ifdef TESTING
void testIntermediateResult();
#endif
private:
void _fun1();
void _fun2();
};
The philosophy here is when this class has been finished testing and will be given to the clients then TESTING will be defined and this function will not be inside the class. Therefore, my question is: is really needed to remove the testing function when the class is given to the client? Suppose the client will not use the testing function, what's the effect of adding several redundant testing functions inside a class? Thanks.
I'm assuming that your client is not going to be involved in testing the class that you're handing over. I would not want to add testing functions directly into the class itself as there's a possibility that they could disrupt the behaviour of the class. As for the client, I would prefer not to give them something with a test function in it since:
It's not necessary - it's not their job to test it.
They could decide to try to use it - who knows what'll happen then?
It's just simpler if they don't see it. As for using the pre-processor, this could be fraught with problems particularly if you have attributes that need to be guarded in the same way. If any of them are missed or your macro is redefined in the build process by the client then you could get run-time crashes due to class size mismatches etc.
I'd favour having a one-to-one external class to test your deliverable classes. Something like TestSimpleClass which performs the testing. There are a number of advantages to this approach:
It's entirely separate from your code and not built into it so you aren't bloating code or causing any potential issues with it.
It's going to test your class interface the way your client sees it (i.e. black box testing)
Because it's separate, you don't have to give it to your client - they never have to know about it.
If you really want to test the internals of the class, you can always make your test class a friend of your deliverable class. It's only one line extra in the deliverable class and you still don't have to ship your test classes or libraries.
I'm attempting to help design some unit tests around controllers in a Qt C++ application.
To be frank, I have two large drawbacks. One, my testing background is heavily based on .NET projects, so my knowledge of best practice in the c++ world is slim at best. Two, the designer of the application I am looking at did not architect the code with unit testing in mind.
One specific point point, I'm looking at a controller class that includes boost/filesystem/operations.hpp. The controller constructor goes on to check directory existence and create directories using functions from the boost filesystem code.
Is there any way to overload or mock this behavior? I'm used to setting up an IoC container or at least dependency injected constructors in .NET, and then being able to pass mock objects in the unit test code. I'm not sure how a templated header file would work with that concept, though, or if it is even typical practice in c++.
Currently, I have no flexibility to suggest code changes, as there is a release build coming up this week. But after that, if there are some simple code changes that could improve testability, that is definitely an option. Ideally, there would be a way to overload the filesystem functions in the unit test framework as is, though.
We ended up creating a generic file system wrapper that calls Boost filesystem and accepting it as a parameter to our class constructors so we could send in mock versions at unit test time.
I understand the thought to not mock this, but I think there is value in fast unit tests for our CI environment to run at check in time as well as tests that actually hit the file system.
If you think about it, the only reason why you need to inject an instance of, suppose, IFilesystem into your classes, is to mock it in tests. No part of your non-test codebase gonna use anything but the real filesystem anyway, so it is safe to assume that you can inject not an object but a type into your classes and use them freely within your codebase without type clashes.
So, suppose you have class of interest
struct BuildTree
{
BuildTree(std::string_view dirname) { /*...*/ }
bool has_changed()
{
// iterates through files in directory
// recurs into directories
// checks the modification date against lastModified_
// uses boost::filesystem::last_write_time(), etc.
}
private:
boost::filesystem::file_time_type lastModified_;
};
Now, you could inject a type into the class. This type will be a class with a bunch of static methods. There will be a RealFilesystem type that will redirect to the boost::filesystem methods, and there will be a SpyFilesystem.
template <class Filesystem>
struct BuildTree
{
// ...
bool has_changed()
{
// uses Filesystem::last_write_time(), etc.
}
private:
typename Filesystem::file_time_type lastModified_;
};
The SpyFilesystem will resemble a PIMPL idiom, in that the static methods will redirect calls to the actual implementation.
struct SpyFilesystemImpl;
struct SpyFilesystem
{
using file_time_type = typename SpyFilesystemImpl::file_time_type;
static file_time_type last_time_write(std::string_view filename)
{
return instance.last_time_write(filename);
}
// ... more methods
static SpyFilesystemImpl instance;
};
SpyFilesystemImpl SpyFilesystem::instance{};
// No warranty of completeness provided
struct SpyFilesystemImpl
{
using file_time_type = std::chrono::system_clock::time_point;
void create_directory(std::string_view path) { /*...*/ }
void touch(std::string_view filename)
{
++lastModified_[filename];
}
file_time_type last_time_write(std::string_view filename)
{
return std::chrono::system_clock::time_point{lastModified_[filename]};
}
private:
std::unordered_map<std::string, std::chrono::seconds> lastModified_;
};
Finally, inside each of your tests you would prepare an instance of SpyFilesystemImpl, assign it to the SpyFilesystem::instance and then instantiate your class with SpyFilesystem. like so
// ... our SpyFilesystem and friends ...
// Google Test framework
TEST(BuildTree, PickUpChanges)
{
SpyFilesystemImpl fs{};
fs.create_directory("foo");
fs.touch("foo/bar.txt");
SpyFilesystem::instance = fs;
BuildTree<SpyFilesystem> tree("foo");
EXPECT_FALSE(tree.has_changed());
SpyFilesystem::instance.touch("foo/bar.txt");
EXPECT_TRUE(tree.has_changed());
}
This approach has an advantage that there will be no run-time overhead in the resulting binary (provided that optimizations are enabled). However, it requires more boilerplate code, which might be a problem.
It seems reasonable to me to consider boost::filesystem as an
extension of the standard libraries. So you mock it (or not) in
exactly the same way you mock something like std::istream.
(Generally, of course, you don't mock it, but rather your test
framework provides the necessary environment: the files you need
to read with std::istream, the directories, etc. for
boost::filesystem.)
It known that in C++ mocking/faking nonvirtual methods for testing is hard. For example, cookbook of googlemock has two suggestion - both mean to modify original source code in some way (templating and rewriting as interface).
It appear this is very bad problem for C++ code. How can be done best if you can't modify original code that needs to be faked/mocked? Duplicating whole code/class (with it whole base class hierarchy??)
One way that we sometimes use is to split the original .cpp file into at least two parts.
Then the test apparatus can supply its own implementations; effectively using the linker to do the dirty work for us.
This is called the "Link Seam" in some circles.
I followed the Link Seam link from sdg's answer. There I read about different types of seams, but I was most impressed by Preprocessing Seams. This made me think about exploiting further the preprocessor. It turned out that it is possible to mock any external dependency without actually changing the calling code.
To do this, you have to compile the calling source file with a substitute dependency definition.
Here is an example how to do it.
dependency.h
#ifndef DEPENDENCY_H
#define DEPENDENCY_H
class Dependency
{
public:
//...
int foo();
//...
};
#endif // DEPENDENCY_H
caller.cpp
#include "dependency.h"
int bar(Dependency& dependency)
{
return dependency.foo() * 2;
}
test.cpp
#include <assert.h>
// block original definition
#define DEPENDENCY_H
// substitute definition
class Dependency
{
public:
int foo() { return 21; }
};
// include code under test
#include "caller.cpp"
// the test
void test_bar()
{
Dependency mockDependency;
int r = bar(mockDependency);
assert(r == 42);
}
Notice that the mock does not need to implement complete Dependency, just the minimum (used by caller.cpp) so the test can compile and execute.
This way you can mock non-virtual, static, global functions or almost any dependency without changing the productive code.
Another reason I like this approach is that everything related to the test is in one place. You don't have to tweak compiler and linker configurations here and there.
I have applied this technique successfully on a real world project with big fat dependencies.
I have described it in more detail in Include mock.
Code has to be written to be testable, by whatever test techniques you use. If you want to test using mocks, that means some form of dependency injection.
Non-virtual calls with no dependence on a template parameter pose the same problem as final and static methods in Java[*] - the code under test has explicitly said, "I want to call this code, not some unknown bit of code that's dependent in some way on an argument". You, the tester, want it to call different code under test from what it normally calls. If you can't change the code under test then you, the tester, will lose that argument. You might as well ask how to introduce a test version of line 4 of a 10-line function without changing the code under test.
If the class to be mocked is in a different TU from the class under test, you can write a mock with the same name as the original and link that instead. Whether you can generate that mock using your mocking framework in the normal way, I'm not so sure.
If you like, I suppose it's a "very bad problem for C++" that it's possible to write code that's hard to test. It shares this "problem" with a great number of other languages...
[*] My Java knowledge is quite low-power. There may be some clever way of mocking such methods in Java, which aren't applicable to C++. If so, please disregard them in order to see the analogy ;-)
I think it is not possible to do it with standard C++ right now (but lets hope that a powerful compile-time reflection will come to C++ soon...). However, there are a number of options for doing so.
You might have a look at Injector++. It is Windows only right now, but plans to add support for Linux & Mac.
Another option is CppFreeMock, which seems to work with GCC, but has no recent activities.
HippoMocks also provide such ability, but only for free functions. It doesn't support it for class member functions.
I'm not completely sure, but it seems that all the above achieve this with overwriting the target function at runtime so that it jumps to the faked function.
The there is C-Mock, which is an extension to Google Mock allowing you to mock non-virtual functions by redefining them, and relying on the fact that original functions are in dynamic libraries. It is limited to GNU/Linux platform.
Finally, you might also try PowerFake (for which, I'm the author) as introduced here.
It is not a mocking framework (currently) and it provides the possibility for replacing production functions with test ones. I hope to be able to integrate it to one or more mocking frameworks; if not, it'll become one.
Update: It has an integration with FakeIt.
Update 2: Added support for Google Mock
It also overrides the original function during linking (so, it won't work if a function is called in the same translation unit in which it is defined), but uses a different trick than C-Mock as it uses GNU ld's --wrap option. It also needs some changes to your build system for tests, but doesn't affect the main code in any way (except if you are forced to put a function in a separate .cpp file); but support for easily integrating it into CMake projects is provided.
But, it is currently limited to GCC/GNU ld (works also with MinGW).
Update: It supports GCC & Clang compilers, and GNU ld & LLVM lld linkers (or any compatible linker).
#zaharpopov you can use Typemock IsolatorPP to create mocks of non-virtual class and methods without changing your code (or legacy code).
for example if you have a non-virtual class called MyClass:
class MyClass
{
public:
int GetResult() { return -1; }
}
you can mock it with typemock like so:
MyClass* fakeMyClass = FAKE<MyClass>();
WHEN_CALLED(fakeMyClass->GetResult()).Return(10);
By the way the classes or methods that you want to test can also be private as typemock can mock them too, for example:
class MyClass
{
private:
int PrivateMethod() { return -1; }
}
MyClass* myClass = new MyClass();
PRIVATE_WHEN_CALLED(myClass, PrivateMethod).Return(1);
for more information go here.
You very specifically say "if you can't modify original code", which the techniques you mention in your question (and all the other current "answers") do.
Without changing that source, you can still generally (for common OSes/tools) preload an object that defines its own version of the function(s) you wish to intercept. They can even call the original functions afterwards. I provided an example of doing this in (my) question Good Linux TCP/IP monitoring tools that don't need root access?.
That is easier then you think. Just pass the constructed object to the constructor of the class you are testing. In the class store the reference to that object. Then it is easy to use mock classes.
EDIT :
The object that you are passing to the constructor needs an interface, and that class store just the reference to the interface.
struct Abase
{
virtual ~Abase(){}
virtual void foo() = 0;
};
struct Aimp : public Abase
{
virtual ~Aimp(){}
virtual void foo(){/*do stuff*/}
};
struct B
{
B( Aimp &objA ) : obja( objA )
{
}
void boo()
{
objA.foo();
}
Aimp &obja;
};
int main()
{
//...
Aimp realObjA;
B objB( realObjA );
// ...
}
In the test, you can pass the mock object easy.
I used to create an interface for the parts I needed to mock. Then I simply created a stub class that derived from this interface and passed this instance to my classes under test. Yes, it is a lot of hard work, but I found it worth it for some circumstances.
Oh, by interface I mean a struct with only pure virtual methods. Nothing else!