C++: Separating object's creation from use (for testing purposes) - c++

Suppose I have code like below. http_client is an external dependency (a 3rd party API) I don't have control over. transaction_handler is a class I control and would like to write unit tests for.
// 3rd party
class http_client
{
public:
std::string get(std::string url)
{
// makes an HTTP request and returns response content as string
// throws if status code is not 200
}
};
//code to be tested
enum class transaction_kind { sell, buy };
enum class status { ok, error };
class transaction_handler
{
private:
http_client client;
public:
status issue_transaction(transaction_kind action)
{
try
{
auto response =
client.get(std::string("http://fake.uri/") +
(action == transaction_kind::buy ? "buy" : "sell"));
return response == "OK" ? status::ok : status::error;
}
catch (const std::exception &)
{
return status::error;
}
}
};
Because http_client makes network calls I would like to be able to substitute it in my tests with a mock implementation which cuts off the network and allows for testing different conditions (ok/error/exception). Because transaction_handler is supposed to be internal I can modify it to make it testable but I wouldn't want to go over the border (i.e. I would like to avoid pointers or dynamic polymorphism if possible). Ideally I would like to use a kind of dependency injection where my tests would inject a mock http_client. I don't think I can/want to use a 'poor man's DI' where I would create an http_client implementation in the caller and pass it to the transaction_handler (by const reference? std::shared_ptr?) - because I don't control the http_client I would have to come up with an interface and -in the product code - I would have to wrap the http_client in a wrapper class that implements this interface and forwards the calls to the actual/wrapped http_client instance. In the test code I would create a mock implementation of that interface. The interface would have to be a pure abstract method which entails using runtime polymorphism which I wanted to avoid. Another option is to use templates. If I changed the transaction_handler class to look as follows:
template <typename T = http_client>
class transaction_handler
{
private:
T client;
public:
transaction_handler(const std::function<T()> &create) : client(create())
{}
status issue_transaction(transaction_kind action)
{
// same as above, omitted for brevity
}
}
I could now create a mock http_client class:
class http_client_mock
{
public:
std::string get(std::string url)
{
return std::string("OK");
}
};
and create the transaction_class object in my tests like this:
transaction_handler<http_client_mock> t(
[]() -> http_client_mock { return http_client_mock(); });
while I could use the following in my product code:
transaction_handler<> t1(
[]() -> http_client { return http_client(); });
While it seems to work and fullfill most of my requirements (even though I don't like the fact that the code instantiating transaction_handler need to be aware of the http_client type - maybe it can be somehow hidden as a factory class) - does it make sense at all? Or may be there are better ways of doing this kind of things? I spent a considerable amount of time looking for some simple DI patterns to make unit testing easier bud had hard time finding something that would suit my needs. Also, my background is mostly C so maybe I approach the problem from a wrong angle?

I'm maintaining a DI library, and your case is really interesting for me.
Mocking is about dynamic polymorphism or compile time mocking (at least when using C++). You pay 1 indirection to obtain ability to inject what you want (dependence only on 1 interface).
If you want to make code testable, the best way is using interfaces (pure virtual classes in C++ wich does not have interfaces) and inject dependencies only through constructor.
If you really want to avoid polymorphism (or you can't because of external API) you could still accept to have some code that is not fully testable.
Conventional way of doing things:
class ConcreteHttpClient : public virtual AbstractHttpClient { /*...*/}
class MockHttpClient : public virtual AbstractHttpClient{ /*...*/ }
You just choose what to inject based on needs ( I intentionally use "new" instead of showing some DI framwork at work).
Production code.
new TransactionHandler ( static_cast< AbstractService>( ConcreteService));
Unit testing the transaction handler
new TransactionHandler ( static_cast< AbstractService>( MockService));
If you later need to test some class using the transaction handler and the transaction handler implements a interface
class TransactionHandler: public virtual AbstractTransactionHandler { /*...*/}
You have just to create a TransactionHandlerMock inheriting from AbstractTransactionHandler
When you use interfaces the advantage is that you can use a Dependency Injection framework to avoid poor man's injection.
Compile time mocking.
What you proposed is a viable alternative, basically you assume a "static polymorphism" thanks to templates
Production code:
template <typename T = http_client>
class transaction_handler{/*...*/};
new transaction_handler( /*...*/ );
Unit test code:
using transaction_handler_mocked = transaction_handler< http_client_mock>;
new transaction_handler_mocked( /*...*/ );
However this has few Issues:
You are depending on "transaction_handler" type on each part of your production code so if you change it you have to recompile every file depending on it.
You can't inject a mocked handler as a mock itself, unless you change all classes depending on it to become templates accepting the handler.
Point number 2 means that basically in a complex project you have each class depending on each other, increasin compile times and forcing to whole recompiles of your project just for little changes
You are not using a Interface (pure virtual class) that means that you can still accidentally access fields or members of template parameters that were not intended to be accessed making your debuggin harder.
Other alternatives
Provide mock at link time, you don't have to mock a whole 3rd party library. Just the classes you are testing. Most IDEs are not thinked to work that way, but probably you can work around with some bash script or custom makefile. (In example, I do that for testing code dependent on C functions, in particular OpenGL)
Wrap interesting functionalities of libraries you want to test behind a class implementing a pure virtual class (don't know why you want to avoid it). You have good chances that you don't need to wrap all methods and you'll end with a smaller API to test against (just wrap parts you need to use, don't start up front wrapping the whole library)
Use a mocking framework, and possibly a dependency Injection framework.

Just write test routines that match whichever exports of http_client you're using. You're source will be linked in preference to any lib.

I think it's a good idea to mock it. Since http_client is an external dependency, I chose Typemock Isolator++ to handle with it. Look at the code below:
TEST_METHOD(FakeHttpClient)
{
//Arrange
string okStr = "OK";
http_client* mock_client = FAKE_ALL<http_client>();
WHEN_CALLED(mock_client->get(ANY_VAL(std::string))).Return(&okStr);
//Act
transaction_handler my_handler;
status result = my_handler.issue_transaction(transaction_kind::buy);
//Assert
Assert::AreEqual((int)status::ok, (int)result);
}
Method FAKE_ALL<> allows me to set the behavior of all http_client instances, so no injection needed. Simple API, the code looks accurate, and you don't need to change the production code.
Hope it helps!

Related

Writing testable code in c++ that mocks out third-party dependencies

I am trying to understand what could be the best design that helps in writing testable code. Let's consider below code ..
void Connection::Connect(const string& url)
{
// My code here that I want to test
ThirdpartClass::Connect(modifiedUrl)
// My code here that I want to test
}
In above example ThirdpartClass::Connect(modifiedUrl) is a third part dependency that actually connects to some third-party server which I want to omit as a part of my unit test code and replace it with mock connect call while executing unit tests. My existing code is tightly bound with third party dependency which makes non testable without involving third party. Is there a better design that makes my code test ready.
Low latency is important so run time polymorphism should be a last resort.
I am planning to use google test framework for writing unit test code.
You can give static polymorphism (i.e. templates) a try if you really want to, although I'm not sure the extra complication for the code will be worth it in the end.
First, make your Connection class templated on the type of the class you'll be calling for the connection:
template <typename C = ThirdPartyClass>
class Connection { ... }
Then, your Connect function now looks like this:
void Connection<C>::Connect(const string& url)
{
// My code here that I want to test
C::Connect(modifiedUrl)
// My code here that I want to test
}
Whatever type you pass in as a template parameter to Connection will be used, instead of forcing you to use ThirdPartyClass.
For testing, you can then do the following:
using TestConnection = Connection<MockClass>;
And for actual code, use the ThirdPartyClass instead of the MockClass.
You have to somehow remove the dependency to the third party class when you are testing and unfortunately, you will have to slightly modify your code.
You can create a wrapper around your third party functions. For example:
class ThirdPartyWrapperInterface {
public:
virtual bool Connect(std::string url) = 0;
};
// Use this in production
class ThirdPartyWrapper : public ThirdPartyWrapperInterface {
public:
bool Connect(std::string url) override {
return ThirdpartClass::Connect(modifiedUrl);
}
};
// Use this for test
class MockThirdPartyWrapper : public ThirdPartyWrapperInterface {
public:
MOCK_METHOD(bool, Connect, (std::string), (override));
};
Now your original code should use the wrapper instead and it can select between the two classes using dependency injection as you would normally do when you mock a function.
I understand that runtime polymorphism is not ideal, so you can use templates instead. See here or here for an example.

Unit Testing application interface to hardware - to mock or not

im very curious for peoples opinions on what way a software side application that interfaces with hardware should be unit tested.
For example, the main class of the software application "Connection" would be constructing a handle to a USB device.
I want to test the "Connection" class base function, say "OpenConnection" that would attempt to connect to a USB hardware device.
so far I have constructed a MOCK hardware device, and have included in my connection class a compiler flag, so if its built in unit test mode, it will use a mock object, otherwish it will use the actual hardware interface.
See example below
class TConnection
{
public:
static TConnection* GetConnection();
static void Shutdown();
bool DidInitialise();
bool Write(uint8_t* _pu8_buffer);
bool Read(uint8_t* _pu8_buffer);
protected:
TConnection();
virtual ~TConnection();
bool init();
private:
static TConnection* mp_padConnection;
static bool mb_DidInitialise;
#ifdef _UNIT_TEST_BUILD
static mock_device* mp_handle;
#else
static device* mp_handle;
#endif
};
then in the source file I include something like
#include "connection.h"
#ifdef _UNIT_TEST_BUILD
mock_device* TConnection::mp_handle = nullptr;
#else
device* TConnection::mp_handle = nullptr;
#endif // _UNIT_TEST_BUILD
TConnection::TConnection()
{
...
init();
...
}
bool TConnection::init()
{
mp_handle = hid_open( _VENDOR_ID, _PRODUCT_ID, nullptr );
if (mp_hidHandle == nullptr) {
return false;
}
if (hid_set_nonblocking(mp_hidHandle, _DISABLE_NB) == _ERROR_CODE) {
return false;
}
return true;
}
The only thing I really dislike about my code is that my actual connection class contains test code. I would much prefer them to be separate.
Saying that, I also dont agree with having an entirely new mocked connection class written solely for the purpose of unit testing, it makes me feel like im just writting something designed to work as expected.
So I ask, what would be a better approach to testing such a class
Thank you in advance for your time and advice
You can avoid adding test code to your class by using dependency injection. Create an interface IDevice and make class Device implement that interface. Then, in class TConnection, use pointers to this interface instead of a member of type Device. Also create a helper method that allows you to set a new device, something like:
void setDevice(IDevice *device);
Now, for your production code simple use an instance of class Device, while in your test code use setDevice to swap implementation of device with a mock object. This mock object will be an instance of class MockDevice which will also implement interface IDevice. That way you can change the implementation in tests and use the mock class. Since you are using gtest already, I would suggest you do not write the mock class yourself but use C++ mocking framework gmock (which is fully compatible with gtest) instead. This way, you will also need to create a separate class but almost everything will be handled by the mocking framework. All you need to do is define mocked methods. Creation of an additional interface and mock class seems like overkill at first, but it definitely pays off in the long run. If you want to do any serious test-driving of code, learning to use interfaces, dependency injection and mock classes is essential. Check the documentation for more details:
https://github.com/google/googlemock/blob/master/googlemock/docs/CheatSheet.md
Personally I would have the mock as either a separate class, or part of the test code. To differentiate between mock and actual library I would do the changes in the build script, which I assume would be to include the test file (and mock) or linking to a library.
Creating a separate class is not wasted effort. It should behave as expected, but this can be simplified to the bare minimum needed for the test. A more interesting thing is to make the class generate error events, to make sure your code handles these events correctly. The alternative is sometimes to wait for the error to occur, which I wouldn't recommend.
Two highly recommended books on the subject:
Test-Driven Development for Embedded C: Part 2 gives a good overview of test doubles (mocks being one example) and how to use them.
Modern C++ Programming with Test-Driven Development: Less on test doubles, but more geared towards C++.

C++ or D: idiom to decouple classes without dynamic dispatch?

I am a true lover of performance, but I am aware about the importance of execute automated tests too (until better formal verification methods become mainstream).
The drawback is that when you try to design for testability you start to introduce interfaces to represent your dependencies. As you know, interfaces make your invocations dynamic dispatched over that dependency and reduce the optimization opportunities at compile-time.
class MyDependency
{
void someMethod();
}
class MyUnit
{
// Concrete type reference of MyDependency allows
// to bypass the dynamic dispatch
this(MyDependency mayBeAMock)
{
mayBeAMock.someMethod();
}
unittest
{
// Now how can I get a mock of my dependency without to instantiate it.
auto dep = someBlackMagic.getMock();
auto uut = new MyUnit(dep);
}
}
Does exist a better way to unit test classes without take the dynamic dispatch cost at production. If need, I would can to afford that cost for the unit testing execution but not for the production release.
Im am interested in D and C++ solutions.
One technique I use in D very often is to do compile-time policy replacement:
private struct MyUtilImpl ( HTTPClient )
{
void foo ( )
{
HTTPClient.makeRequest("url");
}
}
version (unittest)
alias MyUtil = MyUtilImpl!FakeHTTPClient;
else
alias MyUtil = MyUtilImpl!RealHTTPClient;
It is very similar to classical dependency injection in spirit but instead of mocking I/O utilities and swapping via interfaces, I do the same at compile time using template parameters.
It also has the benefit of not needing to change anything in rest of the program to start passing interface around.
2 ideas.
You could either make MyUnit a template and pass in a class that implements (or mocks) "someMethod"
Or, if your unit tests rebuild the class you are testing, you could define the Mock and the real version of the class in separate files and conditionally include / link to the one you want.
Of course you have to be careful not to make your tested version too different from the non-test version.

Is it possible to mock or overload a direct reference to Boost::Filesystem(.hpp)?

I'm attempting to help design some unit tests around controllers in a Qt C++ application.
To be frank, I have two large drawbacks. One, my testing background is heavily based on .NET projects, so my knowledge of best practice in the c++ world is slim at best. Two, the designer of the application I am looking at did not architect the code with unit testing in mind.
One specific point point, I'm looking at a controller class that includes boost/filesystem/operations.hpp. The controller constructor goes on to check directory existence and create directories using functions from the boost filesystem code.
Is there any way to overload or mock this behavior? I'm used to setting up an IoC container or at least dependency injected constructors in .NET, and then being able to pass mock objects in the unit test code. I'm not sure how a templated header file would work with that concept, though, or if it is even typical practice in c++.
Currently, I have no flexibility to suggest code changes, as there is a release build coming up this week. But after that, if there are some simple code changes that could improve testability, that is definitely an option. Ideally, there would be a way to overload the filesystem functions in the unit test framework as is, though.
We ended up creating a generic file system wrapper that calls Boost filesystem and accepting it as a parameter to our class constructors so we could send in mock versions at unit test time.
I understand the thought to not mock this, but I think there is value in fast unit tests for our CI environment to run at check in time as well as tests that actually hit the file system.
If you think about it, the only reason why you need to inject an instance of, suppose, IFilesystem into your classes, is to mock it in tests. No part of your non-test codebase gonna use anything but the real filesystem anyway, so it is safe to assume that you can inject not an object but a type into your classes and use them freely within your codebase without type clashes.
So, suppose you have class of interest
struct BuildTree
{
BuildTree(std::string_view dirname) { /*...*/ }
bool has_changed()
{
// iterates through files in directory
// recurs into directories
// checks the modification date against lastModified_
// uses boost::filesystem::last_write_time(), etc.
}
private:
boost::filesystem::file_time_type lastModified_;
};
Now, you could inject a type into the class. This type will be a class with a bunch of static methods. There will be a RealFilesystem type that will redirect to the boost::filesystem methods, and there will be a SpyFilesystem.
template <class Filesystem>
struct BuildTree
{
// ...
bool has_changed()
{
// uses Filesystem::last_write_time(), etc.
}
private:
typename Filesystem::file_time_type lastModified_;
};
The SpyFilesystem will resemble a PIMPL idiom, in that the static methods will redirect calls to the actual implementation.
struct SpyFilesystemImpl;
struct SpyFilesystem
{
using file_time_type = typename SpyFilesystemImpl::file_time_type;
static file_time_type last_time_write(std::string_view filename)
{
return instance.last_time_write(filename);
}
// ... more methods
static SpyFilesystemImpl instance;
};
SpyFilesystemImpl SpyFilesystem::instance{};
// No warranty of completeness provided
struct SpyFilesystemImpl
{
using file_time_type = std::chrono::system_clock::time_point;
void create_directory(std::string_view path) { /*...*/ }
void touch(std::string_view filename)
{
++lastModified_[filename];
}
file_time_type last_time_write(std::string_view filename)
{
return std::chrono::system_clock::time_point{lastModified_[filename]};
}
private:
std::unordered_map<std::string, std::chrono::seconds> lastModified_;
};
Finally, inside each of your tests you would prepare an instance of SpyFilesystemImpl, assign it to the SpyFilesystem::instance and then instantiate your class with SpyFilesystem. like so
// ... our SpyFilesystem and friends ...
// Google Test framework
TEST(BuildTree, PickUpChanges)
{
SpyFilesystemImpl fs{};
fs.create_directory("foo");
fs.touch("foo/bar.txt");
SpyFilesystem::instance = fs;
BuildTree<SpyFilesystem> tree("foo");
EXPECT_FALSE(tree.has_changed());
SpyFilesystem::instance.touch("foo/bar.txt");
EXPECT_TRUE(tree.has_changed());
}
This approach has an advantage that there will be no run-time overhead in the resulting binary (provided that optimizations are enabled). However, it requires more boilerplate code, which might be a problem.
It seems reasonable to me to consider boost::filesystem as an
extension of the standard libraries. So you mock it (or not) in
exactly the same way you mock something like std::istream.
(Generally, of course, you don't mock it, but rather your test
framework provides the necessary environment: the files you need
to read with std::istream, the directories, etc. for
boost::filesystem.)

How can I refactor and unit test complex legacy Java EE5 EJB methods?

My colleagues and I are currently introducing unit tests to our legacy Java EE5 codebase. We use mostly JUnit and Mockito. In the process of writing tests, we have noticed that several methods in our EJBs were hard to test because they did a lot of things at once.
I'm fairly new to the whole testing business, and so I'm looking for insight in how to better structure the code or the tests. My goal is to write good tests without a headache.
This is an example of one of our methods and its logical steps in a service that manages a message queue:
consumeMessages
acknowledgePreviouslyDownloadedMessages
getNewUnreadMessages
addExtraMessages (depending on somewhat complex conditions)
markMessagesAsDownloaded
serializeMessageObjects
The top-level method is currently exposed in the interface, while all sub-methods are private. As far as I understand it, it would be bad practice to just start testing private methods, as only the public interface should matter.
My first reaction was to just make all the sub-methods public and test them in isolation, then in the top-level method just make sure that it calls the sub-methods. But then a colleague mentioned that it might not be a good idea to expose all those low-level methods at the same level as the other one, as it might cause confusion and other developers might start using when they should be using the top-level one. I can't fault his argument.
So here I am.
How do you reconcile exposing easily testable low-level methods versus avoiding to clutter the interfaces? In our case, the EJB interfaces.
I've read in other unit test questions that one should use dependency injection or follow the single responsibility principle, but I'm having trouble applying it in practice. Would anyone have pointers on how to apply that kind of pattern to the example method above?
Would you recommend other general OO patterns or Java EE patterns?
At first glance, I would say that we probably need to introduce a new class, which would 1) expose public methods that can be unit tested but 2) not be exposed in the public interface of your API.
As an example, let's imagine that you are designing an API for a car. To implement the API, you will need an engine (with complex behavior). You want to fully test your engine, but you don't want to expose details to the clients of the car API (all I know about my car is how to push the start button and how to switch the radio channel).
In that case, what I would do is something like that:
public class Engine {
public void doActionOnEngine() {}
public void doOtherActionOnEngine() {}
}
public class Car {
private Engine engine;
// the setter is used for dependency injection
public void setEngine(Engine engine) {
this.engine = engine;
}
// notice that there is no getter for engine
public void doActionOnCar() {
engine.doActionOnEngine();
}
public void doOtherActionOnCar() {
engine.doActionOnEngine();
engine.doOtherActionOnEngine(),
}
}
For the people using the Car API, there is no way to access the engine directly, so there is no risk to do harm. On the other hand, it is possible to fully unit test the engine.
Dependency Injection (DI) and Single Responsibility Principle (SRP) are highly related.
SRP is basicly stating that each class should only do one thing and delegate all other matters to separate classes. For instance, your serializeMessageObjects method should be extracted into its own class -- let's call it MessageObjectSerializer.
DI means injecting (passing) the MessageObjectSerializer object as an argument to your MessageQueue object -- either in the constructor or in the call to the consumeMessages method. You can use DI frameworks to do this for, but I recommend to do it manually, to get the concept.
Now, if you create an interface for the MessageObjectSerializer, you can pass that to the MessageQueue, and then you get the full value of the pattern, as you can create mocks/stubs for easy testing. Suddenly, consumeMessages doesn't have to pay attention to how serializeMessageObjects behaves.
Below, I have tried to illustrate the pattern. Note, that when you want to test consumeMessages, you don't have to use the the MessageObjectSerializer object. You can make a mock or stub, that does exactly what you want it to do, and pass it instead of the concrete class. This really makes testing so much easier. Please, forgive syntax errors. I did not have access to Visual Studio, so it is written in a text editor.
// THE MAIN CLASS
public class MyMessageQueue()
{
IMessageObjectSerializer _serializer;
//Constructor that takes the gets the serialization logic injected
public MyMessageQueue(IMessageObjectSerializer serializer)
{
_serializer = serializer;
//Also a lot of other injection
}
//Your main method. Now it calls an external object to serialize
public void consumeMessages()
{
//Do all the other stuff
_serializer.serializeMessageObjects()
}
}
//THE SERIALIZER CLASS
Public class MessageObjectSerializer : IMessageObjectSerializer
{
public List<MessageObject> serializeMessageObjects()
{
//DO THE SERILIZATION LOGIC HERE
}
}
//THE INTERFACE FOR THE SERIALIZER
Public interface MessageObjectSerializer
{
List<MessageObject> serializeMessageObjects();
}
EDIT: Sorry, my example is in C#. I hope you can use it anyway :-)
Well, as you have noticed, it's very hard to unit test a concrete, high-level program. You have also identified the two most common issues:
Usually the program is configured to use specific resources, such as a specific file, IP address, hostname etc. To counter this, you need to refactor the program to use dependency injection. This is usually done by adding parameters to the constructor that replace the ahrdcoded values.
It's also very hard to test large classes and methods. This is usually due to the combinatorical explosion in the number of tests required to test a complex piece of logic. To counter this, you will usually refactor first to get lots more (but shorter) methods, then trying to make the code more generic and testable by extracting several classes from your original class that each have a single entry method (public) and several utility methods (private). This is essentially the single responsibility principle.
Now you can start working your way "up" by testing the new classes. This will be a lot easier, as the combinatoricals are much easier to handle at this point.
At some point along the way you will probably find that you can simplify your code greatly by using these design patterns: Command, Composite, Adaptor, Factory, Builder and Facade. These are the most common patterns that cut down on clutter.
Some parts of the old program will probably be largely untestable, either because they are just too crufty, or because it's not worth the trouble. Here you can settle for a simple test that just checks that the output from known input has not changed. Essentially a regression test.