TL;DR: You can use GMock to add mocking capability to your Microsoft native c++ unit tests. See my answer below for details.
I want to start adding mocks to my existing set of native unit tests. The tests are written using Microsoft's CppUnitTestFramework framework, which doesn't have support for mocking. I don't really want to convert the entire test suite to another framework just to add a few mocks.
Google's GMock framework seems to provide everything I need and the documentation suggests it can be used with frameworks other than gtest. So using advice from blog posts like this one, I created a couple of unit tests.
TEST_MODULE_INITIALIZE(ModuleInitialize)
{
// Enable google mock
GTEST_FLAG(throw_on_failure) = true;
int argc = 0;
TCHAR **argv = nullptr;
InitGoogleMock(&argc, argv);
}
TEST_CLASS(GMockTests)
{
public:
MockTestClass _mockObj;
TEST_METHOD(Method1_ParamIsOne_Method2CalledWithOne)
{
EXPECT_CALL(_mockObj, Method2(1)).Times(1);
_mockObj.Method1(1);
}
TEST_METHOD(Method1_ParamIsZero_IntentionallyFail)
{
// Expectation will fail
EXPECT_CALL(_mockObj, Method2(1)).Times(1);
_mockObj.Method1(0);
}
};
The results are less than satisfactory. The expectations do work (first method passes), but if any expectation fails the entire run is aborted with only the following unhelpful message in the test output:
[3/27/2019 11:39:17 AM Error] The active test run was aborted. Reason:
[3/27/2019 11:39:17 AM Informational] ========== Run test finished: 0 run (0:00:22.3042194) ==========
The Visual Studio Test Explorer window doesn't indicate what the problem is either. It just shows that one test succeeded and the other one didn't run:
So what I'm looking for from this integration is:
A GMock test failure doesn't abort the entire run.
Tests that fail GMock expectations are shown as failed.
All GMock messages should be shown in the test output.
I was able to get GMock working correctly with the CppUnitTestFramework by creating a custom TestEventListener. I then created a simple set of interface functions to make it easier to work with.
I used the gmock 1.7.0 NuGet package to install the GMock framework into my project, and added the two files shown below.
GMockUtils.h
#pragma once
#include <CppUnitTest.h>
#include <gmock/gmock.h>
namespace testing {
namespace GMockUtils {
// Call once per test class or module to set up everything needed by GoogleMock.
void InitGoogleMock();
// Call once per test method to check for GoogleMock messages.
void CheckGoogleMock();
}
}
GMockUtils.cpp
#include "stdafx.h"
#include "GMockUtils.h"
using namespace Microsoft::VisualStudio::CppUnitTestFramework;
namespace testing {
namespace GMockUtils {
namespace {
// Test event listener for use with CppUnitTestFramework
class CppUnitTestReporter : public EmptyTestEventListener
{
public:
CppUnitTestReporter() : _failed(false)
{
}
// Helper for converting std::string to std::wstring
std::wstring to_wstring(const std::string& str) const
{
std::wstring output;
std::copy(str.begin(), str.end(), std::back_inserter(output));
return output;
}
// Called after a failed assertion or a SUCCEED() invocation.
void OnTestPartResult(const ::testing::TestPartResult& result) override
{
// Log this result to the CppUnitTestFramework output
Logger::WriteMessage(result.summary());
// Note: You cannot do an Assert directly from a listener, so we
// just store the messages until CheckGoogleMock() is called.
if (result.failed())
{
_failed = true;
// Append this result to the running summary
_failedSummary += result.message();
_failedSummary += "\n";
}
}
// Clear any previous failures
void ResetFailures()
{
_failed = false;
_failedSummary.clear();
}
// Assert if any failures have been detected. Also resets failures.
void CheckFailures()
{
auto failed = _failed;
auto failedSummary = _failedSummary;
ResetFailures();
Assert::IsFalse(failed, to_wstring(failedSummary).c_str());
}
protected:
bool _failed;
std::string _failedSummary;
} *_listener;
}
// Initialize the Google Mock framework for use with CppUnitTestFramework
void InitGoogleMock()
{
// Avoids calling the function unnecessarily
if (_listener != nullptr)
return;
// Dummy command line parameters (could pass exe path here)
int argc = 0;
char** argv = nullptr;
// Initialize the framework
::testing::InitGoogleMock(&argc, argv);
// We don't want exceptions thrown, regardless what the doc says
GTEST_FLAG(throw_on_failure) = false;
// Remove default listener
auto &listeners = UnitTest::GetInstance()->listeners();
delete listeners.Release(listeners.default_result_printer());
// Create and install the new listener
_listener = new CppUnitTestReporter();
listeners.Append(_listener);
}
// Reset any previous failures detected by the listener
void ResetGoogleMock()
{
_listener->ResetFailures();
}
// Prints messages and asserts if any Google Mock messages are found.
void CheckGoogleMock()
{
Assert::IsNotNull(_listener, L"Google Mock framework not initialized by InitGoogleMock()");
_listener->CheckFailures();
}
}
}
I use the GMockUtils functions in unit test classes like this:
#include "GMockUtils.h"
using namespace Microsoft::VisualStudio::CppUnitTestFramework;
using namespace ::testing;
namespace GMockUtilsDemo
{
TEST_CLASS(GMockUtilTests)
{
public:
TEST_CLASS_INITIALIZE(ClassInitializer)
{
// IMPORTANT: This must be called before any mock object constructors
GMockUtils::InitGoogleMock();
}
TEST_METHOD_CLEANUP(MethodCleanup)
{
// Checks for GoogleMock messages. Asserts if found.
GMockUtils::CheckGoogleMock();
}
TEST_METHOD(Method1_ParamIsOne_Method2CalledWithOne)
{
MockTestClass mockObj;
EXPECT_CALL(mockObj, Method2(1)).Times(1); // Expectation will be met
mockObj.Method1(1);
}
TEST_METHOD(Method1_ParamIsZero_IntentionallyFail)
{
MockTestClass mockObj;
EXPECT_CALL(mockObj, Method2(1)).Times(1); // Expectation will not be met
mockObj.Method1(0);
}
};
}
The Console Output
The console output now shows all GMock messages, and the run does not abort on the first test failure.
[3/27/2019 12:23:46 PM Informational] ------ Run test started ------
[3/27/2019 12:23:46 PM Informational]
Unexpected mock function call - returning directly.
Function call: Method2(0)
Google Mock tried the following 1 expectation, but it didn't match:
c:\...\gmockutilsdemo.cpp(64): EXPECT_CALL(_mockObj, Method2(1))...
Expected arg #0: is equal to 1
Actual: 0
Expected: to be called once
Actual: never called - unsatisfied and active
[3/27/2019 12:23:46 PM Informational] Actual function call count doesn't match EXPECT_CALL(_mockObj, Method2(1))...
Expected: to be called once
Actual: never called - unsatisfied and active
[3/27/2019 12:23:46 PM Informational] ========== Run test finished: 2 run (0:00:00.8631468) ==========
Test Explorer View
If I run the tests via Visual Studio Test Explorer, I can also see all GMock messages related to a particular test. It also works with the VsTest task on Azure DevOps.
Hopefully this will be useful to anyone who finds themselves in the same situation.
Related
I am trying to mock an Open function present as part of the CDatabase Class by using Google Mock. Visual studio version: 2010
Below is the code snippet:
class TestRLanguage: public RLanguage, public ::testing::Test
{
};
class FakeDatabase : public CDatabase
{
public:
MOCK_METHOD5(Open_impl, BOOL(LPCTSTR lpszDSN, BOOL bExclusive,BOOL bReadonly, LPCTSTR lpszConnect,BOOL bUseCursorLib));
BOOL Open(LPCTSTR lpszDSN, BOOL bExclusive = FALSE,
BOOL bReadonly = FALSE, LPCTSTR lpszConnect = _T("ODBC;"),
BOOL bUseCursorLib = TRUE) override
{
Open_impl( lpszDSN, bExclusive, bReadonly, lpszConnect, bUseCursorLib);
}
};
// Test case for testing initialize function
TEST_F(TestRLanguage, initializeSuccess)
{
FakeDatabase database;
EXPECT_CALL(database, Open(NULL, FALSE, FALSE, "ODBC;DSN=", FALSE)).Times(testing::Exactly(1));
}
I get the below error:
error C2770: invalid explicit template argument(s) for 'AddReference<const ::std::tr1::tuple_element<I,std::tr1::tuple<_Arg0,_Arg1,_Arg2,_Arg3,_Arg4>>::type>::type testing::internal::TuplePolicy<TupleT>::get(const std::tr1::tuple<_Arg0,_Arg1,_Arg2,_Arg3,_Arg4> &)' c:\src\googlemockandtest\aqtsw\util\googletest\googletest-release-1.8.1\googletest\include\gtest\gtest-printers.h
Can anyone help me in solving this problem?
I had the same issue. In my case it was related to the version of Google Test. I had the issue with version 1.8.1 -- "the last release supporting pre-C++11 compilers" (as stated here). The issue was fixed with pull request #1817. I am using HEAD of branch 1.8.x now (commit dea0216) and the issue is gone.
There are two errors in your test:
you put in the EXPECT_CALL a method function, that is not a mock method
you do not call any method in your test.
This is a proper way:
TEST_F(TestRLanguage, initializeSuccess)
{
FakeDatabase database;
EXPECT_CALL(database, Open_impl(NULL, FALSE, FALSE, "ODBC;DSN=", FALSE)).Times(testing::Exactly(1));
database.Open(NULL);
}
I know that the unit test for Java is very simple. We just need use JUnit and run test classes as JUnit test.
Now I'm working with C++ and I find a test library: cppunit.
It seems that I need to run my test with a specific function main.
However, a cpp project can have ONLY ONE main.
What should I do? I have to switch mains when I do my test and when I run my project?
What should I do? I have to switch mains when I do my test and when I run my project?
You should separate the code to test into a library project, that can be linked from your target application and the unit test runner.
Then have two more projects providing a main() function:
one for the target app, that just forwards main() implementation to a call of a MyTargetApp class member function, e.g. MyTargetApp::run()
one for the unit testing, that calls a TestRunner, and contains all of the test suite and fixture classes (including these for the MyTargetApp).
The target application project can be configured without linking against the cppunit library and your test suite/fixture implementations (which will certainly reduce the final artifacts footprint).
However, a cpp project can have ONLY ONE main.
Another option is to use just one project with a narrow main() function that can be compiled using a conditional preprocessor statement (as from their example on the TestRunner class):
#ifdef TESTING
int runUnitTests(int argc, char* argv[]);
#endif
int main(int argc, char* argv[]) {
#ifdef TESTING
// run the unit tests
// -----------------------------------------------------------------
return runUnitTests(argc,argv);
#else
// run the application
// -----------------------------------------------------------------
MyTargetApp app;
return app.run(argc,argv);
#endif
#ifdef TESTING
int runUnitTests(int argc, char* argv[]) {
std::string testPath = (argc > 1) ? std::string(argv[1]) : "";
// Create the event manager and test controller
CppUnit::TestResult controller;
// Add a listener that colllects test result
CppUnit::TestResultCollector result;
controller.addListener( &result );
// Add a listener that print dots as test run.
CppUnit::TextTestProgressListener progress;
controller.addListener( &progress );
// Add the top suite to the test runner
CppUnit::TestRunner runner;
runner.addTest( CppUnit::TestFactoryRegistry::getRegistry().makeTest() );
try {
std::cout << "Running " << testPath;
runner.run( controller, testPath );
std::cerr << std::endl;
// Print test in a compiler compatible format.
CppUnit::CompilerOutputter outputter( &result, std::cerr );
outputter.write();
}
catch ( std::invalid_argument &e ) {
// Test path not resolved
std::cerr << std::endl
<< "ERROR: " << e.what()
<< std::endl;
return 0;
}
return result.wasSuccessful() ? 0 : 1;
}
#endif
Then have 2 different project configurations, one that defines TESTING and one without.
One disadvantage of this solution is though, you'll get all the unit test suites and fixtures left in your program, because cppunit macros will register and instantiate them automatically, regardless if the will be called from the execution path.
I am starting to use googlemock with googletest but am getting an SEH exception that I can't figure out.
The error message is:
unknown file: error: SEH exception with code 0xc0000005 thrown in the test body.
I have read some similar questions on SO and elsewhere but am yet to find an answer for such a simple example.
i.e. This is happening on my real code, but I've also reproduced the error on the very simple example below. I am building with MSVC2008.
code that reproduces the error:
#include "gtest/gtest.h"
#include "gmock/gmock.h"
#include <iostream>
using testing::Exactly;
class Production
{
public:
virtual ~Production() {};
virtual void fn() = 0;
};
class ProductionCode : public Production
{
public:
virtual ~ProductionCode() {};
void fn()
{
std::cout << "CALLED ProductionCode::fn" << std::endl;
}
};
class MockProduction : public Production
{
public:
virtual ~MockProduction() {};
MOCK_METHOD0(fn, void());
};
class ProductionUser
{
public:
void methodUnderTest(Production *p)
{
p->fn();
}
};
TEST(ProductionTest, CallTheProductionFunction) {
ProductionCode p;
ASSERT_NO_THROW( p.fn() );
}
TEST(ProductionTest, CallTheMethodUnderTest) {
Production* p = new ProductionCode;
ProductionUser u;
ASSERT_NO_THROW( u.methodUnderTest(p) );
delete p;
}
TEST(ProductionTest, CallTheMethodUnderTestWithMock) {
MockProduction m;
EXPECT_CALL(m, fn())
.Times(Exactly(1));
ProductionUser u;
ASSERT_NO_THROW(u.methodUnderTest(&m));
}
my test output from the console:
[==========] Running 3 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 3 tests from ProductionTest
[ RUN ] ProductionTest.CallTheProductionFunction
CALLED ProductionCode::fn
[ OK ] ProductionTest.CallTheProductionFunction (4 ms)
[ RUN ] ProductionTest.CallTheMethodUnderTest
CALLED ProductionCode::fn
[ OK ] ProductionTest.CallTheMethodUnderTest (2 ms)
[ RUN ] ProductionTest.CallTheMethodUnderTestWithMock
unknown file: error: SEH exception with code 0xc0000005 thrown in the test body.
[ FAILED ] ProductionTest.CallTheMethodUnderTestWithMock (0 ms)
[----------] 3 tests from ProductionTest (10 ms total)
[----------] Global test environment tear-down
[==========] 3 tests from 1 test case ran. (13 ms total)
[ PASSED ] 2 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] ProductionTest.CallTheMethodUnderTestWithMock
1 FAILED TEST
.\simple.cpp(59): ERROR: this mock object (used in test ProductionTest.CallTheMe
thodUnderTestWithMock) should be deleted but never is. Its address is #000000000
014F800.
ERROR: 1 leaked mock object found at program exit.
Press any key to continue . . .
I am using my own main function as follows:
#include "gtest/gtest.h"
#include "gmock/gmock.h"
int main(int argc, char** argv) {
// The following line must be executed to initialize Google Mock
// (and Google Test) before running the tests.
::testing::InitGoogleMock(&argc, argv);
return RUN_ALL_TESTS();
}
I am guessing that I am making a pretty basic mistake here, Can anyone see where I am going wrong?
Thanks!
[Original Edited to make code & console output match]
I think you could force gtest to don't cloak the exact exception (what might be done using the code:
::testing::GTEST_FLAG(catch_exceptions) = false;
or the same from the command line)
And if then you use a debugger, you easily get the stack. Or even if you don't, I expect *nix-like OS to write core file
I met the same problem when I compiled the gmock as DLL and linked it in another project.
After a lot of try, I found the reason is:
You have to compile the gmock and your project in the same configuration!
That means you have to compile the gmock in DEBUG(RELEASE) configuration, if you want to link it in the DEBUG(RELEASE) mode. If not, the
unknown file: error: SEH exception with code 0xc0000005 thrown in the test body.
always occurs.
I hope my experience could help you, though you may encounter this problem in a different scene.
I was getting this error because I was dereferencing a null pointer.
I've just learned about value-parametrized unit tests in googletest and would like to use them in my project.
I wrote a simple parametrized test.
Header:
#include <gtest/gtest.h>
namespace EnsembleClustering {
class ParametrizedGTest: public testing::TestWithParam<int> {
public:
ParametrizedGTest();
virtual ~ParametrizedGTest();
};
} /* namespace EnsembleClustering */
Source:
#include "ParametrizedGTest.h"
namespace EnsembleClustering {
ParametrizedGTest::ParametrizedGTest() {
// TODO Auto-generated constructor stub
}
ParametrizedGTest::~ParametrizedGTest() {
// TODO Auto-generated destructor stub
}
TEST_P(ParametrizedGTest, testParameter) {
int n = GetParam();
EXPECT_EQ(n, GetParam());
}
INSTANTIATE_TEST_CASE_P(ParametrizedGTestInstance,
ParametrizedGTest,
::testing::Values(100));
} /* namespace EnsembleClustering */
Now, when I run googletest as usual, the program crashes without any output. The gdb stack trace is
EnsembleClustering-D [C/C++ Application]
EnsembleClustering
Thread [1] (Suspended : Signal : EXC_BAD_ACCESS:Could not access memory)
__gnu_debug::_Safe_sequence_base::_M_attach_single() at 0x100528add
__gnu_debug::_Safe_sequence_base::_M_attach() at 0x100528a74
__gnu_debug::_Safe_iterator_base::_M_attach() at 0x100528bfe
__gnu_debug::_Safe_iterator_base::_Safe_iterator_base() at safe_base.h:90 0x1000016e9
__gnu_debug::_Safe_iterator<__gnu_cxx::__normal_iterator<testing::internal::ParameterizedTestCaseInfoBase**, std::__cxx1998::vector<testing::internal::ParameterizedTestCaseInfoBase*, std::allocator<testing::internal::ParameterizedTestCaseInfoBase*> > >, std::__debug::vector<testing::internal::ParameterizedTestCaseInfoBase*, std::allocator<testing::internal::ParameterizedTestCaseInfoBase*> > >::_Safe_iterator() at safe_iterator.h:154 0x100002e9c
std::__debug::vector<testing::internal::ParameterizedTestCaseInfoBase*, std::allocator<testing::internal::ParameterizedTestCaseInfoBase*> >::begin() at vector:207 0x100001fbe
testing::internal::ParameterizedTestCaseRegistry::GetTestCasePatternHolder<EnsembleClustering::ParametrizedGTest>() at gtest-param-util.h:574 0x1000025b0
EnsembleClustering::ParametrizedGTest_testParameter_Test::AddToRegistry() at ParametrizedGTest.cpp:22 0x100001d3f
__static_initialization_and_destruction_0() at ParametrizedGTest.cpp:22 0x100001349
_GLOBAL__sub_I_ParametrizedGTest.cpp() at ParametrizedGTest.cpp:32 0x100001424
<...more frames...>
gdb
Am I doing something wrong or is this a bug in googletest? Can you reproduce this error?
EDIT: I am on Mac OS X 10.8.
From looking at the source code of gtest the only case if there are no parametrized tests available is on Windows using VC7.1 with disabled exceptions:
// We don't support MSVC 7.1 with exceptions disabled now. Therefore
// all the compilers we care about are adequate for supporting
// value-parameterized tests.
#define GTEST_HAS_PARAM_TEST 1
So, you'll need to check how your MinGW was built and probably update it? And can you run the gtest unit tests to see if they execute the typed parameters test?
More information on MinGW:
On their FAQ they report that when using MinGW the following compile option for building gtest is required: PATH/TO/configure CC="gcc -mno-cygwin" CXX="g++ -mno-cygwin".
Complete Example:
#include <gtest/gtest.h>
namespace EnsembleClustering {
class ParametrizedGTest: public testing::TestWithParam<int> {
public:
ParametrizedGTest();
virtual ~ParametrizedGTest();
};
ParametrizedGTest::ParametrizedGTest() {
}
ParametrizedGTest::~ParametrizedGTest() {
}
TEST_P(ParametrizedGTest, testParameter) {
int n = GetParam();
EXPECT_EQ(n, GetParam());
}
INSTANTIATE_TEST_CASE_P(ParametrizedGTestInstance,
ParametrizedGTest,
::testing::Values(100));
} /* namespace EnsembleClustering */
int main(int argc, char* argv[]) {
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
I compiled this code using the following compiler call on Mac OS X 10.8:
g++ -IGTEST_INCLUDE_DIR -LGTEST_LIB_DIR -lgtest -o tt2 tt2.cpp
Where GTEST_INCLUDE_DIR and GTEST_LIB_DIR are the path where header and library files are stored. When you compile and execute, what happens?
Thanks #ChristianStaudt and #grundprinzip
I would like to point future readers to following link that explains this problem.
http://libcwd.sourceforge.net/reference-manual/group__enable__glibcxx__debug.html
This is a link to the documentation for GLIBCXX_DEBUG flag. It states the following important points.
"Note that this flag changes the sizes and behavior of standard class templates such as std::vector, and therefore you can only link code compiled with debug mode and code compiled without debug mode if no instantiation of a container is passed between the two translation units."
"When to use it
It is a good idea to use this if you suspect problems related to iterators."
Now, if you look at the stack trace posted originally, the crash happens due to vector<testing::internal::ParameterizedTestCaseInfoBase*> as gtest tries to get an iterator on this container, using begin() method.
In my case, gtest lib was compiled without GLICXX_DEBUG flag, but my test code was compiled with this flag. The test code worked like a charm when I compiled without this flag.
Does anybody know how to register my custom exception translator when using auto test cases in Boost.Test? I've found some examples (very few actually), but they do not show how to use this feature with auto test cases which are the biggest advantage of boost.test in my opinion. My example test suite:
#define BOOST_TEST_MODULE StateMachineTest
#define BOOST_TEST_DYN_LINK
#include <boost/test/unit_test.hpp>
BOOST_AUTO_TEST_SUITE (FirstTest);
BOOST_AUTO_TEST_CASE (testBasic)
{
BOOST_CHECK (true);
}
BOOST_AUTO_TEST_SUITE_END ();
Thanks in advance.
(Note: I'm still using Boost 1.34.1)
Regardless of the AUTO_TEST_CASE feature, to register custom exception handlers you need to implement the init_unit_test_suite main function. (You do not need to register any of your auto tests there.)
All my unit tests projects use a ut_main.cpp file that contains (roughly) the following:
(This is in addition to all the other cpp files containing the actual auto tests.)
void translate_mfc_exception(CException* pMfcEx) {
...
BOOST_ERROR(msg);
}
// ...
using namespace ::boost::unit_test;
test_suite* init_unit_test_suite(int argc, char* argv[])
{
// Initialize global Handlers:
unit_test_monitor.
register_exception_translator<CException*>( &translate_mfc_exception );
// Return dummy suite to make framework happy:
test_suite* test = BOOST_TEST_SUITE( "Empty Test Suite" );
return test;
}
This should be all you need in addition to your auto test cases.
Alternatively you can register translator in global fixture