What are the most modern approaches to unit testing for the C++ language? The class of the languages with bigger introspection power (like Python) have unit testing frameworks that are somehow more natural to use. Unit tests can be defined more easily. In comparison, the classical CppUnit (based on JUnit) seems to take very conservative approach. Is there anything newer and better that uses the specific power of C++ (or even C++11) to make the life easier?
I have been using CppUnit framework for a while in rather simplistic way for parts of the project on Windows native C++ (Visual Studio 2005 and 2010). We did not choose the Test Driven Development approach earlier, because there already was a lot of legacy code, and we had found it quite difficult to add tests for it. We had to refactor the application, but adding all the nice tests would be time consuming even in the case.
Recently, we have switched to Visual Studio 2013 (because of the C++11 standard implementation), and we are going to start new, rather long-term project.
Having the previous good (small) experience with unit testing, I would like to try the Test Driven Development approach. As the project is not a tiny one (expected size about the same as the older one, i.e. about 200 k lines of code), I prefer rather more easy (but not less capable) framework.
There is a chance the new project could lead to cross-platform implementation (Windows and Linux). There is a unit testing support in Visual Studio 2013, but I have no experience with it and how would it fit with the cross-platform.
So far, I have found the list of unit testing frameworks for C++. However, one cannot see how they differ in principle. I currently have three candidates (conservative choice):
Boost -- the probable candidate; testbed for C++ standards; hence it is likely it is to be widely accepted; probably the biggest user group. It seems to be more advanced than CppUnit.
CppUnit -- I know it, but it writing all of the code around is not a pleasure.
Visual Studio 2013 built-in -- new to me, somehow can generate the skeletons probably.
Anyway, it seems that all of the three use similar approach. Possibly the VS2013 supports generating the code, but it does not mean it leads to anything simpler.
Is there any radically new approach?
The only test framework worth considering:
Catch
For an introduction to the lib, see also here and here
It's easy to use (a header-only lib consisting of just one header), portable, and has by far the simplest, cleanest syntax of any C++ unit testing framework.
And unlike other libraries, you don't need to remember two dozen different macros f or different types of assertions.
You just use REQUIRE:
int one = 1;
REQUIRE( one == 2 );
which, through some clever operator overloading, will show both the original expression and the expanded argument values in the output:
test.cc(7): FAILED:
REQUIRE( one == 2 )
with expansion:
1 == 43
Compared to this, every other framework is a chore to use IMO.
I used to use Boost.Test before I found this, but that was a lot more cumbersome to set up and use. We use CppUnit at work, and that seems to be designed to be as fragile and painful as possible.
I've looked briefly at the VS2013 test framework, but haven't tried it, and it looks tolerable, but very much like it's emulating "the old guard". It doesn't really try to be cleaner, easier or better than CppUnit, Boost.Test and all the other ones that came before Catch. So I'd say don't bother with it. Tests should be easy to write (and to understand), and Catch is lightyears ahead of every other framework I've seen on that front.
I've been using theVisual Studio 2013 built in test framework for about 6 weeks now and really like it. The integration is excellent and it's very easy to pick up. If you're working on a project that only targets Windows then I thoroughly recommend it.
There are several major unit tests frameworks, but as far as I know all of them needs to process the source code in some way.
For example, I am using cxxunit, and it requires the unit tests to be processed using some python script. The problem with this is that it increases the build time.
Does anyone know of a unit testing framework for C++ code that doesn't preprocess the unit tests code?
PS: I need it for linux, but it is ok if it is multi platform.
You could use Boost.Test. I've had good experiences with it. It does not require any special preprocessing.
You could use google test framework. You need just build library source code one time. Then you can create your tests as .cpp files, then compile and link them with gtest and needed project libraries as ordinary c++ sources.
Besides that, it is multiplatform.
I'm happy with googletest.
CPPUnit is my personal choice at the moment, and is in plain C++.
I've been using TestDog. You construct your tests using the code to test and it produces HTML output summaries.
I'd recommend considering Andrew Marlow's FRUCTOSE http://fructose.sourceforge.net/ in your evaluation too... he's a very thorough and professional developer who compared existing offers carefully before crafting his own. See also an ACCU article disucssing the library: http://accu.org/index.php/journals/1305
Is there any way to generate automated unit test cases for c++ code written using VS2005. I do some basic R&D and found that there is a support for automated testing in VS2005 but its only for Managed Code, so i am looking for something specific to:
"Automatic generation of Test cases of Native C++ Code"
It doesn't matter whether its a plug-in that works with VS2005 or any standalone application. But it is preferred to have some plug-in sort of solution.
I can't tell if you are asking about a unit testing framework (if yes, try cppUnit), or if you really want tests code-generated. If the latter, I imagine the answer is no. Usually a capability like that is linked to reflection and/or built-in design-by-contract capabilities (e.g Eiffel or C# with .net 4.0).
When I create a program using C# and VS2008 , then I can create a test case just by Rightclicking on the method. But I am not sure If I can create the tests in the same way if it is a C++ project.
Due to the lack of reflection in C++, you propably wont be able to have these kind of unit tests, VS provides.
I'm pretty sure you can't. You can create a C++/CLI test project and tests manually though. The IDE will create a C++/CLI test class with stubs etc. for you.
I agree with the answers above, just adding...or use the boost library:
http://www.boost.org/doc/libs/1_35_0/libs/test/doc/components/utf/index.html
You might want to take a look at CppUnit or googletest for unit testing with C++. You won't have the IDE generation of test cases, but there are unit testing frameworks out there.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
I use the Boost Test framework for my C++ code but there are two problems with it that are probably common to all C++ test frameworks:
There is no way to create automatic test stubs (by extracting public functions from selected classes for example).
You cannot run a single test - you have to run the entire 'suite' of tests (unless you create lots of different test projects I guess).
Does anyone know of a better testing framework or am I forever to be jealous of the test tools available to Java/.NET developers?
I've just pushed my own framework, CATCH, out there. It's still under development but I believe it already surpasses most other frameworks.
Different people have different criteria but I've tried to cover most ground without too many trade-offs.
Take a look at my linked blog entry for a taster. My top five features are:
Header only
Auto registration of function and method based tests
Decomposes standard C++ expressions into LHS and RHS (so you don't need a whole family of assert macros).
Support for nested sections within a function based fixture
Name tests using natural language - function/ method names are generated
It doesn't do generation of stubs - but that's a fairly specialised area. I think Isolator++ is the first tool to truly pull that off. Note that Mocking/ stubbing frameworks are usually independent of unit testing frameworks. CATCH works particularly well with mock objects as test state is not passed around by context.
It also has Objective-C bindings.
[update]
Just happened back across this answer of mine from a few years ago. Thanks for all the great comments!
Obviously Catch has developed on a lot in that time. It now has support for BDD style testing (given/ when/ then), tags, now in a single header, and loads of internal improvements and refinements (e.g. richer command line, clear and expressive output etc). A more up-to-date blog post is here.
Take a look at the Google C++ Testing Framework.
It's used by Google for all of their in-house C++ projects, so it must be pretty good.
http://googletesting.blogspot.com/2008/07/announcing-new-google-c-testing.html
http://code.google.com/p/googletest
Boost.Test does allow to run test case by name. Or test suite. Or several of them.
Boost.Test does NOT insists on implementing main, though it does make it easy to do so.
Boost.Test is NOT necessary to use as a library. It has single header variant.
I just responded to a very similar question. I ended up using Noel Llopis' UnitTest++. I liked it more than boost::test because it didn't insist on implementing the main program of the test harness with a macro - it can plug into whatever executable you create. It does suffer from the same encumbrance of boost::test in that it requires a library to be linked in. I've used CxxTest, and it does come closer than anything else in C++-land to automatically generating tests (though it requires Perl to be part of your build system to do this). C++ just does not provide the reflection hooks that the .NET languages and Java do. The MsTest tools in Visual Studio Team System - Developer's Edition will auto-generate test stubs of unmanaged C++, but the methods have to be exported from a DLL to do this, so it does not work with static libraries. Other test frameworks in the .NET world may have this ability too, but I'm not familiar with any of those. So right now we use UnitTest++ for unmanaged C++ and I'm currently deciding between MsTest and NUnit for the managed libraries.
I'm a big fan of UnitTest++, it's very lightweight, but does the job. You can run single tests there easily.
Great question! A few years ago I looked around forever for something worth using and came up short. I was looking for something that was very lightweight and did not require me to link in some libraries... you know something I could get up and running in minutes.
However, I persisted and ended up running across cxxtest.
From the website:
Doesn't require RTTI
Doesn't require member template functions
Doesn't require exception handling
Doesn't require any external libraries (including memory management, file/console I/O, graphics libraries)
Is distributed entirely as a set of header files (and a python script).
Wow... super simple! Include a header file, derive from the Test class and you're off and running. We've been using this for the past four years and I've still yet to find anything that I'm more pleased with.
Try WinUnit. It sounds excellent, and is recommended by John Robbins.
I like the Boost unit test framework, principally because it is very lightweight.
I never heard of a unit-test framework that would generate stubs. I am generally quite unconvinced by code generation, if only because it gets obsolete very quickly. Maybe it becomes useful when you have a large number of classes?
A proponent of Test Driven Development would probably say that it is fundamental that you run the whole test suite every time, as to make sure that you have not introduced a regression. If running all the tests take too much time, maybe your tests are too big, or make too many calls to CPU intensive functions that should be mocked out? If it remains a problem, a thin wrapper around the boost unit-tests should allow you to pick your tests, and would probably be quicker than learn another framework and port all your tests.
http://groups.google.com/group/googletestframework, but it's pretty new
I'm using tut-framework
Aeryn is another framework worth looking at
Visual Studio has a built-in unit testing framework, this is a great link to setting up a test project for win32 console application:
http://codeketchup.blogspot.ie/2012/12/unit-test-for-unmanaged-c-in-visual.html
If you are working on a static DLL project it is much easier to set up as other have pointed out external tesing frameworks like GTest and Boost have extra features.
CppUnit was the original homage to JUnit.
Eclipse/JUnit is a solid package for java, and there are C++ extensions/equivalents for both. It can do what you're talking about. Of course, you'd have to change IDEs...
I too am a fan of UnitTest++.
The snag is that the source distribution contains almost 40 seperate files. This is absurd. Managing the source control and build tasks for a simple project is dominated by looking after all these unit testing files.
I have modified UnitTest++ so that it can be integrated with a project by adding one .h and .cpp file. This I have dubbed "cutest". Details are at http://ravenspoint.com/blog/index.php?entry=entry080704-063557
It does not automatically generate test stubs, as asked for in the original question. I cannot help thinking that such a feature would be more trouble than it is worth, generating vast amounts of useless code "testing" accessor functions.
I would imagine automatically stubbing out test functions would be more of a function (of scripts for the framework or) the development environment in question. Supposedly CodeGear's C++Builder applications will quickly generate test code for user functions.
Andrew Marlow's Fructose library's worth checking out... http://fructose.sourceforge.net/
I recall his documents containing a fairly detailed analysis and comparison of other offering at the time he wrote Fructose, but can't find a URL direct to that document.
I'm trying out Igloo, also a header only C++ test suite, even it's two included dependencies are header only.
So, it's pretty straightforward and simple. Besides the included example on github, there's examples and more details at the main site, igloo-testing.org. I'll update this later as I get more experience with it and other frameworks.