I have been looking for specific properties from a unit test framework in c++ but I have struggled find all of the following properties in a framework:
Creation of Stubs (only in c++test).
Has Code-Coverage tool (only in c++test).
VS integration (Common enough)
Fixtures (very common in most c++ frame works)
Integration with a continuous build system (unable to find it at all in any Framework)
Predicate support (Very rare, only able to find that boost has this)
Anyone know of any frameworks which have all these properties (or most of them)?
GoogleMock from Google or Isolator++ from Typemock. Isolator++ is a mocking framework which works with several UT frameworks.
You don't need that as part of a UT framework (you can instrument binaries and get coverage in VS).
Isolator++ has VS integration AFAIK.
Like you are saying this is very common, so it will be hard to not find it in mainstream frameworks.
The continuous build system should support your framework if you really want that (TeamCity supports a couple of them for example). Otherwise you can always wrap it with some scripts.
Not sure what you require here.
HTH.
Related
OpenCV provides unit tests for its important functions. In order to do that, it provides a unit test framework, which is built on top of gtest. However, the documentation on this test framework is really limited as it is not supposed to be used outsider OpenCV. I tried to manage to use OpenCV test framework based on this question. However, I cannot make the test framework only run specific test functions I wrote due to lack of documentation. Any ideas?
I finally find the solution: use --gtest_filter= option from the command line of the test program. This comes from the fact that OpenCV unit test is an improved version of GTEST.
Visual Studio
I use NUnit in my project with unit tests. But I need some tests to write with using JustMock and NSubstitute frameworks. I want to have one project with the tests for each my tested project. I.e. I don't want to have the individual project with tests for each unit framework (NUnit, JustMock and NSubstitute). Will I have the problems (for example, conflicts with their test adapters) if my tests will be in a single project?
NUnit is a testing framework. NSubstitute is a mocking framework. From the name, I suppose JustMock is also a mocking framework. Testing frameworks and mocking frameworks are different things, so they don't generally conflict.
Between two different mocking frameworks, it's possible but not likely to have conflicts. I think most of them would show up at compile time, in case the framework uses the same names and you are using both namespaces in a file.
It will be easier to answer, however, if you have some specific concerns.
We are planning to integrate our native c++ projects into a maven build process. Further we want to formulate unit tests that are run automatically using the standard maven syntax (as for java unit tests) also for the c++ projects. Is this possible with c++ unit testing frameworks and if yes, which framework integrates well with maven ?
I would suggest to take a deep look into the maven-nar-plugin in relationship with Boost library which should fit your needs.
I have some projects developed with C++ builder XE.
I would like to add some unit test, but the DUnit framework installed is nice for Delphi, but the integration with C++ builder is not so good (and very limited).
What other xUnit framework can I easily work with ?
In your case I'd start by asking Embarcadero for assistance. They want to fully support the developers who use their stuff, and automated unit testing is really critical to keeping them happy.
Until then, CppUnit works on any C++ code, but does not really integrate all that well with IDEs. The approach we've used is to create a new project to contain the tests, and have its linker include the path to the existing production project's .OBJ files. We set up a a project dependency so the test project depends on the production project.
In the Test project, we'll use different main.cpp files, one each for Debug and Release, and use conditionals to include/exclude the appropriate one from the Debug and Release builds.
For some "fake" integration, at least as far as running the tests go, in the DebugMain.cpp we'll load up the MFC TestRunner GUI, so the developer can click to select the tests they want to execute. In the ReleaseMain.cpp, we'll use the command line test runner, using the CompilerOutputter object which will let the build process know of success or failure. The output is also compatible with IDEs that interpret stuff like that, so you can click on a failed test report in the Output window, and the IDE takes you to the failing test assertion.
What are the best policies for unit testing build files?
The reason I ask is my company produces highly reliable embedded devices. Software patches are just not an option, as they cost our customers thousands to distribute. Because of this we have very strict code quality procedures(unit tests, code reviews, tracability, etc). Those procedures are being applied to our build files (autotools if you must know, I expect pity), but if feels like a hack.
Uh... the project compiles... mark the build files as reviewed and unit tested.
There has got to be a better way. Ideas?
Here's the approach we've taken when building a large code base (many millions of lines of code) across more than a dozen platforms.
Makefile changes are reviewed by the build team. These people know the errors people tend to make in our build environment, and they are the ones who feel the brunt of it when a build breaks, so they're motivated to find issues.
Minimize what needs to go in a Makefile, so there are fewer opportunities for error. We have a layer on top of make, that generates the Makefile. A developer just has to indicate in the higher-level file, using tags, that for example a given target is a shared library or a unit test. Usually a target is defined on one line, which then results in multiple settings/targets in the generated Makefile. Similar things could be done with build tools like scons that allow one to abstract away things like platform-specific details, making targets very simple.
Unit tests of our build tool. The tool is written in Perl, so we use Perl's Test::More unit test framework there to verify that the tool generates the correct Makefile given our higher-level file. If we used something like scons instead, I'd use their testing framework.
Unit tests of our nightly build/test scripts. We have a set of scripts that start nightly builds on each platform, run static analysis tools, run unit tests, run functional tests, and report all results to a central database. We test the various scripts individually, mostly using the shunit2 unit-testing framework for sh/bash/ksh/etc.
End-to-end tests of our build/test process. I am working on an end-to-end test that operates on a tiny source tree rather than our production code, since the latter can take hours to build. These tests are mainly aimed at verifying that our build targets still work and report results into our central database even after, for example, upgrading our code coverage tool or making changes to our build scripts.
Have your build file to compile a known version of your software (or simpler piece of code that is similar from a build perspective) and compare the result obtained with your new build tools to a expected result (built with a validated version of the build tools).
In my projects build-files don't change very often. Even more, I can reuse build-files from earlier projects, only changing some variables (that I moved to an easy to recognize section). That's why for me it is unneeded to unit-test the build-files. That can be different in other projects.