We have a project using C#, C++/Cli and some native C++ code. We use TeamCity for building and testing.
We run the tests using vstest.console (VS2012 test runner).
For managed code, dotCover (which is integrated into TeamCity) is used for code coverage. However, it doesn't work with native C++ code (which is to be expected).
How do I get code coverage results our unit tests for the native C++ parts into TeamCity? Ideally, the solution would be free.
We use Bullseye Coverage for C++ code coverage. We then use the provided covxml tool to convert the binary coverage files into an XML file, which we then read out a bunch of useful attributes for function and conditional coverage (e.g. fn_total, fn_cov, cd_total, cd_cov) and provide these to TeamCity via the statistics service messages using the predefined coverage keys.
It was a bit of work to set up, but I think TeamCity still has no support for any C++ coverage tool, so our solution still works well for us years later.
Edit: I've uploaded the XML parsing code for our in-house tool to a Gist.
We should select tool for our C++ code.
Is there a same tool or the tools of the same company for unit test framework
and code coverage tests?
Our aim if the both tool can work compatibly.
Sorry for my english
Thank you very much
For code coverage you can use the lcov tool and as Unit test Framework you can take a look at the Boost library
I used both CxxTest and GoogleTest and both do the job, with a main difference that the former is using pre-build script to generate the test driver code and the later uses static registration of tests which the driver later uses.
For coverage under Windows Bullseye Cover is a great product.
I come from the world of Java and JUnit. I made a demonstration of Hudson and all what I achieve there with JUnit among other things. I would like to do the same with C++ code on an embedded device but can't find where to start.
THe project is compiled with iccarm.exe (IAR compiler) Right now the output is converted into an image file using romutil.exe to be flashed to the ARM9 board.
I tried to follow this tutorial: http://netbeans.org/kb/docs/cnd/c-unit-test.html but I'm having issues trying to figure out how to port it to my case.
Can I run unit tests on the C++ code outside of the device? (i.e. for doing it with Hudson to gather reports and so on)
Can I turn the output into an exe? (looks like Netbeans expects it to be)
What's the most appropriate unit framework for my case? (CppUnit, CUnit, etc)
Any help/direction is more than welcome.
1) You could build a cross-compiler, and then instead of using iccarm.exe, use the cross compiler. There are lots of tutorials on the net how to build a cross compiler using gcc. Then instead of building for the target (using cross compiler) build for your host using normal compiler.
3) Whatever you like. cppunit, google unit tests, etc
I have a unique combination of platforms in my solution that has stumped me.
We need to generate code coverage statistics for unit tests that we're writing for our Windows CE code. We're using Visual Studio 2008 to write the CE code, of course, because VS 2010 doesn't support smart devices.
Unfortunately, MSTest won't instrument CE assemblies for collecting coverage data, so we're compiling our test assembly as a regular Win32 library and using it to call the CE code we need to test. This is working fine on a desktop installation of Visual Studio, but we also need to incorporate this into our nightly build on the server.
I have tried everything I can think of to get this to work. It's frustrating because I can manually do everything on my workstation, I just can't get TFS 2010 to do it for me automatically.
Is anybody else writing unit tests for Windows CE code? I can't believe I'm the only one!
Thanks in advance for whatever tips or tricks you might be able to offer...
If you are writing C++ or C# code, our SD Test Coverage Tools can be used to instrument your source code in a way that is compatible with embedded devices. These tools provide automated instrumentation of your source code, and instrumentation data collection procedures in source code so that you can control how the instrumentation data is collected and exactly how it is dumped, which is ideal for embedded software. A GUI display provides a programmer view of the coverage data imposed on the source code, and an XML report (which you can format at HTML using XSLT) provides summary data.
I have a very large code base that contains extensive unit tests (using CppUnit). I need to work out what percentage of the code is exercised by these tests, and (ideally) generate some sort of report that tells me on a per-library or per-file basis, how much of the code was exercised.
Here's the kicker: this has to run completely unnatended (eventually inside a continuous integration build), and has to be cross platform (well, WIN32 and *nix at least).
Can anyone suggest a tool, or set of tools that can help me do this? I can't change away from CppUnit (nor would I want to - it kicks ass), but otherwise I'm eager to hear any recommendations you might have.
Cheers,
Which tool should I use?
This article describes another developers frustrations searching for C++ code coverage tools. The author's final solution was Bullseye Coverage.
Bullseye Coverage features:
Cross Platform Support (win32, unix, and embedded), (supports linux gcc compilers and MSVC6)
Easy to use (up and running in a few hours).
Provides "best" metrics: Function Coverage and Condition/Decision Coverage.
Uses source code instrumentation.
As for hooking into your continuous integration, it depends on which CI solution you use, but you can likely hook the instrumentation / coverage measurement steps into the make file you use for automated testing.
Testing Linux vs Windows?
So long as all your tests run correctly in both environments, you should be fine measuring coverage on one or the other. (Though Bullseye appears to support both platforms). But why aren't you doing continuous integration builds in both environments?? If you deliver to clients in both environments then you need to be testing in both.
For that reason, it sounds like you might need to have two continuous build servers set up, one for a linux build and one for a windows build. Perhaps this can be easily accomplished with some virtualization software like vmware or virtualbox. You may not need to run code coverage metrics on both OSs, but you should definitely be running your unit tests on both.
If you can use GNU GCC as your complier, then the gcov tool works well. It's very easy to fully automate the whole process.
If you are using the GCC toolchain, gcov is going to get you source, functional, and branch coverage statistics. gcov works fine for MinGW and Cygwin. This will allow you to get coverage statistics as well as emitting instrumented source code that allows you to visualize unexecuted code.
However, if you really want to hit it out of the park with pretty reports, using gcov in conjunction with lcov is the way to go. lcov will give you bar reports scoped to files and directories, functional coverage statistics, and color coded source file browsing to show coverage (green means executed, red means not...).
lcov is easy on Linux, but may require some perl hacking on Cygwin. I personally have had some problems executing the scripts (lcov is implemented in perl) on Windows. I've gotten a hacked up version to work, but be forewarned.
Another approach is doing the gcov emit on windows, and doing the lcov post processing on Linux, where it will surely work out of the box.
Check out our SD C++ Test Coverage tool. It can be obtained for GCC, and for MSVC6.
It has low overhead probe data collection, a nice display of coverage data overlayed on your code, and complete report generation with rollups on coverage across the method/class/file/directory levels.
EDIT: Aug 2015: Now supports GCC5 and various MS dialects through Visual Studio 2015. To use these tools under Linux, you need Wine, but there the tools provide Linux-native sh scripting and a Linux/Java based UI, so the tool feels like a native Linux tool there.
I guess I should have specified the compiler - we're using gcc for Linux, and MSVC 6 (yeah I know, it's old, but it works (mostly) for us) for WIn32.
For that reasons, gcov won't work for our Win32 builds, and Bullseye won't work for our Linux builds.
Then again maybe I only need coverage in one OS...