I'm using meson and ninja as build system in my C++ project and I've configured catch2 as testing framework.
I was wondering how to perform code coverage with the tests that I've written.
I read this page, https://mesonbuild.com/Unit-tests.html but seems pretty unclear to me, can anybody help?
You should use one of coverage related targets: coverage-text, coverage-html, coverage-xml as described here. Or just coverage that tries all of these if possible:
$ ninja coverage -C builddir
Results are written to ./builddir/meson-logs directory.
Note that to produce html coverage reports you need lcov and genhtml binaries which are installed by lcov package.
Related
I have to do unit test coverage analysis on a large project with hundreds of c++ sources. It can be built only on Linux with an .sh script, and it doesn't compile the sources with gcov in mind, and it has multiple makefiles. I know this isn't very much info, but what would be a good approach to this?
I have a simple module to test with a few inline pa_ounit tests, i've setup the directory in the oasis style and got it all to build.
For a reference I've been using: https://github.com/janestreet/textutils
How would one execute the unit-tests for the above repo? I'm assuming there's an executable .ml file to write but what goes in this, how it is it built and does it extend the tests described at the module level in any way?
I've read the docs for pa_ounit and they just make me more confused ha.
As pa_ounit readme says, run the executable that contains tests with inline-test-runner argument.
Even without pa_ounit (when using plain OUnit), the file with tests is compiled and then executed. You should probably try OUnit itself before you start using the syntax extension so you can get the feel of the system.
OASIS, a popular build automation tool, allows you to build tests and run them with "make test" easily. See https://ocaml.org/learn/tutorials/setting_up_with_oasis.html#Tests
I am trying to use GoogleTest and bullseye for testing C application code. I am in a Red Hat Linux environment using an i686-type processor. I got the gtest and bullseye libraries. The process for writing stubs and test code is also understandable. The step where I'm facing issues is making changes in the makefile to get the binary and coverage file.
I went through the official documentation available for gtest and bullseye, but I'm not able to quite grasp the process.
If anybody has experienced the same situation and finally got the walk-through, please share with me. It'll be very helpful.
For googleTest, just add library and include path, add -I${GTEST_DIR}/include -L${GTEST_DIR}/lib/.libs/libgtest.a when doing gcc compiling.
Or if you are using automake to generate of Makefile, add two lines to Makefile.am which can be used by automake also works.
autotest_LDADD=#LIBS# ${GTEST_DIR}/lib/.libs/libgtest.a
autotest_CXXFLAGS=#CXXFLAGS# -I${GTEST_DIR}/include
The project name is autotest, and ${GTEST_DIR} should be a environment variable before you compile.
Never used bullseye, but it should be close to this.
I have a small, but non-trivial project, which for architectural reasons is built as three separete projects, they are inter-dependent, so unless I'm particularly focused, or improving test-coverage having spotted a hole, it makes sense for me to work from the proejct root.
The layout is as such:
/CMakeLists.txt
/build/
/src/command-line-application/
/src/command-line-application/CMakeLists.txt
/src/command-line-application/build/
/src/command-line-application/src/
/src/command-line-application/tests/
/src/command-line-application/include/
/src/vlc-plugin/
/src/vlc-plugin/src/
/src/libmyproject/
/src/libmyproject/CMakeLists.txt
/src/libmyproject/build/
/src/libmyproject/src/
/src/libmyproject/tests/
/src/libmyproject/include/
/src/libmyotherproject/
/src/libmyotherproject/CMakeLists.txt
/src/libmyotherproject/build/
/src/libmyotherproject/src/
/src/libmyotherproject/tests/
/src/libmyotherproject/include/
A word on the architecture, libmyproject is the real meat of my application, it's built this way because a CLI is a horrible way to ship code to end-users, as a library, it is also used from C# and Objective-C applications. (and all that works as expected)
The libmyotherproject is some platform specific support code, not directly connected to libmyproject, it has a few unit tests.
The vlc-plugin isn't important here, except to show that not everything in /src/*/ has unit tests.
My workflow is typically to hack on the CLI app until something useful crops up, and then refactor it into the library, and make sure it's portable.
When I'm working in /src/*/build/, typically running cmake ../ && make && ctest --output-on-failure, everything works.
When I'm working in /build, and run cmake, the individual components are built correctly (using add_subdirectories()) from CMake, but CTest does not recursively find the tests.
The documentation for CTest is a little unhelpful in what you should do:
USAGE
ctest [options]
DESCRIPTION
The "ctest" executable is the CMake test driver program. CMake-generated build trees created for
projects that use the ENABLE_TESTING and ADD_TEST commands have testing support. This program will
run the tests and report results.
I would have expected since the ADD_TEST() calls live in /src/libmyotherproject/tests/CMakeLists.txt, that they would be run? (They are at least compiled when I run cmake from /build/)
I hope I have been able to provide enough informaton, thank you.
Put
include(CTest)
in your top level CMakeLists.txt file before you make any add_subdirectory calls.
That will call enable_testing for you, and also set things up if you ever want to run a ctest dashboard script on the project to send results to a CDash server.
Is it possible to use gcov for coverage testing of multi-threaded applications?
I've set some trivial tests of our code-base up, but it would be nice to have some idea of the coverage we're achieving. If gcov isn't appropriate can anyone recommend an alternative tool (possible oprofile), ideally with some good documentation on getting started.
We've certainly used gcov to get coverage information on our multi-threaded application.
You want to compile with gcc 4.3 which can do coverage on dynamic code.
You compile with the -fprofile-arcs -ftest-coverage options, and the code will generate .gcda files which gcov can then process.
We do a separate build of our product, and collect coverage on that, running unit tests and regression tests.
Finally we use lcov to generate HTML results pages.
Gcov works fine for multi-threaded apps. The instrumentation architecture is properly serialized so you will get coverage data of good fidelity.
I would suggest using gcov in conjunction with lcov. This will give you great reports scoped from full project down to individual source files.
lcov also gives you a nicely color coded HTML version of your source so you can quickly evaluate your coverage lapses.
I have not used gcov for multi-threaded coverage work. However, on MacOS the Shark tool from Apple handles multiple threads. It's primarily a profiler, but can do coverage info too.
http://developer.apple.com/tools/sharkoptimize.html