I'm confused about how to use gcov. I've got a cmake project that has two test executables which use googletest. I've added the required flags to my cmake script:
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fprofile-arcs -ftest-coverage -g -O0")
And I've recompiled my code (using CLion and g++ 10.1).
Now I switch to the build directory and manually run both test suites. This, according to the docs should generate some files that can be used to generate the coverage report. Now I should be able to run
gcovr .
from the root of the build tree (right?), however the output is this:
(base) ciaran#DESKTOP-K0APGUV:/mnt/d/libOmexMeta/cmake-build-release-wsl-ubuntu1804-gcc101$ gcovr .
------------------------------------------------------------------------------
GCC Code Coverage Report
Directory: .
------------------------------------------------------------------------------
File Lines Exec Cover Missing
------------------------------------------------------------------------------
------------------------------------------------------------------------------
TOTAL 0 0 --%
------------------------------------------------------------------------------
Any idea what I've doing wrong?
#Edit
Also running
gcovr -r . --html --html-details -o example-html-details.html
Works, but generate an empty report
You have to give the source files:
-r , --root
The root directory of your source files. Defaults to ‘.’, the current directory. File names are reported relative to this root.
If you build and run out of sources dir it may fail to find what it needs.
Related
I have been trying to get code coverage working for Qt but im struggling with generating any coverage information
Project
example.pro (contains include for googletest.pri)
gtest_dependency (contains googletest.pri
headerfiles for gtest
testsuite.h
sourcefiles for gtest
test.cpp
testsuite.cpp
main.cpp
All files above are pretty much empty just the code to run the test and one test case.
I included the linker flags in the example.pro
QMAKE_CXXFLAGS += -O0 -g --coverage
QMAKE_LFLAGS += -O0 -g --coverage
The problem is when I build the project it creates all the .gcna .gcno files, but when i execute lcov it says
WARNING: no .gcda files found in ./coverage -skipping!
Finished .info-file creation
Reading tracefile ./coverage/coverage.info
lcov: ERROR: no valid records found in tracefile ./coverage/coverage.info
Reading data file ./coverage/coverage-filtered.info
genhtml: ERROR: cannot read file ./coverage/coverage-filtered.info
Qt Version 5.14.2
Compiler mingw730_64
lcov version 1.14
gcov version 7.5.0
Ok i managed to fix it
If you run into the same problem as me check if you have googletest installed on /usr/include, since I use lcov and so on over the linux bash it couldnt find anything since everything was installed over /mnt/ and it couldnt find gtest since it searches in /usr.
Same goes for qt it couldnt find the files even if included with -L and -I.
I had to install both on /usr/
A recent OS upgrade made my coverage script fail miserably.
lcov 1.13
gcov (GCC) 9.1.1
The part of my CMake that is used to generate coverage data:
if ($ENV{COVERAGE})
message("Setting up for coverage")
enable_testing()
include(CodeCoverage)
setup_target_for_coverage(${PROJECT_NAME}_coverage tests coverage)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} --coverage")
endif ()
The lcov command I issued after building tests:
lcov --capture --directory build/ --output-file coverage.info
Unfortunately now fails with:
Capturing coverage data from build/
Found gcov version: 9.1.1
Scanning build/ for .gcda files ...
geninfo: WARNING: no .gcda files found in build/ - skipping!
Finished .info-file creation
The error message makes sense because there are no .gcda files - only .gcno files. I am not sure if they serve the same purpose and / or can be used with lcov.
I issued nm some_binary | grep gcov and there are a lot of symbols in the form of:
00000000004b3520 d __gcov_._ZZZN6__pstl10__internal15__pattern_walk2IRKNS_9execution2v115parallel_policyEN9__gnu_cxx17__normal_iteratorIPKiSt6vectorIiSaIiEEEENS8_IPiSD_EEZSt9transformIS6_SE_SG_ZN12_GLOBAL__N_150ParallelTransformTest_NoDataShouldReturnEmpty_Test8TestBodyEvEUlRKT_E_ENSt9enable_ifIXsrNS3_19is_execution_policyINSt5decayISK_E4typeEEE5valueET1_E4typeEOSK_T0_SY_SU_T2_EUlRS9_RiE_St17integral_constantIbLb0EEEESU_SX_SY_SY_SU_SZ_T3_S13_IbLb1EEENKUlvE_clEvENKUlSE_SE_E_clESE_SE_
So I guess the CMake is still correctly trying to give me coverage data.
It worked fine on gcc 7 if I recall correctly.
Is there a new solution / CMake flag to issue / lcov flag to issue? Or is it broken right now and there is no workaround? Or perhaps I was doing something odd the whole time?
I believe GCC 9 outputs coverage data as JSON by default now, as mentioned in the change notes.
The gcov tool has changed its intermediate format to a new JSON format.
It also looks like lcov have an open issue for handling this new format.
In CMake tests configuration I added flags to generate codecoverage
IF( "${CMAKE_CXX_COMPILER_ID}" STREQUAL "Clang" OR
"${CMAKE_CXX_COMPILER_ID}" STREQUAL "GNU")
SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g -O0 -fprofile-arcs -ftest-coverage")
SET(CMAKE_C_FLAGS "-g -O0 -fprofile-arcs -ftest-coverage ")
endif()
Each time I run tests - my output is spammed with those messages:
profiling: /xxxx/xxxxxj/projects/build-xxxxx-CLang-Debug/tests/CMakeFiles/xxxxxxxxxxxxxt_ut.dir/tests/gui/ship_design/ut_ship_stats_header.cpp.gcda: cannot merge previous GCDA file: mismatched number of counters (14)
profiling:
/xxxx/xxxxxx/projects/build-xxxxxxxxxxxxxxx-CLang-Debug/tests/CMakeFiles/xxxxxxxxxxxxxxx_ut.dir/tests/gui/ship_design/ut_ship_stats_header.cpp.gcda:
cannot merge previous GCDA file: corrupt arc tag (0x2b8e100f)
profiling:
/xxxx/xxxxxx/projects/build-xxxxxxxxxxxxxxx-CLang-Debug/tests/CMakeFiles/xxxxxxxxxxxxxxx_ut.dir/tests/ut_generate_hex_path.cpp.gcda:
cannot merge previous GCDA file: corrupt arc tag (0x65646f6e)
I don't have to say that this make reading test run results at least difficult. Above problem vanished when I remove all gcda files generated previously. So there are two possible solutions.
I may miss some valuable configuration of code coverage data (ie. call lcov --zerocounters
FIND_PROGRAM( LCOV_PATH lcov )
COMMAND ${LCOV_PATH} -z --directory ${PROJECT_BINARY_DIR}}
I need to add custom target removing all gcda files before test run is committed.
How should I approach this problem?
I think you might be doing something unusual in your workflow, as the problem you're seeing shouldn't happen all of the time. But, what I've added here should help you figure that out, or work around it completely.
Firstly, you should take advantage of CMake's build-types and create your own "coverage type".
## coverage flags
set(CMAKE_CXX_FLAGS_COVERAGE "-g -O0 -fprofile-arcs -ftest-coverage" CACHE STRING "Flags used by the C++ compiler during coverage builds.")
set(CMAKE_C_FLAGS_COVERAGE "-g -O0 -fprofile-arcs -ftest-coverage" CACHE STRING "Flags used by the C compiler during coverage builds.")
set(CMAKE_EXE_LINKER_FLAGS_COVERAGE "-g -O0 -fprofile-arcs -ftest-coverage" CACHE STRING "Flags used for linking binaries during coverage builds.")
set(CMAKE_SHARED_LINKER_FLAGS_COVERAGE "-g -O0 -fprofile-arcs -ftest-coverage" CACHE STRING "Flags used by the shared libraries linker during coverage builds.")
mark_as_advanced(
CMAKE_CXX_FLAGS_COVERAGE
CMAKE_C_FLAGS_COVERAGE
CMAKE_EXE_LINKER_FLAGS_COVERAGE
CMAKE_SHARED_LINKER_FLAGS_COVERAGE)
## Update the documentation string of CMAKE_BUILD_TYPE for GUIs
set(CMAKE_BUILD_TYPE "${CMAKE_BUILD_TYPE}" CACHE STRING "Choose the type of build, options are: None Debug Release RelWithDebInfo MinSizeRel RelWithAssert Coverage." FORCE)
Then create a custom target.
## create our "make coverage" target
add_custom_target(coverage
COMMAND if test ! -d ../output \; then mkdir ../output\; fi
COMMAND find ${CMAKE_BINARY_DIR} -name \*.gcda -delete
COMMAND lcov -b CMakeFiles/ -d . -z
COMMAND lcov -b -d . -c -i -o test_base.info
COMMAND ./env-shell.sh ctest -j2 || true
COMMAND lcov -b CMakeFiles/ -d . -c -o test_run.info
COMMAND lcov -b CMakeFiles/ -d . -a test_base.info -a test_run.info -o test_total.info
COMMAND lcov -o reports.info -r test_total.info '/usr/include/*' '/usr/local/*' '/cvmfs/*' '*/numpy' '/usr/lib/gcc/*' ${p} '${CMAKE_BINARY_DIR}/CMakeFiles/' '${CMAKE_BINARY_DIR}/steamshovel/*'
COMMAND genhtml --ignore-errors source --legend -o ../output/`date +%Y-%m-%d` reports.info
)
Now, taking advantage of out-of-source builds, run cmake in a "coverage" directory parallel to your source.
$ pwd
/home/user/my_project/src
$ mkdir ../coverage
$ cd ../coverage
$ cmake -DCMAKE_BUILD_TYPE=Coverage ../src
[ ... cmake's output here ...]
-- Configuring done
-- Generating done
-- Build files have been written to: /home/user/my_project/coverage
$
Now, build your project, and your tests, and run your "coverage" target.
$ make
$ make test-bins
$ make coverage
[ ... make's output here ... ]
$
The make coverage target we defined in our CMakeLists.txt will:
if it doesn't exist, create an "output" directory parallel to our source and build directories
find and delete all *.gcda files in our current build directory
zero and initialize our lcov counters and output file
run our tests via ctest
"compile" the lcov output and generate our HTML coverage report
Note that you may need to adjust things for your particular project.
At this point I highly recommend automating all of this. Add it to your continuous integration, if you have some, or even set up a cron job so that this runs overnight and you have a fresh new coverage report to start your day. (These examples are from a working project that does a nightly coverage report handled by buildbot.)
I'm struggling to get coverage information for gcov. No errors during compilation and linking, but when I run the executable, no coverage data is produced.
I'm using CMake with a separate build directory, passing flags to the compiler and linker in this way:
add_definitions(--coverage)
set(CMAKE_EXE_LINKER_FLAGS ${CMAKE_EXE_LINKER_FLAGS} " --coverage")
Does the executable expect the source code to be in a specific location?
What do I have to add to my CMakeLists.txt to get things going?
Kind regards,
Bjoern
CMake seems to put the code coverage (*.gcda, *.gcdo) files with the object files of your project. If your executable was named "tester" then they would appear in the following path
${CMAKE_BINARY_DIR}/CMakeFiles/tester.dir/
CMake seems to name the source files in a way that isn't very compatible with gcov though. For example if I had a source file called "mytestprog.cpp" it would be build
mytestprog.cpp.o
mytestprog.cpp.gcda
mytestprog.cpp.gcdno
where as gcov seems to expect
mytestprog.gcda
mytestprog.gcdno
I'm not really sure how to fix it. I've tried using LCov instead and that "appeared" to work but I'm not really sure if it did.
Delcypher pointed out the problem.
Solution 1: you can ask cmake to name object files as main.o instead of main.cpp.o etc. using the undocumented CMAKE_CXX_OUTPUT_EXTENSION_REPLACE switch:
cmake -DCMAKE_CXX_OUTPUT_EXTENSION_REPLACE=ON ...
Solution 2: if you do not need the .gcov files you can call lcov from the build directory:
lcov --capture --directory . --output-file coverage.info
genhtml coverage.info --output-directory out
You will find the coverage information in the out directory in html form.
Not sure where you got --coverage from, but these are the arguments I use on Linux to get coverage information using gcc and gcov:
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fprofile-arcs -ftest-coverage")
set(CMAKE_EXE_LINKER_FLAGS
"${CMAKE_EXE_LINKER_FLAGS} -fprofile-arcs -ftest-coverage")
Here's what gcc --help --verbose has to say about those options:
-ftest-coverage Create
data files needed by "gcov"
-fprofile-arcs Insert
arc-based program profiling code
You don't need to pass --coverage to the linker. --coverage will pass -fprofile-arcs -ftest-coverage to the compiler and -lgcov to the linker.
Are you sure that it isn't creating any gcdo or gcda files? Where are you looking for these files? It should put the gcov file for each object file into the same directory as the object file. Do a find for .gcda files at the top of your build directory. If nothing shows up, gcov might not be getting linked in. Run the following command to see if it is:
nm name_of_binary | grep "gcov"
If it is getting linked in, then gcov might not have permission to write files to where you are running the executable. If it has permission, then I am stumped.
I'm working on larger project which has the following directory layout:
Source
MyA
aa.cpp
ab.cpp
ac.cpp
MyB
ba.cpp
bb.cpp
bc.cpp
MyTest
testaa.cpp
testab.cpp
testac.cpp
testba.cpp
testbb.cpp
testbc.cpp
main.cpp
Build
MyA
aa.o
ab.o
ac.o
libMyA.a (static library)
MyB
ba.o
bb.o
bc.o
libMyB.a (static library)
MyTest
testaa.o
testab.o
testac.o
testba.o
testbb.o
testbc.o
MyTest (executable)
After compiling with -fprofile-arcs -ftest-coverage I execute the MyTest application inside the Build/MyTest directory. As expected there are *.gcno and *.gcda files inside the Build directory. After running gcov inside the MyTest directory different *.gcov files are produced but unfortunately not for everything inside MyA and MyB, although every function is called inside this two libraries. Tried different options but somehow I'm unable to create useful (means correct) *.gcov files with this layout.
If I copy every cpp inside one directory and repeat the steps everything works as expected and the coverage analysis is perfect.
You must specify source files as absolute paths to g++/gcc. Don't use relative paths with ".." or like "foo/bar.cpp", else you'll get errors like "geninfo: WARNING: no data found for XXXX".
Don't include any header files on the command line to g++/gcc. Else you'll get "stamp mismatch with graph file" errors.
So, following should work when having multiple directories:
g++ --coverage -DDEBUG -g3 heyo.cpp /app/helper/blah.cpp /app/libfoo/foo.cpp -o program
./program
lcov --directory . --capture --output-file app.info
genhtml --output-directory cov_htmp app.info
Or, if you're in a Makefile that uses relative paths already, it's convenient to use:
g++ --coverage -DDEBUG -g3 $(abspath heyo.cpp helper/blah.cpp ../foo/bar/baz.cpp) -o program
To be able to keep your directory structure, you need to run gcov once inside each source file folder, but use the -o option to tell gcov where the data files are.
I think it should be like this:
gcov -o ../../Build/MyA *.cpp
I have a project with a similar source file structure, but I let the compiler dump object files etc into the source folders. I then run gcov multiple times from the root folder, once for each source file, but I specify the relative path of the source file and use the -o option to specify the relative folder like this:
gcov -o Source/MyA Source/MyA/aa.cpp
If you performed your product or application testing thoroughly and manually and spent lot of effort on it. If your objective is to get code coverage report using lcov and gcov but by mistake deleted gcno files. You can regenerate gcno files by recompiling the code but it will be generated with new timestamp and gcov reports error saying "stamp mismatch with graph file" and no code coverage report will be generated. This will result in all your testing effort getting wasted.
There is a shortcut to still generate the code coverage report. This is just a workaround and should not be relied upon all the time. Its recommended to preserve *.gcno files till your testing completes.
Note down your gcc version(gcc -v) and download its source code from one of the mirror sites
Eg - ftp://gd.tuwien.ac.at/gnu/sourceware/gcc/releases/gcc-4.4.6/gcc-4.4.6.tar.bz2
After extracting downloaded file, gcc the folder structure will be as follows
gcc-4.4.6
gcc-4.4.6/gcc
If you directly go inside gcc-4.4.6/gcc and try to do ./configure and compile(make) from there then you will encounter below problem
build/genmodes -h > tmp-modes.h
/bin/sh: build/genmodes: No such file or directory
Solution is do ./configure and make from gcc-4.4.6 and no errors will be shown related to genmodes. This will compile all modules including gcc. You may have to install mpfr and gmp modules which are needed by gcc if any error shown by ./configure
goto gcc-4.4.6/gcc/gcov.c and comment below lines and then recompile with above command
/* if (tag != bbg_stamp)
{
fnotice (stderr, "%s:stamp mismatch with graph file\n", da_file_name);
goto cleanup;
}*/
Example path of new gcov binary after compilation is gcc-4.4.6/host-x86_64-unknown-linux-gnu/gcc/gcov
Place this binary in /usr/bin and regenerate code coverage report with command as shown in below example
lcov --capture --directory ./ --output-file coverage.info ; genhtml coverage.info --output-directory /var/www/html/coverage
Now you should not get "stamp mismatch with graph file" error and you will get code coverage report properly