Using the C++ unit testing framework with Visual Studio 2013 Professional, one can write unit tests and run them from within the IDE, but in order to generate any coverage reports, apparently, one needs to have the Premium or Ultimate edition of Visual Studio.
Is it possible to get code coverage reports with the Professional edition, preferably without installing any third party tools? If not, what alternate options exist for people who are not using the more expensive editions of the IDE?
Please note that it's possible to generate the coverage data by using command line tools, but I am unable to find a way to look at the results. For the sake of reference, here are the steps for command line generation of the coverage statistics:
Build the code to test with with /PROFILE linker switch
Run vsinstr /coverage <binaryName> to instrument the code; make sure that you are inside VS 2013 command prompt
Run start vsperfmon -coverage -output:results to run the profiler
Run your unit tests
Stop the profiler by running vsperfcmd -shutdown
The above will give you a results.coverage file, with no way to view it without the Premium or Ultimate editions as far as I know.
With VS2013 Professional you are out of luck if you want to do it without third party tools Requirements: Visual Studio Ultimate, Visual Studio Premium (http://msdn.microsoft.com/en-us/library/dd537628.aspx). From what I understand you already managed to generate your *.coverage file and you are having problems opening it. Visual Coverage (https://github.com/jsargiot/visual-coverage) tool can help you with that, its very simple to use and it is opensource. If you would like to find more alternatives, see another SO thread: Viewing Code Coverage Results outside of Visual studio. The tools are meant for C# coverage files but from what I understand there should not be any difference.
If you are ready to use third-party libraries, you can use OpenCPPCoverage. It works for me like a command line app. But I couldn't run it like a visual studio plugin at Professional Visual Studio 2013.
One possible way you could get coverage data is by running an instrumentation toolkit on your Unit Test program. The instrumentation tool will tell you how much was covered during execution and you can easily relate this data back to how much coverage your unit tests give you. I did this with an XCode project and OpenPAT but you could do the same with any Visual Studio friendly toolkit.
Related
Are there any plugins for visual studio 2017 which enable to calculate unit test code coverage with C++?
I only found
OpenCppCoverage:
https://marketplace.visualstudio.com/items?itemName=OpenCppCoverage.OpenCppCoveragePlugin
but it seems to be outdated / not working.
Other than that it seems like there are only Code coverage plugins for .Net-Languages but not C++
Are there any other options (besides upgrading to MSVC Enterprise)
I recently tested CPPCoverage, an open source and free extension that works perfectly to perform code coverage of native Visual C++ unit tests.
It is a visual extension (VSIX) and provide details reports. Once installed, Right-click in solution explorer on the test or application project and click "Run code coverage".
We have a C++ project we're developing on Linux with gcc. We'd like to use Visual Studio Code--if we can ever get it configured properly--but we can't if it won't meet our needs.
Does Visual Studio Code show C++ unit test coverage? For example, like how Eclipse shows code coverage of JUnit tests? Can it even be configured to run unit tests with Google Test, for example?
Code coverage reports seems to be available only in real Visual Studio Enterprise edition. For VS Code you can only utilize some command line utils, such as gcov.
Here's a link to Gcov Viewer plugin for vscode. that
uses the output from gcc to Highlight the covered lines. and automate usage of e gcov .
Friends,
I have been looking at unit test code coverage tools for the last 2 weeks, evaluating them for our specific needs. The last one I would like to evaluate is Visual Studio's own code coverage for native C++ unit tests. In our solution, we have the following:
full Visual Studio 2013 development environment with large projects
of multi-100 k lines of code
900+ unit tests already written in GTEST (working and executing in
the build process) and working in this environment native C++ code
(unmanaged)
a dozen projects in the solution or so
What I am trying to do is figure out
If I can use the Visual Studio code coverage Analysis tool to report results on my gtest unit tests
get code coverage results generated each time the unit tests or run (our unit tests are automatically run in a post-build step)
I have looked around these forums, MSDN and other sites (like stackoverflow) and I can't find a good source of guidance to making this combination of tools work. As I say, I have the solution, projects and unit tests running and working well, but to add code coverage I am not seeing how to do so using Microsoft's own Visual Studio code coverage.
I have already evaluated a few other packages like Software Verify, Semantic Designs, OpenCPPCoverage and Bullseye. I was hoping to try our Visual Studio's code coverage component since it is already part of our dev environment, but I can't seem to figure out how or if that will work.
Any help?
Thanks for any feedback, info you can give.
After searching more, I found that what I was missing was an adapter to run the tests in VS 2013. While my tests were executing as part of the build process, they weren't generating results in the Test Explorer because it wasn't being interfaced with Visual Studio's test explorer. This can be done simply by installing an extension. The extension I found that works well for us is GoogleTest Runner, which is found in the extensions library here:
GoogleTest Runner
I am leaving this question and answer here in case others have this same issue and need some help
I have a C++ googletest project building in Visual Studio 2010 professional. This builds a console application which generates XML test result output. How can I set it up to get code coverage information?
I have tried instructions here to instrument the code but viewing the Code Coverage results seems to require Visual Studio Ultimate. Is there any converter for the .coverage files Visual Studio generates?
Bonus points for portable/free/open-source solutions!
You can try OpenCppCoverage: http://opencppcoverage.codeplex.com.
OpenCppCoverage is a code coverage tools for C++ under Windows. It is open-source and works well with google test / Visual Studio C++.
Disclaimer: I am the author of this tool.
Hope that help.
Edit to follow brasofilo advice:
To be more precise about your question, OpenCppCoverage should work with Visual Studio C++ 2010 professional. The only requirement is to support native Pdb files (http://msdn.microsoft.com/en-us/library/yd4f8bd1%28vs.71%29.aspx).
It is open source and free and you can find sources here: https://opencppcoverage.codeplex.com/SourceControl/latest.
What do you mean by portable?
If you mean "Multi platform" -> No, it works only on Windows
If you mean "Portable application" (http://en.wikipedia.org/wiki/Portable_application): OpenCppCoverage required only Redistributable for Visual Studio 2013 to be installed on host (http://www.microsoft.com/en-US/download/details.aspx?id=40784) but it does not write to registry and you can move the binaries where you want (usb key for example).
Hope that help.
I think you can use the VSTEST.Consol.exe for this purpose http://msdn.microsoft.com/en-us/library/vstudio/jj155796.aspx.
According to documentation, it would run tests written against any test framework. You may enable the
/Enablecodecoverage option, which Enables data diagnostic adapter CodeCoverage in the test run.
More information may be found here: http://blogs.msdn.com/b/bhuvaneshwari/archive/2012/06/16/vstest-console-exe-commandline-test-runner.aspx
There is also an open source code coverage tool called ggcov: http://ggcov.sourceforge.net/index.html
Hope that these helps.
We would like to measure code coverage for our own automated regression test system run over a fairly large native app. This is a sophisticated, scripted test system using the inbuilt scripting of our app. It has thousands of tests and is not going to be replaced by MSTest unit tests.
Whilst we're using VS2012 (Premium) as the IDE currently it is still compiled with the VS2010 compilers & libraries. That could change sooner if it was a prerequisite to getting code coverage going.
We can do separate builds for this - instrumenting is not a problem.
I'm just confused reading the MS documentation which seems to all start from an assumption you're running unit tests using their inbuilt test framework. That's when I'm not struggling to find stuff which actually talks about native support for ALM in the first place!
thanks
Visual Studio 2012's code coverage tool is entirely separate from the test execution system (full disclosure: I wrote it, but the team that inherited it after I left Microsoft removed some fairly useful functionality). It was rewritten from the ground up in VS 2012 to dynamically instrument native (x86 and x86-64) and managed code (.NET and Silverlight) when it loads into the process instead of modifying executables on disk.
You can find CodeCoverage.exe in "%ProgramFiles%\Microsoft Visual Studio 11.0\Team Tools\Dynamic Code Coverage Tools".
To collect data:
CodeCoverage.exe collect /output:foo.coverage foo.exe foos_args
A configuration file (there's a default one in that directory called CodeCoverage.config) can be specified to control collection.
To analyze the coverage data, you can open foo.coverage in Visual Studio 2012 or use the coverage tool itself to do the analysis:
CodeCoverage.exe analyze /output:results.xml foo.coverage
Note: for instrumentation to take place, .pdb files must be discovered for your modules. Since you are building with 2010, they may not work with 2012's DIA so you may have to rebuild with 2012's toolset. If you are not seeing the modules you expect in the coverage analysis, pass /include_skipped_modules to the analyze command; there will be a "reason" attribute telling you why the module was skipped (excluded, no debug information, etc.).
Edit: Also, unlike previous versions of Visual Studio, 2012's coverage file format is completely self-contained. The modules and .pdbs don't need to be present at analysis time.
I realize this is an old post, but I believe the answer still is relevant.
With all the things that I used to have at my disposal in C#, I didn't really like what I saw when I moved to Visual C++. Also, like you the MSTests only partially worked for me; I'm used to have my own test applications as well.
Basically what I wanted was the following:
Run MS tests or an EXE file
Get code coverage right in Visual Studio.
After doing some research, I noticed that VS Enterprise supports this feature today with test adapters.
If you're not on VSE, I noticed there are some other tools, each providing users with an independent UI. Personally I don't like that; I want my coverage right in Visual Studio, preferably in Visual Studio Community edition.
So I decided to build this addin myself and - while it's not as sophisticated as VSE - it does the trick for me.
I wrote a VSIX code coverage tool on https://github.com/atlaste/CPPCoverage . Basically it manages the highlighting in Visual Studio, generates a clickable report, and integrates in solution explorer.
For the coverage measurements themselves, I used to use https://opencppcoverage.codeplex.com/ . Basically that allows you to perform code coverage tests on any debuggable (native) executable. Nowadays, I'm using my own code coverage measuring tools (which are open sourced above as well).