I have a project upgraded from Visual Studio 2010 to 2012 and the .testrunconfig file was included in the upgrade process.
I noticed that it was possible to click "Analyze code coverage" on any of the unit tests that I had run and it would correctly display the result. However, my test run configuration (originally from VS 2010) had code coverage disabled.
After doing a bit of research I learned that the VS 2010 configuration files have been deprecated and replaced by .runsettings files. It would appear that VS 2012 enforces assembly instrumentation by default which has a massive overhead associated with it.
Therefore, I would like to know how I can disable code coverage in VS 2012. Based upon my current findings it does not seem to be a trival task. One recent article I read had me creating an XML file manually and naming it "MYCONFIGURATION.runsettings" and manually manipulating XML attribute values.
Does anyone know how this should be done?
This is what I understand from your post:
You have a Test project with .testsettings file. You have not enabled code coverage in the test settings.
Code coverage instrumentation is not enabled by default in your scenario. Binaries will be instrumented if you do 'analyze code coverage' from VS.
Additional Info:
You can confirm that .coverage file is not generated by running the following command from visual studio developer command prompt:
vstest.console.exe /Settings:<your test settings file> test.dll
A coverage file will only get generated if you have enabled coverage in test settings.
Code coverage is only enabled through the Test Explorer using data driven adapters. The metadata for tests ran through the test explorer is almost completely different than that of tests ran straight from the unit test session window. Have you tried simply running it straight from the code (the MSTest gui bubbles) or from the unit test session window?
Related
Friends,
I have been looking at unit test code coverage tools for the last 2 weeks, evaluating them for our specific needs. The last one I would like to evaluate is Visual Studio's own code coverage for native C++ unit tests. In our solution, we have the following:
full Visual Studio 2013 development environment with large projects
of multi-100 k lines of code
900+ unit tests already written in GTEST (working and executing in
the build process) and working in this environment native C++ code
(unmanaged)
a dozen projects in the solution or so
What I am trying to do is figure out
If I can use the Visual Studio code coverage Analysis tool to report results on my gtest unit tests
get code coverage results generated each time the unit tests or run (our unit tests are automatically run in a post-build step)
I have looked around these forums, MSDN and other sites (like stackoverflow) and I can't find a good source of guidance to making this combination of tools work. As I say, I have the solution, projects and unit tests running and working well, but to add code coverage I am not seeing how to do so using Microsoft's own Visual Studio code coverage.
I have already evaluated a few other packages like Software Verify, Semantic Designs, OpenCPPCoverage and Bullseye. I was hoping to try our Visual Studio's code coverage component since it is already part of our dev environment, but I can't seem to figure out how or if that will work.
Any help?
Thanks for any feedback, info you can give.
After searching more, I found that what I was missing was an adapter to run the tests in VS 2013. While my tests were executing as part of the build process, they weren't generating results in the Test Explorer because it wasn't being interfaced with Visual Studio's test explorer. This can be done simply by installing an extension. The extension I found that works well for us is GoogleTest Runner, which is found in the extensions library here:
GoogleTest Runner
I am leaving this question and answer here in case others have this same issue and need some help
I have the following setup...
TeamCity 7.1.5
Visual Studio 2013
SonarQube 3.7.4
SonarQube C++ Community plugin 0.9.1
We have a number of Visual Studio C++ solutions / projects. They all process successfully through TeamCity - Compile, Unit Test, Nuget Package generation, etc. I am now trying to add the Sonar analysis of those project, using the C++ Community plugin.
Now I understand that the plugin itself does not perform any analysis, that must be done separately and the plugin only imports the results. The plugin is successfully able to identify and import all the Source files, I can seem them listed in within the SonarQube dashboard.
The actual build and analysis is done via Visual Studio / Visual C++ compiler using MSBuild. I have enabled Code Analysis via MSBuild and I can see that it is generating a list of issues. However, I cannot get SonarQube to import that list of issues.
For the MSBuild command I am using the following parameters...
/t:Build
/p:Configuration=Debug
/p:RunCodeAnalysis=True;CodeAnalysisRuleSet=AllRules.ruleset;verbosity=normal
/filelogger
/flp:verbosity=diagnostic
I have confirmed that a MSBuild.log file is being generated and it is finding issues.
The Sonar-Runner steps has the following options...
-Dsonar.language=c++
-Dsonar.projectKey=MYProject
-Dsonar.projectName=MYProject
-Dsonar.projectVersion=0.0.1
-Dsonar.sources=Src
-Dsonar.cxx.compiler.reportPath=*.log
-Dsonar.cxx.compiler.charset=UTF-8
-Dsonar.exclusions=**/packages/**/*
-Dsonar.cxx.includeDirectories=Src/Packages "
-Dsonar.cxx.compiler.parser='Visual C++'"
I have also tried using -Dsonar.cxx.compiler.reportPath=MSbuild.log
The Sonar appears to run fine, but just doesn't pick up the code analysis issues.
Could anyone please suggest what I could be doing wrong, or what else to try.
Any help would be greatly appreciated.
Thanks & Regards,
RG
try the last version of the plugin and make sure all compiler related rules are enabled in your profile. And check your compilation build log, if the paths are relative in there you need to pass /FC flag to the compilation
We would like to measure code coverage for our own automated regression test system run over a fairly large native app. This is a sophisticated, scripted test system using the inbuilt scripting of our app. It has thousands of tests and is not going to be replaced by MSTest unit tests.
Whilst we're using VS2012 (Premium) as the IDE currently it is still compiled with the VS2010 compilers & libraries. That could change sooner if it was a prerequisite to getting code coverage going.
We can do separate builds for this - instrumenting is not a problem.
I'm just confused reading the MS documentation which seems to all start from an assumption you're running unit tests using their inbuilt test framework. That's when I'm not struggling to find stuff which actually talks about native support for ALM in the first place!
thanks
Visual Studio 2012's code coverage tool is entirely separate from the test execution system (full disclosure: I wrote it, but the team that inherited it after I left Microsoft removed some fairly useful functionality). It was rewritten from the ground up in VS 2012 to dynamically instrument native (x86 and x86-64) and managed code (.NET and Silverlight) when it loads into the process instead of modifying executables on disk.
You can find CodeCoverage.exe in "%ProgramFiles%\Microsoft Visual Studio 11.0\Team Tools\Dynamic Code Coverage Tools".
To collect data:
CodeCoverage.exe collect /output:foo.coverage foo.exe foos_args
A configuration file (there's a default one in that directory called CodeCoverage.config) can be specified to control collection.
To analyze the coverage data, you can open foo.coverage in Visual Studio 2012 or use the coverage tool itself to do the analysis:
CodeCoverage.exe analyze /output:results.xml foo.coverage
Note: for instrumentation to take place, .pdb files must be discovered for your modules. Since you are building with 2010, they may not work with 2012's DIA so you may have to rebuild with 2012's toolset. If you are not seeing the modules you expect in the coverage analysis, pass /include_skipped_modules to the analyze command; there will be a "reason" attribute telling you why the module was skipped (excluded, no debug information, etc.).
Edit: Also, unlike previous versions of Visual Studio, 2012's coverage file format is completely self-contained. The modules and .pdbs don't need to be present at analysis time.
I realize this is an old post, but I believe the answer still is relevant.
With all the things that I used to have at my disposal in C#, I didn't really like what I saw when I moved to Visual C++. Also, like you the MSTests only partially worked for me; I'm used to have my own test applications as well.
Basically what I wanted was the following:
Run MS tests or an EXE file
Get code coverage right in Visual Studio.
After doing some research, I noticed that VS Enterprise supports this feature today with test adapters.
If you're not on VSE, I noticed there are some other tools, each providing users with an independent UI. Personally I don't like that; I want my coverage right in Visual Studio, preferably in Visual Studio Community edition.
So I decided to build this addin myself and - while it's not as sophisticated as VSE - it does the trick for me.
I wrote a VSIX code coverage tool on https://github.com/atlaste/CPPCoverage . Basically it manages the highlighting in Visual Studio, generates a clickable report, and integrates in solution explorer.
For the coverage measurements themselves, I used to use https://opencppcoverage.codeplex.com/ . Basically that allows you to perform code coverage tests on any debuggable (native) executable. Nowadays, I'm using my own code coverage measuring tools (which are open sourced above as well).
I'm using TFS, it works fine. Now I want to add tests and code coverage. So I create a Local.testsettings. Running tests on my computer runs fine.
Configuration of the TFS Build is as follows:
- Automated test with test assembly file specification and the same test settings file (as recommended, instead of using a .vsmdi file)
When launching a build with TFS, my tests don't get executed :
Run MSTest for Test Assemblies
The MSTestActivity was invoked without a value for Platform or Flavor. The values Mixed Platforms and Debug were used.
C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe /nologo /usestderr /testSettings:"C:\Builds\1\MyProject\Sources\MyProject\Main\Source\MyProject\Local.testsettings" /searchpathroot:"C:\Builds\1\MyProject\Binaries" /resultsfileroot:"C:\Builds\1\MyProject\TestResults" /testcontainer:"C:\Builds\1\MyProject\Binaries\MyProject.Tests.Module1.dll" /maxpriority:"1" /minpriority:"1" /publish:"http://tfsserver:8080/tfs/Test" /publishbuild:"vstfs:///Build/Build/433" /teamproject:"MyProject" /platform:"Mixed Platforms" /flavor:"Debug"
Loading C:\Builds\1\MyProject\Sources\MyProject\Main\Source\MyProject\Local.testsettings...
Loading C:\Builds\1\MyProject\Binaries\MyProject.Tests.Module1.dll...
Starting execution...
No tests to execute.
There are no results to be published.
So it finds the assembly, but MSTest doesn't actually runs the tests.
Any hint is welcome.
Looking at this:
/maxpriority:"1" /minpriority:"1"
It looks like your priority criteria may be the problem.
How do you enable code coverage for unit testing in TFS2010 automatic build?
In Visual Studio 2010 we enabled Code Coverage for our test projects (Test --> Edit Test Settings --> Local Test Settings --> enable Code Coverage and choose assemblies to run code coverage against).
Now this is working fine (we can read code coverage) when running from Visual Studio 2010 (Test --> Run --> All Test In Solution).
The problem is that no code coverage is reported in TFS 2010 when building.
Note that the test projects are used by the build controller but without any code coverage.
Is very important for us to enable code coverage together with unit testing.
I forgot to mention that the problem I'm encountering has to do with code coverage for a web application project.
At step:
Test --> Edit Test Settings --> Local Test Settings --> enable Code Coverage and choose assemblies to run code coverage against
when choosing assemblies you can choose the web application. I think that the problem is related to Path (the value under the Path column from wizard) which is the one from developer machine (http://localhost...). When building and deploying with TFS the build goes on one server and the deployment on another.
The question should be now: which path should be available in testsettings. The build path or the deployment path?
Currently I'm not at work and can't test my doubts.
I'll let you know as soon as I get in touch with TFS.
You need to tell the build server which test settings file to use. You will find this in the build settings under Process >2. Basic >Automated Tests >1. Test Assembly >TestSettings file.
Once you have that specified then the Code coverage should work.