My c++ unit tests use gtest framework so every test is a separate executable. I'm running the CodeCoverage.exe tool to get the raw coverage data of each executable run. When I open the results in Visual Studio, each executable run appears in a separate branch and the results can't be merged together.
Is there any way to make this work?
One thing I've tried is to specify /session:uniq in the command line for both runs but that didn't have any effect.
Does it help you to capture via vsperfcmd.exe?
Build your gtest with /profile flag specified to the linker
Instrument your gtest exe with Visual Studio coverage instrumentation
vsinstr.exe gtest.exe /COVERAGE
Launch the code coverage capture tool
vsperfmon.exe /COVERAGE /OUTPUT:gtest.coverage
Run the gtest
Stop the capture tool
vsperfcmd.exe -shutdown
Launch the output file (gtest.coverage) to see the output in Visual Studio
OK, this should have been a comment, but I don't have the permissions yet.
You can run all your unit tests with the macro:
RUN_ALL_TESTS();
Related
I can't find the place where VS 2017 keeps the test result file. I have a lot of Specflow tests which are running fine but i want to find the place where VS is keeping the results. I am using a Library called Pickles which generates documentation from specflow, so it was be accessed via an HTML page. This library needs the VS test results file to add info about which tests are passing etc.
Since SpecFlow is just generating unit tests under the hood, if you run those tests from visual studio 2017 it does not generate test result files.
Instead, you need to use the vstest.console.exe command line interface to run the tests.
I am having main project file and testing project file.
when i build the solution i need to automatically trigger dotcover code coverage analysis and generate a report in html format.
It should happen in every build in visual studio.
I need suggestion regarding this to implement this in efficient way.
Thanks in advance.
If you use Visual Studio, you can add Post-build event command line.
Regarding to post build event, You can run dotcover coverage analysis from the command line. You can find related information in below links
how to add post build event to Visual Studio.
https://msdn.microsoft.com/en-us/library/ke5z92ks.aspx
How to run DotCover code coverage from command line.
https://www.jetbrains.com/help/dotcover/10.0/Running_Coverage_Analysis_from_the_Command_LIne.html
I'm trying to setup TeamCity 9.1.7 on a server and I'm encountering an issue when trying to add a Testing step.
Runner Type: Visual Studio Tests
Test engine type: VSTest
Test engine version: VSTest 2015
Test file names: C2.Tests\bin\Debug\C2.Tests.dll
Target platform: x86
When I added this Step, I was prompted to add a Config parameter:
Name: teamcity.dotnet.vstest.14.0
Value: C:\Program Files (x86)\MSBuild\14.0\Bin\MSBuild.exe
When I run all my steps I get an error in the Unit Testing step:
[14:23:17][Step 3/3] VSTest report watcher [14:23:17][VSTest report
watcher] No reports found for paths: [14:23:17][VSTest report watcher]
C:\BuildAgent\work\d28aa71801c772cb\TestResults*.trx [14:23:17][Step
3/3] Step Unit Testing (Visual Studio Tests) failed
I've had to do several things differently on this setup (on server) then when I was testing locally on my machine. For example setting up the 2015 Build Tools. I'm not sure where to look to correct this issue.
Please advise.
From the documentation:
The Visual Studio Tests runner integrates MSTest runner and VSTest console runner. Support for both frameworks enables TeamCity to execute tests and automatically import their test results.
The Visual Studio Test Runner requires Visual Studio Test Agent or Microsoft Visual Studio installed on the build agent.
https://confluence.jetbrains.com/display/TCD10/Visual+Studio+Tests
So you can use VSTests perfectly fine from version 10.
For VSTests, you need to specify a different directory for the config parameter:
teamcity.dotnet.vstest.14.0: C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe
This file will be available after installing the test agent (or visual studio).
If you still get the error then it means there is something wrong with the tests you have specified, you probably didn't reference to the assemblies correctly, make sure the path and .dll files exist.
There are two problems in the current configuration:
You are using as TestEngine the VSTest that will behave differently than the MsBuild
When you run the Test using the MSBuild this will not generate the .trx files that the Report watcher needs to display the execution results.
So in order to fix your problem you need to change the Test Engine to [MSTest]
This link may help you understand the capabilities of each Test Engine
Choose and configure a test runner
If this helps anyone, in my case this exact error was generated when I wrongly identified the DLL file containing the tests. Instead of "IntegrationTests.dll" I wrote "ItegrationTests.dll". You can identify this issue by looking into the build log where team city says:
Command line params:
and then lists paths to all test DLL files. If it's empty, it means the files you have specified in the build step have not been found.
If you're using VS Test runner, you'll see these lines telling you what happened:
[Step 3/3] No test source files were specified.
[Step 3/3] Process exited with code 1
[Step 3/3] VSTest execution failure
If you're using MS Test runner, you'll see these lines instead:
[Step 3/3] Please specify tests to run, or specify the /publish switch to publish results.
[Step 3/3] For switch syntax, type "MSTest /help"
[Step 3/3] Process exited with code 1
[Step 3/3] MSTest execution failure
Try building the tests using the MSBuild step before the testing step - it seems if it can't find the DLLs containing your tests you get the "No reports found" error
I have a project upgraded from Visual Studio 2010 to 2012 and the .testrunconfig file was included in the upgrade process.
I noticed that it was possible to click "Analyze code coverage" on any of the unit tests that I had run and it would correctly display the result. However, my test run configuration (originally from VS 2010) had code coverage disabled.
After doing a bit of research I learned that the VS 2010 configuration files have been deprecated and replaced by .runsettings files. It would appear that VS 2012 enforces assembly instrumentation by default which has a massive overhead associated with it.
Therefore, I would like to know how I can disable code coverage in VS 2012. Based upon my current findings it does not seem to be a trival task. One recent article I read had me creating an XML file manually and naming it "MYCONFIGURATION.runsettings" and manually manipulating XML attribute values.
Does anyone know how this should be done?
This is what I understand from your post:
You have a Test project with .testsettings file. You have not enabled code coverage in the test settings.
Code coverage instrumentation is not enabled by default in your scenario. Binaries will be instrumented if you do 'analyze code coverage' from VS.
Additional Info:
You can confirm that .coverage file is not generated by running the following command from visual studio developer command prompt:
vstest.console.exe /Settings:<your test settings file> test.dll
A coverage file will only get generated if you have enabled coverage in test settings.
Code coverage is only enabled through the Test Explorer using data driven adapters. The metadata for tests ran through the test explorer is almost completely different than that of tests ran straight from the unit test session window. Have you tried simply running it straight from the code (the MSTest gui bubbles) or from the unit test session window?
I'm using TFS, it works fine. Now I want to add tests and code coverage. So I create a Local.testsettings. Running tests on my computer runs fine.
Configuration of the TFS Build is as follows:
- Automated test with test assembly file specification and the same test settings file (as recommended, instead of using a .vsmdi file)
When launching a build with TFS, my tests don't get executed :
Run MSTest for Test Assemblies
The MSTestActivity was invoked without a value for Platform or Flavor. The values Mixed Platforms and Debug were used.
C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe /nologo /usestderr /testSettings:"C:\Builds\1\MyProject\Sources\MyProject\Main\Source\MyProject\Local.testsettings" /searchpathroot:"C:\Builds\1\MyProject\Binaries" /resultsfileroot:"C:\Builds\1\MyProject\TestResults" /testcontainer:"C:\Builds\1\MyProject\Binaries\MyProject.Tests.Module1.dll" /maxpriority:"1" /minpriority:"1" /publish:"http://tfsserver:8080/tfs/Test" /publishbuild:"vstfs:///Build/Build/433" /teamproject:"MyProject" /platform:"Mixed Platforms" /flavor:"Debug"
Loading C:\Builds\1\MyProject\Sources\MyProject\Main\Source\MyProject\Local.testsettings...
Loading C:\Builds\1\MyProject\Binaries\MyProject.Tests.Module1.dll...
Starting execution...
No tests to execute.
There are no results to be published.
So it finds the assembly, but MSTest doesn't actually runs the tests.
Any hint is welcome.
Looking at this:
/maxpriority:"1" /minpriority:"1"
It looks like your priority criteria may be the problem.