I'm trying to create a code coverage report using the Build pipeline.
I have added the task of typeVisual Studio code in the build pipeline and have enabled the Code Coverage.
When the build is triggered. I'm getting :
Data collector 'Code Coverage' message: Data collector 'Code Coverage' failed to provide initialization information. Error: System.TypeInitializationException: The type initializer for 'Microsoft.VisualStudio.Diagnostics.Logging.ProfilerInterop' threw an exception. ---> Microsoft.VisualStudio.Diagnostics.Common.InvariantException: Failed to load IntelliTrace Profiler binary or failed to locate functions.
and
---> System.ComponentModel.Win32Exception: The system cannot find the path specified
This is running the tests and all the tests are passed. However I'm not able to view the code coverage report. The report which it has created contains only information about the tests
Any input on where we specify the path will be useful.
You basically need Visual Studio Test Agent for Code Coverage.
There are 2 possible ways to install for this:
Option 1
Install Agents for Visual Studio 2019 on the Build server (download from here, see under Tools for Visual Studio 2019).
In the build pipeline edit the Visual Studio Test Assemblies task. Set Select test platform using to Specific location and set Path to vstest.console.exe to for example C:\Program Files (x86)\Microsoft Visual Studio\2019\TestAgent\Common7\IDE\Extensions\TestPlatform\vstest.console.exe.
Option 2
Add the Visual Studio test platform installer build task to your pipeline. (Add this task before the testing task).
In the Visual Studio Test Assemblies task you have to select Installed by Tools Installer as the Test platform version
Note: in my experience I had some tests who succeeded with option 1, but failed in option2. Sadly I don't have the time to figure out why...
I faced this issue when configuring a build container.
1 To install the Visual Studio 2019 Test Agent, I used Chocolatey
1.2 Install Chocolatey
ENV chocolateyUseWindowsCompression = false
SHELL ["powershell.exe", "-ExecutionPolicy", "Bypass", "-Command"]
RUN Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1')); `
[System.Environment]::SetEnvironmentVariable('PATH', "\"${env:PATH};%ALLUSERSPROFILE%\chocolatey\bin\"", 'Machine'); `
choco feature enable -n allowGlobalConfirmation;
1.2 Install the TestAgent with chocolatey
RUN choco install visualstudio2019testagent -y
2. Edit the VSTest task on your pipeline to use a specific location.
In my case, I've installed into the container the VSBuildTools. The VSTest task used the VSBuildTools vstest.console.exe but the execution needs some libraries that are located at the TestAgent folders.
vstestLocationMethod: location
vstestLocation: 'C:\Program Files (x86)\Microsoft Visual Studio\2019\TestAgent\Common7\IDE\Extensions\TestPlatform\vstest.console.exe'
Had a similar problem after updating build agent and adding VS2017 capabilities.
Downloaded (from https://www.opendll.com): microsoft.intellitrace.profiler.dll
Add to folder C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\Common7\IDE\Extensions\TestPlatform\x64
Good luck
Related
I'm working on a project with multiple Unit Tests. I have a visual studio .sln file with around 10 XXPrj in it. Those projects are made with Google Test. Everything works well if I want to run them using Visual Studio 2019, I can build and run the unit tests.
I would like to know what is the best way to run them an automated way with commandline. Purpose is to then integrate this commandline stuff in a jenkins to have everything automated.
Build
Building a Visual Studio solution/project through the command line is done with msbuild.exe. It works best to add the path of MSBuild to the PATH environment variable.
MSBuild is usually installed somewhere in the Visual Studio folders. E.g. on my machine the path is as follows:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin\MSBuild.exe
Build the solution containing all your projects as follows:
msbuild.exe Example.sln
# Or if you want to build a release version or add additional arguments
msbuild.exe Example.sln /property:Configuration=Debug
See MSBuild CLI Docs for more options.
Side Note: Jenkins has a msbuild plugin you can use with a build step called "Build a Visual Studio project or solution using MSBuild" (Important: this does not install MSBuild, it only provides a GUI to use MSBuild in a build plan).
Run Tests
To run the tests you have two options:
Run each project's executable in your build pipeline and the executable's exit code will indicate the success/failure of the unit tests for that project. However, you will need to call each executable separately; or
Use the vstest.console.exe in combination with the Google Test Adapter
You can use the Google Test Adapter the same way in which Visual Studio uses it when you click Test -> Run -> All tests to discover & execute tests in your projects.
In my environment, vstest.console.exe is located here:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\Extensions\TestPlatform\vstest.console.exe
You also need to provide the path to the test adaptor. Then execute all the tests as follows:
# Assuming vstext.console.exe is included in the PATH
# and the current working directory is the relevant project executable
# output folder:
vstest.console.exe Project1.exe Project2.exe Project3.exe /TestAdapterPath:"<path to adapter>"
Once again the path is hidden somewhere in the Visual Studio Folders. I found it through searching for GoogleTestAdapter.TestAdapter.dll. On my machine it is located in:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\Extensions\drknwe51.xnq
Conclusion
Thus, an automated way to build and run GoogleTest unit tests, which are split over multiple projects, with the commandline can be performed in these two steps:
Build the solution/projects using msbuild.exe
Run the tests using vtest.console.exe in combination with the Google Test Adapter
Our CI environment setup with [TFS Server 2015 + Visual studio build tools 2017]. When I try to run C# based unit test on environment it shows this error
TF900547: The directory containing the assemblies for the Visual Studio Test Runner is not valid ''.
Then I tried to fix the error by fallowing ways
Installed VS test agent 2015 in the machine https://learn.microsoft.com/en-us/visualstudio/test/lab-management/install-configure-test-agents
Copied C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow from my computer to exact location to CI machine.
Changed test runner from Visual Studio Test Runner to MS Test Runner
But still same error. Looks like tfs server not able to find the test agent folder. Anyone has idea how to fix this
Seems you have VS 2017 Build Tools installed only, no VS IDE on the CI environment.
You need to install VS IDE. VS build tools doesn't contain the "test
platform". In order to run the tests, you require to install full
IDE.
Currently to run the VsTest task, the task has dependency on Visual Studio IDE. But some changes will have in the near future.
We have decided to go with a 'tools installer' which can be used to
get the test platform instead of needing the IDE. If you have the IDE,
great. If you don't, the tools installer will get the test platform on
the agent for you. It is already on backlog and expected to be available on VSTS in a few sprints.
More details please take a look at the discussion in this issue: vstest capability not detected when installed without Visual Studio
Test Logs:
"Queued the TMI run for test run [1594]." TestOutcome 'Warning';
Message 'Warning: Test Run deployment issue: The assembly or module
'Microsoft.VisualStudio.Shell.15.0' directly or indirectly referenced
by deployment item 'C:\Program Files (x86)\Microsoft Visual Studio'
specified by the test settings was not found.'."
Several other similar errors.
Not sure why is it showing so.
It sounds like you need to install the appropriate version of the Agents for Visual Studio package on the machine where you're trying to run these tests.
The 2017 version should be what you're after if you're using Visual Studio 2017.
I'm trying to setup TeamCity 9.1.7 on a server and I'm encountering an issue when trying to add a Testing step.
Runner Type: Visual Studio Tests
Test engine type: VSTest
Test engine version: VSTest 2015
Test file names: C2.Tests\bin\Debug\C2.Tests.dll
Target platform: x86
When I added this Step, I was prompted to add a Config parameter:
Name: teamcity.dotnet.vstest.14.0
Value: C:\Program Files (x86)\MSBuild\14.0\Bin\MSBuild.exe
When I run all my steps I get an error in the Unit Testing step:
[14:23:17][Step 3/3] VSTest report watcher [14:23:17][VSTest report
watcher] No reports found for paths: [14:23:17][VSTest report watcher]
C:\BuildAgent\work\d28aa71801c772cb\TestResults*.trx [14:23:17][Step
3/3] Step Unit Testing (Visual Studio Tests) failed
I've had to do several things differently on this setup (on server) then when I was testing locally on my machine. For example setting up the 2015 Build Tools. I'm not sure where to look to correct this issue.
Please advise.
From the documentation:
The Visual Studio Tests runner integrates MSTest runner and VSTest console runner. Support for both frameworks enables TeamCity to execute tests and automatically import their test results.
The Visual Studio Test Runner requires Visual Studio Test Agent or Microsoft Visual Studio installed on the build agent.
https://confluence.jetbrains.com/display/TCD10/Visual+Studio+Tests
So you can use VSTests perfectly fine from version 10.
For VSTests, you need to specify a different directory for the config parameter:
teamcity.dotnet.vstest.14.0: C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe
This file will be available after installing the test agent (or visual studio).
If you still get the error then it means there is something wrong with the tests you have specified, you probably didn't reference to the assemblies correctly, make sure the path and .dll files exist.
There are two problems in the current configuration:
You are using as TestEngine the VSTest that will behave differently than the MsBuild
When you run the Test using the MSBuild this will not generate the .trx files that the Report watcher needs to display the execution results.
So in order to fix your problem you need to change the Test Engine to [MSTest]
This link may help you understand the capabilities of each Test Engine
Choose and configure a test runner
If this helps anyone, in my case this exact error was generated when I wrongly identified the DLL file containing the tests. Instead of "IntegrationTests.dll" I wrote "ItegrationTests.dll". You can identify this issue by looking into the build log where team city says:
Command line params:
and then lists paths to all test DLL files. If it's empty, it means the files you have specified in the build step have not been found.
If you're using VS Test runner, you'll see these lines telling you what happened:
[Step 3/3] No test source files were specified.
[Step 3/3] Process exited with code 1
[Step 3/3] VSTest execution failure
If you're using MS Test runner, you'll see these lines instead:
[Step 3/3] Please specify tests to run, or specify the /publish switch to publish results.
[Step 3/3] For switch syntax, type "MSTest /help"
[Step 3/3] Process exited with code 1
[Step 3/3] MSTest execution failure
Try building the tests using the MSBuild step before the testing step - it seems if it can't find the DLLs containing your tests you get the "No reports found" error
I Have VS 2015 only installed on my TFS 2013 build server, and get the following error:
TF900547: The directory containing the assemblies for the Visual Studio Test Runner is not valid ''.
Other sites say to install VS 2013, but why would I have to install VS 2013 on the build server to run unit tests when I have VS 2015 installed?
I have the following switches on my build:
/p:VisualStudioVersion=14.0 /tv:14.0
(This is not a duplicate question as I am using TFS 2013 and am trying to use the Agile Test Runner not MS Test)
Check the solution in this case:
Install VS 2015 or Microsoft Build Tools 2015.
Edit build to use a build process template called TfvcTemplate.12.xaml.
Set MSBuild arguments to /tv:14.0.
But this setting only works for build, not test. As there is no way to set the tool version or tool path for "Run VS Test Runner" activity, installing VS 2013 should be the only way to get rid of this error.
I had a similar problem - was getting the error
TF900547: The directory containing the assemblies for the Visual Studio Test Runner is not valid 'D:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow'.
It ended up being that even though Visual Studio 2015 was installed on the build agent, the user that the Build Agent was running as didn't have read/execute permissions to that folder.