Environment : SonarQube 5.6 - SonarQube Runner 2.4 - MSBuild.SonarQube.Runner-2.0 - TFS 2015 - C#6
I've a Visual Studio 2015 solution with C# 6 projects and unit tests. On my TFS 2015 server, I define a build (task-based, not xaml). On my build, I've added the following steps :
- SonarQube for MSBuild - Begin Analysis.
- Visual Studio Test
- SonarQube for MSBuild - End Analysis.
Everything run fine (build, unit tests execution, code coverage, analysis results based on SonarLint, ... except that I don't see the tests results in the Sonar report (code coverage is there !).
I've tried to add some parameters :
- begin analysis : I've added : /d:sonar.cs.vstest.reportsPaths=../TestResults/*.trx
- vstests console : /Logger:trx
In the end analysis logs, I see this :
Attempting to locate a test results (.trx) file...
Located a test results file: E:\agent_work\3\TestResults\tests_results_2016-06-23 14_07_22.trx
Sensor org.sonar.plugins.csharp.CSharpUnitTestResultsProvider$CSharpUnitTestResultsImportSensor
Sensor org.sonar.plugins.csharp.CSharpUnitTestResultsProvider$CSharpUnitTestResultsImportSensor (done) | time=0ms
Any idea why I see always Unit tests=0 in the Sonar report ?
You need to provide full path to the .trx file. With below changes in begin analysis and try it again.
Change
/d:sonar.cs.vstest.reportsPaths=../TestResults/*.trx
To
/d:sonar.cs.vstest.reportsPaths=../TestResults/Mytest.trx
Related
I have a VSTS build with Unit tests that was working previously in VSTS build. This week, it stopped working due to timing out.
The failure occurs due to timing out after 60 minutes.
Here's what we know:
The test library only contains 1 test. We run the test by pointing to the test assembly in the Unit Tests configuration of VSTS.
The test still works locally (runs in <1 second).
Even if I remove the actual test and replace it with a single trivial test that just checks Assert.IsTrue(true), the test still time out. So we conclude that it's something about the test configuration, rather than the test itself, causing the issue.
I tried adding additional Debug logging to the test, but it appears that the test execution never actually starts.
No trx log file appears to be generated. It doesn't appear to actually attempt to run the test at all.
We have tried this in a VSTS hosted VM and on our own VM with the same result.
We tried adding a new test step to the build and pointing that the the test DLL with no other modifications to the defaults, and it still gets stuck.
We pointed the original test step to a different test DLL, and the tests pass! So there is something related to this particular lib that is different (but again, even if I remove all the tests it still gets stuck).
Does anyone have alternate suggestions for other things to try to narrow down what could be causing this (e.g., trace logging in VSTS)? The log file below shows some of the specifics of our unit test configuration, but please let me know if other details would be helpful.
The failure log looks like this:
> ============================================================================== 2018-07-18T20:14:25.1963123Z Run the tests locally using
> vstest.console.exe 2018-07-18T20:14:25.1964566Z
> ======================================================== 2018-07-18T20:14:25.1965645Z Test selector : Test assemblies
> 2018-07-18T20:14:25.1966925Z Test assemblies : **\<PathToTestLib>.dll
> 2018-07-18T20:14:25.1968187Z Test filter criteria : null
> 2018-07-18T20:14:25.1968810Z Search folder : C:\vsts-agent\_work\4\s
> 2018-07-18T20:14:25.1969311Z Run settings file :
> C:\vsts-agent\_work\4\s 2018-07-18T20:14:25.1970327Z Run in parallel :
> false 2018-07-18T20:14:25.1970792Z Run in isolation : false
> 2018-07-18T20:14:25.1972913Z Path to custom adapters : null
> 2018-07-18T20:14:25.1973153Z Other console options : null
> 2018-07-18T20:14:25.1973381Z Code coverage enabled : false
> 2018-07-18T20:14:25.1974004Z Rerun failed tests: false
> 2018-07-18T20:14:25.1974423Z VisualStudio version selected for test
> execution : latest 2018-07-18T20:14:25.3811086Z
> ======================================================== 2018-07-18T20:14:30.6014691Z [command]"C:\Program Files
> (x86)\Microsoft Visual
> Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe"
> #C:\vsts-agent\_work\_temp\2aa40f41-8ac7-11e8-b190-a1cdff2b8b30.txt
> 2018-07-18T20:14:30.8198002Z Microsoft (R) Test Execution Command Line
> Tool Version 15.7.2 2018-07-18T20:14:30.8206458Z Copyright (c)
> Microsoft Corporation. All rights reserved.
> 2018-07-18T20:14:30.8206862Z 2018-07-18T20:14:30.8227447Z
> vstest.console.exe 2018-07-18T20:14:30.8228082Z
> "C:\vsts-agent\_work\4\s\partners\exooutlook\OutlookAnalysisSolution.Test\bin\Debug\OutlookAnalysisSolution.Test.dll"
> 2018-07-18T20:14:30.8228647Z /logger:"trx"
> 2018-07-18T20:14:31.0672644Z Starting test execution, please wait...
> 2018-07-18T21:14:05.8979923Z ##[error]The operation was canceled.
> 2018-07-18T21:14:05.9027909Z ##[section]Finishing: Unit Tests
We found the source of the hang. It was in a test class method marked with the "AssemblyInitialize" attribute.
[TestClass]
public class TestClassInitializer
{
[AssemblyInitialize]
public static void AssemblyInit(TestContext context)
{
LocalResourceDeployment.CopyResources();
}
}
There was a bug in CopyResources() that did not manifest locally but caused a powershell window in VSTS to wait for console input.
We have improved the trace logging in this initialization method so that if it fails again in the future we'll at least have a pointer to where the test got stuck. E.g.:
Trace.WriteLine("Starting test assembly initializtion.");
TFS 2017 Build is failing on the Visual Studio Test task when it tries to publish tests result. Using task version 2.* ... version 1.* was not working anymore (couldn't find path vstest.console.exe and acted strange when I put specific path)... how can I get this to work? using tfs 2017.5 on the build server, and TFS Version 15.117.26714.0, using the latest agent from that.
2017-09-21T20:29:13.5158841Z Results File: D:\agent1_work\1\s\Dev\20170305_MD_4.4\TestResults\TfsService_DEVBUILD5 2017-09-21 15_14_33.trx
2017-09-21T20:29:14.5470386Z ##[warning]Vstest failed with error. Check logs for failures. There might be failed tests.
2017-09-21T20:29:14.5470386Z ##[error]Error: C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe failed with return code: 1
2017-09-21T20:29:17.9532729Z ##[warning]Failed to publish test results: '', hexadecimal value 0x10, is an invalid character. Line 10256, position 10.
2017-09-21T20:29:17.9532729Z ##[error]Vstest exited with return code: 1.
OK found an answer, have to use a newer version of vstest.console.exe provided by the TFS tasks team. see last item here:
https://github.com/Microsoft/vstest/issues/683
basically, need pass custom vstest.console.exe path which is VS-Insallation-Directory\Common7\IDE\Extensions\TestPlatform\vstest.console.exe in your vstest V2 task.
I'm using tfs 2017 to run CodedUI ordered unit tests. These are my build steps:
These are my "Run Functional Tests" configurations:
And these are the "Publish Test Results" (I'm not sure they are correct):
The TestAgent is deployed and the tests are running fine, The problem is that the test results appear as only one result and I can't see detailed result for each test. This is how my tests results looks like (The attachments are screen shots I take for each test):
I can reproduce your situation and TFS will treat the tests as one in the log:
DistributedTests: Total Tests : 1, Passed Tests : 0
It's a know issue, please refer below link:
How to display test results of individual test in the ordered test
suite in TFS Web
This is a limitation of the run functional test task. You can
publish the .trx file using the "publish test results" task and it
will show you all the tests but you wont know which ".orderedtest"
they were associated with etc.
You need either open *.trx file in Visual Studio or use Publish Test Results task(need to check continue on error).
Besides also change the outcome from failed to all in the test result page.
I have created simple MS Unit Test Project. Assert.Equal(true, true).
When I want to run a test I receive a error from Resharper and Visual Studio.
"Unit test Runner failed to run tests
System.Xml.XmlException: An Error occured while parsing EnityName. Line 1, position 17."
Do I missing something.
Edit:
I can't add screenshot and I can't select text to copy. Ok I will write it the best I can.
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.xml.XmlTextReaderImpl.Throw(String res,String arg)
at System.Xml.XmlText.ParseEnityName()
at System.Xml.XmlTextReaderImplparseEnityReference
at System.Xml.XmlTextReaderImpl.Read()
at System.Xml.XmlTextLoader.ParsePartialContent(XmlNode parentNode,String innerxmltext, xmlNodeType nt)
at System.Xml.XmlElement.set_InnerXml(string value)
at Microsoft.VisualStudio.TestPlatform.Utilities.InferSettingHelper.AppendOrModifyChild(XpathNavigator,String nodePatH,String nodeNAME,sTRING INNERxML)
at Microsoft.VisualStudio.TestPlatform.Utilities.InferSettingsHelper.UpdatedRunConfiguration(XPathNavigator, ArchitectureeffectivePlatform,FrameworkVersion effectiveFramework,String resultsDirectory,String solutionDirectory,String binariesRootDirectory)
Microsoft.VisualStudio.TestPlatform.Utilities.InferSettingsHelper.MergeRunSettingsAndFindCompatibleSource()
at JetBrains.ReSharper.UnitTestProvider.MSTest11.MsTest11Runner.Run(IRemoteTaskServer server,String assemblyLocation,String runConfigurationFilename,IUnitTestRun run)
To solve this problem you don't need to reinstall Visual Studio or Resharper.
The problem was in a file path to my projects. This would explains why other Unit Test projects were fine because there were in a different location.
I guess this a bug inside Visual Studio Unit Testing Framework. If you have unit test project inside folders with special charts like '&' then Visual Studio will crash or Resharper will return this message.
I wonder where I can report this bug.
I have just run into this problem too and followed the advice above.
First my path had a folder named ...\m&e...
Then I copied the whole project folder to a new location without an '&' in the path.
After that everything was working well.
I have a strange problem,
my MSBuild runs tests, code-coverage and publishing fine (part of the build.txt shown):
Results Top Level Tests
------- ---------------
Passed BuildTestProject.UnitTest1.TestMethod1
Passed BuildTestProject.UnitTest1.TestMethod2
2/2 test(s) Passed
...
Results file: W:\BuildWorkspace\XXX\Test Release\TestResults\XXX_XXX 2009-08-20 11_47_09_Any CPU_Release.trx
Run Configuration: Local Test Run
Waiting to publish...
Publishing results of test run XXX#XXX 2009-08-20 11:47:09_Any CPU_Release to http://XXX:8080/Build/v1.0/PublishTestResultsBuildService2.asmx...
....Publish completed successfully.
When I import these testresults on my local machine i get to see the code coverage-data as expected. But the code-coverage details are not shown in the details of the build that Visual Studio shows when you expand the 'results details'.
Any tips?
Finally found the solution today, found out that my TeamFoundationServer itself had problems with publishing the results to my buildAgent. I read somewhere (after searching for CoverAn.exe) that it is installed as a service, then i checked the credentials for the service 'Code Coverage Analysis Service'.
These were running under TFSMachine\NETWORK instead of our service-account for the TeamFoundationServer. Changed this, reran a build with tests in it and it now publishes the results.
W00t!
Check that the .testrunconfig used by the server has coverage enabled. It may be different than your local .testrunconfig.