When i build my project in TFS 2015, I have set up to run all Unit tests/code coverage. The tests run and all of them pass. The issue i am having is that the results don't show up at the summary of the build or any where. Any reason why not showing up?
I looked at the logs and i am getting this warning:
Results File:
C:\BuildAgent_work\1\TestResults\serverName$_serverName 2015-12-03
15_56_33.trx
Total tests: 533. Passed: 533.
Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 29.5985 Seconds
Publishing Test Results...
##[warning]Invalid results file.
Please make sure the Test Result Format field in the task matches the
result format of the file:
C:\BuildAgent_work\1\TestResults\serverName$_serverName 2015-12-03
15_56_33.trx
I looked at the file
serverName$_serverName 2015-12-03 15_56_33.trx`
<?xml version="1.0" encoding="UTF-8"?>
<TestRun id="730b84f3-e06c-4d30-8994-15aa6ff385df" name="serverName$#serverName 2015-12-03 15:56:33" runUser="NT AUTHORITY\NETWORK SERVICE" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<Times creation="2015-12-03T15:56:33.2204061-05:00" queuing="2015-12-03T15:56:33.2204061-05:00" start="2015-12-03T15:56:33.2204061-05:00" finish="2015-12-03T15:57:01.4723683-05:00" />
<TestSettings name="default" id="d53ac855-fee0-4c64-9372-227e08990a04">
<Execution>
<TestTypeSpecific />
</Execution>
<Deployment runDeploymentRoot="serverName$_serverName 2015-12-03 15_56_33" />
<Properties />
</TestSettings>
<TestLists>
<TestList name="Results Not in a List" id="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
<TestList name="All Loaded Results" id="19431567-8539-422a-85d7-44ee4e166bda" />
</TestLists>
<ResultSummary outcome="Completed">
<Counters total="0" executed="0" passed="0" failed="0" error="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
<Output>
<StdOut>NUnit 1.2.0.0 executing tests is startedLoading tests from C:\BuildAgent\_work\1\a\App.Tests.dllRun started: C:\BuildAgent\_work\1\a\App.Tests.dllLoading tests from C:\BuildAgent\_work\1\a\Microsoft.VisualStudio.QualityTools.UnitTestFramework.dllRun started: C:\BuildAgent\_work\1\a\Microsoft.VisualStudio.QualityTools.UnitTestFramework.dllNUnit 1.2.0.0 executing tests is finished</StdOut>
</Output>
<RunInfos>
<RunInfo computerName="serverName" outcome="Warning" timestamp="2015-12-03T15:56:31.8163881-05:00">
<Text>Diagnostic data adapter message: Could not find diagnostic data adapter of type 'Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector' and Uri 'datacollector://microsoft/CodeCoverage/2.0'.</Text>
</RunInfo>
<RunInfo computerName="serverName" outcome="Warning" timestamp="2015-12-03T15:56:32.8460013-05:00">
<Text>Unable to create DiaSession for C:\BuildAgent\_work\1\a\App.Tests.dll
No source location data will be available for this assembly.</Text>
</RunInfo>
<RunInfo computerName="serverName" outcome="Warning" timestamp="2015-12-03T15:57:01.1759645-05:00">
<Text>System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain. This can happen if the test(s) started a thread but did not stop it. Make sure that all the threads started by the test(s) are stopped before completion.</Text>
</RunInfo>
</RunInfos>
</ResultSummary>
</TestRun>
My issue was with the Visual Studio in the build server. Once i updated Visual Studio from 2013 to 2015 in the build server everything worked fine.
Related
my project solution is based on Sitecore Helix Visual Studio Templates by Ander's Laub. Im trying to write Unit tests using XUnit, but none of the tests are running and not even failing.
For test projects, I have tried using Project based on Class Library(.NET Framework), Projects based on Sitecore Helix Visual Studio Templates by Ander's Laub.
Packages used:
XUnit(2.4.1)
XUnit.Runner.VisualStudio(2.4.3)
XUnit.Runner.Console(2.4.1)
Framework: .NET 4.8
Visual Studio 2019: v16.8.5
The output im getting:
Since Im not able to post images, here is the Text Explorer details:
Group Summary
Linde.Foundation.SitecoreExtensions.Tests
Tests in group: 1
Outcomes
1 Not Run
DemoTests.cs
public class DemoTests
{
[Fact]
public void DemoTestForPassingOnly()
{
double expected = 5;
double actual = 4;
Assert.Equal(expected, actual);
}
}
Test Run Output window:
ProjectData: Could not find project with GUID 09fe023c-998c-4a4c-a896-bc27884e7aaf
ProjectData: Could not find project with GUID 09fe023c-998c-4a4c-a896-bc27884e7aaf
---------- Starting test run ----------
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.3+1b45f5407b (64-bit Desktop .NET 4.0.30319.42000)
No test matches the given testcase filter `FullyQualifiedName=Linde.Foundation.SitecoreExtensions.Tests.DemoTests.DemoTestForPassingOnly` in C:\SourceTree\lindemh-sitecore\bin\Debug\Linde.Foundation.SitecoreExtensions.Tests.dll
========== Test run finished: 0 Tests run in 1.3 sec (0 Passed, 0 Failed, 0 Skipped) ==========
I have tried building a separate Console Application with Tests project to make sure whether this is a Visual Studio Issue, but it is running/failing/passing inside the console application, but not inside the Sitecore Project.
Do I have to configure the solution differently in order for the Tests to run successfully?
For simple unit tests using xunit I only have those references in my package :
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="xunit" version="2.4.1" targetFramework="net462" />
<package id="xunit.abstractions" version="2.0.3" targetFramework="net462" />
<package id="xunit.analyzers" version="0.10.0" targetFramework="net462" />
<package id="xunit.assert" version="2.4.1" targetFramework="net462" />
<package id="xunit.core" version="2.4.1" targetFramework="net462" />
<package id="xunit.extensibility.core" version="2.4.1" targetFramework="net462" />
<package id="xunit.extensibility.execution" version="2.4.1" targetFramework="net462" />
<package id="xunit.runner.console" version="2.4.1" targetFramework="net462" developmentDependency="true" />
<package id="xunit.runner.visualstudio" version="2.4.1" targetFramework="net462" developmentDependency="true" />
</packages>
So I believe the XUnit.Runner.VisualStudio(2.4.3) should be 2.4.1.
Then a simple test could look like that
using aProject.Foundation.Utilities.Extensions;
using Xunit;
namespace aProject.Foundation.Utilities.Tests
{
public class ExtensionsTests
{
public class StringExtensionTests
{
[Theory]
[InlineData("some awesome string")]
[InlineData(" some awesome string ")]
public void Should_returnValidSlug_When_passingStringWithSpaces(string name)
{
Assert.Equal("some-awesome-string", name.ToSlug());
}
}
}
}
I am attempting to override a few variables in my integration test with secrets sourced from KeyVault via a variable group within the build pipeline.
I have added a .runsettings file & I'm able to retrieve the dummy variables when I debug through the test. The test however exits abruptly as soon as the instruction to read the dummy value has been executed with the error below:
Value of CrmClientSecret is JustSomeSecret
Step into: Stepping over non-user code
'IntegrationTests.Helpers.Helper.DeleteUserFromCRM' Step into: Stepping over
non-user code 'IntegrationTests.Helpers.Helper. d__4..ctor' 'testhost.exe'
(CLR v4.0.30319: TestSourceHost: Enumering source
(C:\source\repos\Azure\Integrat ionTests\bin\Debug\Tests.dll)): Loaded
'C:\source\repos\Azure\IntegrationTests\bin\Debug\Utilities.dll'. Symbols
loaded. The thread 0x527c has exited with code 0 (0x0). The program '[26720]
testhost.exe' has exited with code -1 (0xffffffff). The program '[26720]
testhost.exe: Program Trace' has exited with code 0 (0x0).
Any help is appreciated
The solution is hosted in Visual Studio Ent 2017. The test project is net 4.61 & the following Nuget packages are installed:
Microsoft.NET.Test.Sdk v 15.9.0
MSTest.TestAdapter v 1.3.2
MSTest.TestFramework v 1.3.2
NUnit3TestAdapter version 3.11.0
Here are my test results:
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<RunConfiguration>
<MaxCpuCount>2</MaxCpuCount>
<ResultsDirectory>.\TestResults</ResultsDirectory>
<TargetPlatform>x64</TargetPlatform>
<TargetFrameworkVersion>Framework45</TargetFrameworkVersion>
<TestAdaptersPaths>%systemdrive%\PROGRAM FILES (X86)\MICROSOFT VISUAL STUDIO\2017\ENTERPRISE\COMMON7\IDE\EXTENSIONS\TESTPLATFORM\Extensions</TestAdaptersPaths>
<TestSessionTimeout>10000</TestSessionTimeout>
</RunConfiguration>
<DataCollectionRunSettings>
<DataCollectors>
<DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
<Configuration>
<CodeCoverage>
<ModulePaths>
<Exclude>
<ModulePath>.*CPPUnitTestFramework.*</ModulePath>
</Exclude>
</ModulePaths>
<UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
<AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
<CollectFromChildProcesses>True</CollectFromChildProcesses>
<CollectAspDotNet>False</CollectAspDotNet>
</CodeCoverage>
</Configuration>
</DataCollector>
<DataCollector uri="datacollector://microsoft/VideoRecorder/1.0" assemblyQualifiedName="Microsoft.VisualStudio.TestTools.DataCollection.VideoRecorder.VideoRecorderDataCollector, Microsoft.VisualStudio.TestTools.DataCollection.VideoRecorder, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" friendlyName="Screen and Voice Recorder">
</DataCollector>
</DataCollectors>
</DataCollectionRunSettings>
<TestRunParameters>
<Parameter name="AmbassadorsDbConnectionString"
value="TheConnectionString" />
<Parameter name="CRMClientSecret" value="JustSomeSecret" />
<Parameter name="webAppPassword" value="Password" />
</TestRunParameters>
<MSTest>
<MapInconclusiveToFailed>True</MapInconclusiveToFailed>
<CaptureTraceOutput>false</CaptureTraceOutput>
<DeleteDeploymentDirectoryAfterTestRunIsComplete>False
<DeploymentEnabled>False</DeploymentEnabled>
<AssemblyResolution>
<Directory path="D:\myfolder\bin\" includeSubDirectories="false"/>
</AssemblyResolution>
</MSTest>
</RunSettings>
You can use the blame option to collect the dump and analyze the cause for the crash
https://github.com/Microsoft/vstest-docs/blob/master/RFCs/0024-Blame-Collector-Options.md
I'm getting an error when I try to register an activatable class for WinRT. This is in a C# universal app (UAP) in Visual Studio 2015, on Windows 10.
Previously, you could register activatable classes in your Package.appxmanifest file like this:
<Extension Category="windows.activatableClass.inProcessServer">
<InProcessServer>
<Path>myCode.dll</Path>
<ActivatableClass ActivatableClassId="myNS.MyNativeClass" ThreadingModel="both" />
</InProcessServer>
</Extension>
But Visual Studio doesn't like this. It's violating the schema for a UAP project's appxmanifest file:
Validation error. error C00CE169: App manifest validation error: The app manifest must be valid as per schema: Line 42, Column 20, Reason: 'windows.activatableClass.inProcessServer' violates enumeration constraint of 'windows.backgroundTasks windows.preInstalledConfigTask windows.updateTask windows.restrictedLaunch'. The attribute 'Category' with value 'windows.activatableClass.inProcessServer' failed to parse. TextSecure C:\Users\123\Documents\GitHubVisualStudio\MyProj\MyProj\bin\x86\Debug\AppxManifest.xml
So how do you use Windows Runtime components from a UAP project? Without registering it, you get a TypeLoadException.
Where did you put the Extensions element, in the Application element or Package Element?
If you put it in Application element, this error is expected. You should put in in the Package element.
For example:
<?xml version="1.0" encoding="utf-8"?>
<Package
xmlns="http://schemas.microsoft.com/appx/manifest/foundation/windows10"
xmlns:mp="http://schemas.microsoft.com/appx/2014/phone/manifest"
xmlns:uap="http://schemas.microsoft.com/appx/manifest/uap/windows10"
IgnorableNamespaces="uap mp">
<Capabilities>
<Capability Name="internetClient" />
</Capabilities>
<Extensions>
<Extension Category="windows.activatableClass.inProcessServer">
<InProcessServer>
<Path>yourdll.dll</Path>
</InProcessServer>
</Extension>
</Extensions>
</Package>
After migrating a solution which includes unittests from vs2008 to vs2012, all unittests suddenly stopped working.
When running these unittests (both through the Test Explorer in Visual Studio, aswell as testing through ReSharper 7.1), they end up being failed tests with this error being shown for each test:
"QTAgent32.exe has stopped working"
When I remove the use of the testsettingsfile(vs2008), all tests run. I added its content for reference. Also, when adding a new testsettingsfile originated from vs2012, the tests do not run either.
What is the problem here?
<?xml version="1.0" encoding="UTF-8"?>
<TestSettings name="Local Test Run" id="03ca7a8e-b906-4d31-9608-087e8dfe7c60" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<Description>This is a default test run configuration for a local test run.</Description>
<Execution>
<TestTypeSpecific>
<WebTestRunConfiguration testTypeId="4e7599fa-5ecb-43e9-a887-cd63cf72d207">
<Browser name="Internet Explorer 7.0">
<Headers>
<Header name="User-Agent" value="Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)" />
<Header name="Accept" value="*/*" />
<Header name="Accept-Language" value="{{$IEAcceptLanguage}}" />
<Header name="Accept-Encoding" value="GZIP" />
</Headers>
</Browser>
</WebTestRunConfiguration>
<UnitTestRunConfig testTypeId="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b">
<AssemblyResolution>
<TestDirectory useLoadContext="true" />
</AssemblyResolution>
</UnitTestRunConfig>
</TestTypeSpecific>
<AgentRule name="LocalMachineDefaultRole">
</AgentRule>
</Execution>
</TestSettings>
I have a ugly Teamcity build configuration using MSBuild. It executes custom application (test runner), which is using custom messaging to report test results to teamcity.
##teamcity[testStarted name='test1']
##teamcity[testFailed name='test1' message='failure message' details='message and stack trace']
Which show in teamcity in build overview and tests tab.
Teamcity recognizes failed tests and if any test fails, it marks the build as failed:
http://i.stack.imgur.com/Qz9UT.png
Later in the MSBuild target I would like to label cvs based on the test results.
Is there a way to get the build status (if it is failed, hanging, warning) as a property? something like %build.status%? The format does not matter - if its a string or number.
PS: I know that best solution to my problem would be to modify the application to return non-zero exit code if test fail.
TeamCty does not seem to expose this directly, but the status can be acquired using the REST api. Here is an example using curl; but you could also uwe PowserShell's Invoke-RestMethod for instance.
Here's the msbuild script that casues test failure I used for testing:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="Test">
<Message Importance="high" Text="##teamcity[testStarted name='test1']" />
<Message Importance="high" Text="##teamcity[testFailed name='test1' message='failure message' details='message and stack trace']" />
</Target>
</Project>
Then the script that gets the current build's status, dumps it to a file, reads the file into an msbuild item and then uses regex to get the status out of it. You just have it to supply the tc_user and tc_password properties (or allow guest access) and change the url to match your server.
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="GetBuildStatus">
<Target Name="RunCurl">
<PropertyGroup>
<MyTempFile>curl_out</MyTempFile>
</PropertyGroup>
<Exec Command="curl http://localhost/httpAuth/app/rest/builds/id:$(teamcity_build_id) -basic -u $(tc_user):$(tc_password) > $(MyTempFile)"/>
<ReadLinesFromFile File="$(MyTempFile)">
<Output TaskParameter="Lines" ItemName="CurlOutput"/>
</ReadLinesFromFile>
<Delete Files="$(MyTempFile)"/>
</Target>
<Target Name="GetBuildStatus" DependsOnTargets="RunCurl">
<PropertyGroup>
<CurlOutputFull>#(CurlOutput)</CurlOutputFull>
<BuildStatus>$([System.Text.RegularExpressions.Regex]::Match($(CurlOutputFull), `status="(\w*)"`).Groups[ 1 ].Value)</BuildStatus>
</PropertyGroup>
<Message Text="BuildStatus = $(BuildStatus)"/>
</Target>
</Project>
This prints:
BuildStatus = FAILURE