Unit Test fails to run on solution based on Helix Visual Studio Templates - sitecore

my project solution is based on Sitecore Helix Visual Studio Templates by Ander's Laub. Im trying to write Unit tests using XUnit, but none of the tests are running and not even failing.
For test projects, I have tried using Project based on Class Library(.NET Framework), Projects based on Sitecore Helix Visual Studio Templates by Ander's Laub.
Packages used:
XUnit(2.4.1)
XUnit.Runner.VisualStudio(2.4.3)
XUnit.Runner.Console(2.4.1)
Framework: .NET 4.8
Visual Studio 2019: v16.8.5
The output im getting:
Since Im not able to post images, here is the Text Explorer details:
Group Summary
Linde.Foundation.SitecoreExtensions.Tests
Tests in group: 1
Outcomes
1 Not Run
DemoTests.cs
public class DemoTests
{
[Fact]
public void DemoTestForPassingOnly()
{
double expected = 5;
double actual = 4;
Assert.Equal(expected, actual);
}
}
Test Run Output window:
ProjectData: Could not find project with GUID 09fe023c-998c-4a4c-a896-bc27884e7aaf
ProjectData: Could not find project with GUID 09fe023c-998c-4a4c-a896-bc27884e7aaf
---------- Starting test run ----------
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.3+1b45f5407b (64-bit Desktop .NET 4.0.30319.42000)
No test matches the given testcase filter `FullyQualifiedName=Linde.Foundation.SitecoreExtensions.Tests.DemoTests.DemoTestForPassingOnly` in C:\SourceTree\lindemh-sitecore\bin\Debug\Linde.Foundation.SitecoreExtensions.Tests.dll
========== Test run finished: 0 Tests run in 1.3 sec (0 Passed, 0 Failed, 0 Skipped) ==========
I have tried building a separate Console Application with Tests project to make sure whether this is a Visual Studio Issue, but it is running/failing/passing inside the console application, but not inside the Sitecore Project.
Do I have to configure the solution differently in order for the Tests to run successfully?

For simple unit tests using xunit I only have those references in my package :
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="xunit" version="2.4.1" targetFramework="net462" />
<package id="xunit.abstractions" version="2.0.3" targetFramework="net462" />
<package id="xunit.analyzers" version="0.10.0" targetFramework="net462" />
<package id="xunit.assert" version="2.4.1" targetFramework="net462" />
<package id="xunit.core" version="2.4.1" targetFramework="net462" />
<package id="xunit.extensibility.core" version="2.4.1" targetFramework="net462" />
<package id="xunit.extensibility.execution" version="2.4.1" targetFramework="net462" />
<package id="xunit.runner.console" version="2.4.1" targetFramework="net462" developmentDependency="true" />
<package id="xunit.runner.visualstudio" version="2.4.1" targetFramework="net462" developmentDependency="true" />
</packages>
So I believe the XUnit.Runner.VisualStudio(2.4.3) should be 2.4.1.
Then a simple test could look like that
using aProject.Foundation.Utilities.Extensions;
using Xunit;
namespace aProject.Foundation.Utilities.Tests
{
public class ExtensionsTests
{
public class StringExtensionTests
{
[Theory]
[InlineData("some awesome string")]
[InlineData(" some awesome string ")]
public void Should_returnValidSlug_When_passingStringWithSpaces(string name)
{
Assert.Equal("some-awesome-string", name.ToSlug());
}
}
}
}

Related

NUnit is not running my test cases

I have a test case that looks like:
namespace UnitTests
[<TestFixture>]
module ValidatorTests =
[<Test>]
let VerifyBasicFunctionality () =
Assert.That(bool)
Assert.That(bool)
and when I try to run it in the Visual Stuido test explorer, nothing happens (even with the Test Adapter for NUnit 3) and just says "Successful Build" and no tests discovered. Then when I run from the commandline with the nunit-console runner (tried with v.1, v.2, v.4) I get something different:
$ nunit-console4 bin/Release/UnitTests.dll
NUnit version 2.4.8
Copyright (C) 2002-2007 Charlie Poole.
Copyright (C) 2002-2004 James W. Newkirk, Michael C. Two, Alexei A. Vorontsov.
Copyright (C) 2000-2002 Philip Craig.
All Rights Reserved.
Runtime Environment -
OS Version: Unix 15.6.0.0
CLR Version: 4.0.30319.42000 ( 4.4.2 (Stable 4.4.2.11/f72fe45 Thu Aug 11 06:03:25 BST 2016) )
.N.N.N
Tests run: 0, Failures: 0, Not run: 3, Time: 0.011 seconds
When I run this with the -xmlConsole flag, I get this:
<?xml version="1.0" encoding="utf-8" standalone="no"?>
<!--This file represents the results of running a test suite-->
<test-results name="bin/Release/UnitTests.dll" total="0" failures="0" not-run="3" date="2016-09-08" time="10:26:00">
<environment nunit-version="2.4.8.0" clr-version="4.0.30319.42000" os-version="Unix 15.6.0.0" platform="Unix" cwd="/SOMEUSER/SOMEPATH/UnitTests" machine-name="gmaini-m.jet.local" user="SOMEUSER" user-domain="SOMELOCALBOX" />
<culture-info current-culture="en-US" current-uiculture="en-US" />
<test-suite name="bin/Release/UnitTests.dll" success="True" time="0.010" asserts="0">
<results>
<test-suite name="UnitTests" success="True" time="0.008" asserts="0">
<results>
<test-suite name="ValidatorTests" success="True" time="0.001" asserts="0">
<results>
<test-case name="UnitTests.ValidatorTests.VerifyBasicFunctionality" executed="False">
<reason>
<message><![CDATA[UnitTests.ValidatorTests is an abstract class]]></message>
</reason>
</test-case>
<test-case name="UnitTests.ValidatorTests.VerifyBasicOtherFunctionality" executed="False">
<reason>
<message><![CDATA[UnitTests.ValidatorTests is an abstract class]]></message>
</reason>
</test-case>
<test-case name="UnitTests.ValidatorTests.VerifyBasicSomeFunctionality" executed="False">
<reason>
<message><![CDATA[UnitTests.ValidatorTests is an abstract class]]></message>
</reason>
</test-case>
</results>
</test-suite>
</results>
</test-suite>
</results>
Any idea why it seems to discover the tests, I can dot access them with -run="Namespace.Module.Function" and talk about the fixture with -fixture="Namespace" but it won't run them? Is the UnitTests.ValidatorTests.VerifyBasicFunctionality is an abstract class a hint that there is some C# interop problem?
Thanks for your time :)
This fix was:
namespace UnitTests
[<TestFixtureAttribute>]
type ValidatorTests () =
[<Test>]
member __.VerifyBasicFunctionality() =
Assert.That(bool)
Assert.That(bool)
Basically, making this object oriented. So my hypothesis was correct.

Team Foundation Server 2015 Test Results

When i build my project in TFS 2015, I have set up to run all Unit tests/code coverage. The tests run and all of them pass. The issue i am having is that the results don't show up at the summary of the build or any where. Any reason why not showing up?
I looked at the logs and i am getting this warning:
Results File:
C:\BuildAgent_work\1\TestResults\serverName$_serverName 2015-12-03
15_56_33.trx
Total tests: 533. Passed: 533.
Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 29.5985 Seconds
Publishing Test Results...
##[warning]Invalid results file.
Please make sure the Test Result Format field in the task matches the
result format of the file:
C:\BuildAgent_work\1\TestResults\serverName$_serverName 2015-12-03
15_56_33.trx
I looked at the file
serverName$_serverName 2015-12-03 15_56_33.trx`
<?xml version="1.0" encoding="UTF-8"?>
<TestRun id="730b84f3-e06c-4d30-8994-15aa6ff385df" name="serverName$#serverName 2015-12-03 15:56:33" runUser="NT AUTHORITY\NETWORK SERVICE" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<Times creation="2015-12-03T15:56:33.2204061-05:00" queuing="2015-12-03T15:56:33.2204061-05:00" start="2015-12-03T15:56:33.2204061-05:00" finish="2015-12-03T15:57:01.4723683-05:00" />
<TestSettings name="default" id="d53ac855-fee0-4c64-9372-227e08990a04">
<Execution>
<TestTypeSpecific />
</Execution>
<Deployment runDeploymentRoot="serverName$_serverName 2015-12-03 15_56_33" />
<Properties />
</TestSettings>
<TestLists>
<TestList name="Results Not in a List" id="8c84fa94-04c1-424b-9868-57a2d4851a1d" />
<TestList name="All Loaded Results" id="19431567-8539-422a-85d7-44ee4e166bda" />
</TestLists>
<ResultSummary outcome="Completed">
<Counters total="0" executed="0" passed="0" failed="0" error="0" timeout="0" aborted="0" inconclusive="0" passedButRunAborted="0" notRunnable="0" notExecuted="0" disconnected="0" warning="0" completed="0" inProgress="0" pending="0" />
<Output>
<StdOut>NUnit 1.2.0.0 executing tests is startedLoading tests from C:\BuildAgent\_work\1\a\App.Tests.dllRun started: C:\BuildAgent\_work\1\a\App.Tests.dllLoading tests from C:\BuildAgent\_work\1\a\Microsoft.VisualStudio.QualityTools.UnitTestFramework.dllRun started: C:\BuildAgent\_work\1\a\Microsoft.VisualStudio.QualityTools.UnitTestFramework.dllNUnit 1.2.0.0 executing tests is finished</StdOut>
</Output>
<RunInfos>
<RunInfo computerName="serverName" outcome="Warning" timestamp="2015-12-03T15:56:31.8163881-05:00">
<Text>Diagnostic data adapter message: Could not find diagnostic data adapter of type 'Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector' and Uri 'datacollector://microsoft/CodeCoverage/2.0'.</Text>
</RunInfo>
<RunInfo computerName="serverName" outcome="Warning" timestamp="2015-12-03T15:56:32.8460013-05:00">
<Text>Unable to create DiaSession for C:\BuildAgent\_work\1\a\App.Tests.dll
No source location data will be available for this assembly.</Text>
</RunInfo>
<RunInfo computerName="serverName" outcome="Warning" timestamp="2015-12-03T15:57:01.1759645-05:00">
<Text>System.AppDomainUnloadedException: Attempted to access an unloaded AppDomain. This can happen if the test(s) started a thread but did not stop it. Make sure that all the threads started by the test(s) are stopped before completion.</Text>
</RunInfo>
</RunInfos>
</ResultSummary>
</TestRun>
My issue was with the Visual Studio in the build server. Once i updated Visual Studio from 2013 to 2015 in the build server everything worked fine.

Windows Runtime activatable class in Windows 10

I'm getting an error when I try to register an activatable class for WinRT. This is in a C# universal app (UAP) in Visual Studio 2015, on Windows 10.
Previously, you could register activatable classes in your Package.appxmanifest file like this:
<Extension Category="windows.activatableClass.inProcessServer">
<InProcessServer>
<Path>myCode.dll</Path>
<ActivatableClass ActivatableClassId="myNS.MyNativeClass" ThreadingModel="both" />
</InProcessServer>
</Extension>
But Visual Studio doesn't like this. It's violating the schema for a UAP project's appxmanifest file:
Validation error. error C00CE169: App manifest validation error: The app manifest must be valid as per schema: Line 42, Column 20, Reason: 'windows.activatableClass.inProcessServer' violates enumeration constraint of 'windows.backgroundTasks windows.preInstalledConfigTask windows.updateTask windows.restrictedLaunch'. The attribute 'Category' with value 'windows.activatableClass.inProcessServer' failed to parse. TextSecure C:\Users\123\Documents\GitHubVisualStudio\MyProj\MyProj\bin\x86\Debug\AppxManifest.xml
So how do you use Windows Runtime components from a UAP project? Without registering it, you get a TypeLoadException.
Where did you put the Extensions element, in the Application element or Package Element?
If you put it in Application element, this error is expected. You should put in in the Package element.
For example:
<?xml version="1.0" encoding="utf-8"?>
<Package
xmlns="http://schemas.microsoft.com/appx/manifest/foundation/windows10"
xmlns:mp="http://schemas.microsoft.com/appx/2014/phone/manifest"
xmlns:uap="http://schemas.microsoft.com/appx/manifest/uap/windows10"
IgnorableNamespaces="uap mp">
<Capabilities>
<Capability Name="internetClient" />
</Capabilities>
<Extensions>
<Extension Category="windows.activatableClass.inProcessServer">
<InProcessServer>
<Path>yourdll.dll</Path>
</InProcessServer>
</Extension>
</Extensions>
</Package>

Unit tests stopped working on migration from vs2008 to vs2012

After migrating a solution which includes unittests from vs2008 to vs2012, all unittests suddenly stopped working.
When running these unittests (both through the Test Explorer in Visual Studio, aswell as testing through ReSharper 7.1), they end up being failed tests with this error being shown for each test:
"QTAgent32.exe has stopped working"
When I remove the use of the testsettingsfile(vs2008), all tests run. I added its content for reference. Also, when adding a new testsettingsfile originated from vs2012, the tests do not run either.
What is the problem here?
<?xml version="1.0" encoding="UTF-8"?>
<TestSettings name="Local Test Run" id="03ca7a8e-b906-4d31-9608-087e8dfe7c60" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<Description>This is a default test run configuration for a local test run.</Description>
<Execution>
<TestTypeSpecific>
<WebTestRunConfiguration testTypeId="4e7599fa-5ecb-43e9-a887-cd63cf72d207">
<Browser name="Internet Explorer 7.0">
<Headers>
<Header name="User-Agent" value="Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)" />
<Header name="Accept" value="*/*" />
<Header name="Accept-Language" value="{{$IEAcceptLanguage}}" />
<Header name="Accept-Encoding" value="GZIP" />
</Headers>
</Browser>
</WebTestRunConfiguration>
<UnitTestRunConfig testTypeId="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b">
<AssemblyResolution>
<TestDirectory useLoadContext="true" />
</AssemblyResolution>
</UnitTestRunConfig>
</TestTypeSpecific>
<AgentRule name="LocalMachineDefaultRole">
</AgentRule>
</Execution>
</TestSettings>

Is there a way to get build status as a property?

I have a ugly Teamcity build configuration using MSBuild. It executes custom application (test runner), which is using custom messaging to report test results to teamcity.
##teamcity[testStarted name='test1']
##teamcity[testFailed name='test1' message='failure message' details='message and stack trace']
Which show in teamcity in build overview and tests tab.
Teamcity recognizes failed tests and if any test fails, it marks the build as failed:
http://i.stack.imgur.com/Qz9UT.png
Later in the MSBuild target I would like to label cvs based on the test results.
Is there a way to get the build status (if it is failed, hanging, warning) as a property? something like %build.status%? The format does not matter - if its a string or number.
PS: I know that best solution to my problem would be to modify the application to return non-zero exit code if test fail.
TeamCty does not seem to expose this directly, but the status can be acquired using the REST api. Here is an example using curl; but you could also uwe PowserShell's Invoke-RestMethod for instance.
Here's the msbuild script that casues test failure I used for testing:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Target Name="Test">
<Message Importance="high" Text="##teamcity[testStarted name='test1']" />
<Message Importance="high" Text="##teamcity[testFailed name='test1' message='failure message' details='message and stack trace']" />
</Target>
</Project>
Then the script that gets the current build's status, dumps it to a file, reads the file into an msbuild item and then uses regex to get the status out of it. You just have it to supply the tc_user and tc_password properties (or allow guest access) and change the url to match your server.
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="GetBuildStatus">
<Target Name="RunCurl">
<PropertyGroup>
<MyTempFile>curl_out</MyTempFile>
</PropertyGroup>
<Exec Command="curl http://localhost/httpAuth/app/rest/builds/id:$(teamcity_build_id) -basic -u $(tc_user):$(tc_password) > $(MyTempFile)"/>
<ReadLinesFromFile File="$(MyTempFile)">
<Output TaskParameter="Lines" ItemName="CurlOutput"/>
</ReadLinesFromFile>
<Delete Files="$(MyTempFile)"/>
</Target>
<Target Name="GetBuildStatus" DependsOnTargets="RunCurl">
<PropertyGroup>
<CurlOutputFull>#(CurlOutput)</CurlOutputFull>
<BuildStatus>$([System.Text.RegularExpressions.Regex]::Match($(CurlOutputFull), `status="(\w*)"`).Groups[ 1 ].Value)</BuildStatus>
</PropertyGroup>
<Message Text="BuildStatus = $(BuildStatus)"/>
</Target>
</Project>
This prints:
BuildStatus = FAILURE