VSTS not showing passing NUnit tests in results - unit-testing

Question

I'm very new to powershell unit testing. I wrote my "hello world" tests below:
[CmdletBinding()]
param()
Describe MyFailTest{
Assert-Throws {
throw "I do nothing interesting."
} -MessagePattern "Not that message because I want a failure"
}
Describe MyPassTest{
Assert-Throws {
throw "I will pass"
} -MessagePattern "I will pass"
}
And that looks like I would expect when run locally, the first one fails as the message pattern did not match, and the second test passes.
Now I'm trying to run the test as part of a CI/CD pipeline. I'm using "Pester Test Runner from Black Marble" task to kick off my .tests.ps1 files. In my NUnit output I can see it marked my tests as passing and failing as expected.
So now we get to the odd part. I used the publish results task in VSTS and I only ever see failed tests. Why am I not seeing passing tests as well? Seems odd it reports that 1/1 tests failed if I have a successful test run. I don't see any options to change that. I tried setting the Outcome filter to "All" in the tests tab. Any ideas?
Pertinent log sections:
Evaluating condition for step: 'Pester Test Runner'
Evaluating: succeeded()
Evaluating succeeded:
=> (Boolean) True
Expanded: True
Result: True
##[section]Starting: Pester Test Runner
==============================================================================
Task : Pester Test Runner from Black Marble
Description : Run Pester tests without the need to install Pester in with the PMModule folder or in the source repo (Using Pester 3.4.3 or 4.0.8)
Version : 6.0.9
Author : Richard Fennell
Help : Version: 6.0.9. [More Information](https://github.com/rfennell/vNextBuild/wiki/Pester-Task/)
==============================================================================
Preparing task execution handler.
Executing the powershell script: d:\a\_tasks\Pester_31ef0033-64e3-4c55-b888-f446541474a6\6.0.9\Pester.ps1
PowerShellHandler.Execute - AddCommand(d:\a\_tasks\Pester_31ef0033-64e3-4c55-b888-f446541474a6\6.0.9\Pester.ps1)
PowerShellHandler.Execute - Add inputParameters
PowerShellHandler.Execute - AddParameter(scriptFolder=d:\a\1\s\Tasks\XXXXTWStaging\Tests\)
PowerShellHandler.Execute - AddParameter(resultsFile=d:\a\1\s\Test-Pester.XML)
PowerShellHandler.Execute - AddParameter(CodeCoverageOutputFile=)
PowerShellHandler.Execute - AddParameter(tag=)
PowerShellHandler.Execute - AddParameter(excludeTag=)
PowerShellHandler.Execute - AddParameter(pesterVersion=4.0.8)
PowerShellHandler.Execute - AddParameter(run32Bit=False)
PowerShellHandler.Execute - AddParameter(moduleFolder=)
PowerShellHandler.Execute - Invoke
Running in AMD64 PowerShell
Loading Pester module from [D:\a\_tasks\Pester_31ef0033-64e3-4c55-b888-f446541474a6\6.0.9\4.0.8]
Loading module from path 'D:\a\_tasks\Pester_31ef0033-64e3-4c55-b888-f446541474a6\6.0.9\4.0.8\Pester.psd1'.
Loading module from path 'D:\a\_tasks\Pester_31ef0033-64e3-4c55-b888-f446541474a6\6.0.9\4.0.8\Pester.psm1'.
Processed: ##vso[task.setprogress value=-1;]
Loading module from path 'D:\a\_tasks\Pester_31ef0033-64e3-4c55-b888-f446541474a6\6.0.9\4.0.8\lib\Gherkin.dll'.
namespace Pester
{
using System;
using System.Management.Automation;
public static class ClosingBraceFinder
{
public static int GetClosingBraceIndex(PSToken[] tokens, int startIndex)
{
int groupLevel = 1;
int len = tokens.Length;
for (int i = startIndex + 1; i < len; i++)
{
PSTokenType type = tokens[i].Type;
if (type == PSTokenType.GroupStart)
{
groupLevel++;
}
else if (type == PSTokenType.GroupEnd)
{
groupLevel--;
if (groupLevel <= 0) { return i; }
}
}
return -1;
}
}
}
using System;
namespace Pester
{
[Flags]
public enum OutputTypes
{
None = 0,
Default = 1,
Passed = 2,
Failed = 4,
Pending = 8,
Skipped = 16,
Inconclusive = 32,
Describe = 64,
Context = 128,
Summary = 256,
Header = 512,
All = Default | Passed | Failed | Pending | Skipped | Inconclusive | Describe | Context | Summary | Header,
Fails = Default | Failed | Pending | Skipped | Inconclusive | Describe | Context | Summary | Header
}
}
Exporting function 'Context'.
Exporting function 'Describe'.
Exporting function 'In'.
Exporting function 'It'.
Exporting function 'Mock'.
Exporting function 'Assert-VerifiableMock'.
Exporting function 'Assert-MockCalled'.
Exporting function 'Set-TestInconclusive'.
Exporting function 'Assert-VerifiableMocks'.
Exporting function 'InModuleScope'.
Exporting function 'Invoke-Mock'.
Exporting function 'New-Fixture'.
Exporting function 'Get-TestDriveItem'.
Exporting function 'Setup'.
Exporting function 'Should'.
Exporting function 'Invoke-Pester'.
Exporting function 'BeforeEach'.
Exporting function 'AfterEach'.
Exporting function 'BeforeAll'.
Exporting function 'AfterAll'.
Exporting function 'Set-DynamicParameterVariable'.
Exporting function 'Get-MockDynamicParameter'.
Exporting function 'New-PesterOption'.
Exporting function 'SafeGetCommand'.
Exporting function 'Invoke-Gherkin'.
Exporting function 'Find-GherkinStep'.
Exporting function 'Invoke-GherkinStep'.
Exporting function 'BeforeEachFeature'.
Exporting function 'AfterEachFeature'.
Exporting function 'BeforeEachScenario'.
Exporting function 'AfterEachScenario'.
Exporting function 'GherkinStep'.
Exporting alias 'Given'.
Exporting alias 'When'.
Exporting alias 'Then'.
Exporting alias 'And'.
Exporting alias 'But'.
Exporting function 'Add-AssertionOperator'.
Exporting function 'New-MockObject'.
Importing function 'Add-AssertionOperator'.
Importing function 'AfterAll'.
Importing function 'AfterEach'.
Importing function 'AfterEachFeature'.
Importing function 'AfterEachScenario'.
Importing function 'Assert-MockCalled'.
Importing function 'Assert-VerifiableMock'.
Importing function 'Assert-VerifiableMocks'.
Importing function 'BeforeAll'.
Importing function 'BeforeEach'.
Importing function 'BeforeEachFeature'.
Importing function 'BeforeEachScenario'.
Importing function 'Context'.
Importing function 'Describe'.
Importing function 'Find-GherkinStep'.
Importing function 'Get-MockDynamicParameter'.
Importing function 'Get-TestDriveItem'.
Importing function 'GherkinStep'.
Importing function 'In'.
Importing function 'InModuleScope'.
Importing function 'Invoke-Gherkin'.
Importing function 'Invoke-GherkinStep'.
Importing function 'Invoke-Mock'.
Importing function 'Invoke-Pester'.
Importing function 'It'.
Importing function 'Mock'.
Importing function 'New-Fixture'.
Importing function 'New-MockObject'.
Importing function 'New-PesterOption'.
Importing function 'SafeGetCommand'.
Importing function 'Set-DynamicParameterVariable'.
Importing function 'Set-TestInconclusive'.
Importing function 'Setup'.
Importing function 'Should'.
Importing alias 'And'.
Importing alias 'But'.
Importing alias 'Given'.
Importing alias 'Then'.
Importing alias 'When'.
Running Pester from [d:\a\1\s\Tasks\XXXXTWStaging\Tests\] output sent to [d:\a\1\s\Test-Pester.XML]
Exporting function 'EnterTestGroup'.
Exporting function 'LeaveTestGroup'.
Exporting function 'AddTestResult'.
Exporting function 'AddSetupOrTeardownBlock'.
Exporting function 'GetTestCaseSetupBlocks'.
Exporting function 'GetTestCaseTeardownBlocks'.
Exporting function 'GetCurrentTestGroupSetupBlocks'.
Exporting function 'GetCurrentTestGroupTeardownBlocks'.
Exporting function 'EnterTest'.
Exporting function 'LeaveTest'.
Exporting variable 'Strict'.
Exporting variable 'Show'.
Exporting variable 'TagFilter'.
Exporting variable 'ExcludeTagFilter'.
Exporting variable 'TestNameFilter'.
Exporting variable 'SessionState'.
Exporting variable 'CommandCoverage'.
Exporting variable 'InTest'.
Exporting variable 'TestResult'.
Exporting variable 'TotalCount'.
Exporting variable 'Time'.
Exporting variable 'PassedCount'.
Exporting variable 'FailedCount'.
Exporting variable 'SkippedCount'.
Exporting variable 'PendingCount'.
Exporting variable 'InconclusiveCount'.
Exporting variable 'IncludeVSCodeMarker'.
Exporting variable 'TestSuiteName'.
Exporting variable 'TestActions'.
Exporting variable 'TestGroupStack'.
Executing all tests in 'd:\a\1\s\Tasks\XXXXTWStaging\Tests\'
Executing script D:\a\1\s\Tasks\XXXXTWStaging\Tests\L0ValidateDestinationPath.tests.ps1
Importing module: TestHelpersModule
Stubbing command: Import-Module
Setting copyFilesToMachinesPath
Asserting script block should throw: {
Validate-DestinationPath -value "" -environmentName $validEnvironmentName
}
Success. Matched exception message. Pattern: WFC_ParameterCannotBeNullorEmpty targetPath ; Message: WFC_ParameterCannotBeNullorEmpty targetPath
Asserting script block should throw: {
Validate-DestinationPath -value $invalidTargetPath -environmentName $validEnvironmentName
}
Success. Matched exception message. Pattern: WFC_RemoteDestinationPathCannotContainEnvironmentVariables $env:abc\123 ; Message: WFC_RemoteDestinationPathCannotContainEnvironmentVariables $env:abc\123
Executing script D:\a\1\s\Tasks\XXXXTWStaging\Tests\L0ValidateSourcePath.tests.ps1
Importing module: TestHelpersModule
Invoking mock command: Import-Module
Arguments: D:\a\1\s\Tasks\XXXXTWStaging\Tests\lib/TestHelpersModule -Verbose: False
Command is stubbed.
Stubbing command: Import-Module
Mocking command: Test-Path
ParametersEvaluator: { $LiteralPath -eq $invalidSourcePath }
Func: { return $false }
Asserting script block should throw: {
Validate-SourcePath -value ""
}
Success. Matched exception message. Pattern: WFC_ParameterCannotBeNullorEmpty sourcePath ; Message: WFC_ParameterCannotBeNullorEmpty sourcePath
Asserting script block should throw: {
Validate-SourcePath -value "$invalidSourcePath"
}
Invoking mock command: Test-Path
Arguments: -LiteralPath Invalid
Matching implementation found using parameters evaluator: { $LiteralPath -eq $invalidSourcePath }
Invoking Func: { return $false }
Success. Matched exception message. Pattern: WFC_SourcePathDoesNotExist Invalid ; Message: WFC_SourcePathDoesNotExist Invalid
Executing script D:\a\1\s\Tasks\XXXXTWStaging\Tests\L0ValidInputSequentialCopy.tests.ps1
Importing module: TestHelpersModule
Invoking mock command: Import-Module
Arguments: D:\a\1\s\Tasks\XXXXTWStaging\Tests\lib/TestHelpersModule -Verbose: False
Command is stubbed.
Stubbing command: Import-Module
Setting copyFilesToMachinesPath
Mocking command: Test-Path
ParametersEvaluator: { $LiteralPath -eq $validSourcePackage }
Func: { return $true }
Mocking command: Test-Path
ParametersEvaluator: { $LiteralPath -eq $invalidSourcePath }
Func: { return $false }
Mocking command: Invoke-Command
Func: { }
Mocking command: ConvertTo-SecureString
Func: { return $password }
Mocking command: Get-VstsInput
ParametersEvaluator: { $Name -eq "SourcePath" }
Func: { return $validSourcePackage }
Mocking command: Get-VstsInput
ParametersEvaluator: { $Name -eq "TargetPath" }
Func: { return $validApplicationPath }
Mocking command: Get-VstsInput
ParametersEvaluator: { $Name -eq "CleanTargetBeforeCopy" }
Func: { return $true }
Mocking command: Get-VstsInput
ParametersEvaluator: { $Name -eq "TechWintelDeployService" }
Func: { return "TechWintelDeployService" }
Mocking command: Get-VstsEndpoint
ParametersEvaluator: { $Name -eq "TechWintelDeployService" }
Func: { return $myVar }
Entering D:\a\1\s\Tasks\XXXXTWStaging\WindowsMachineFileCopy.ps1.
Invoking mock command: Get-VstsInput
Arguments: -Name TechWintelDeployService -Require
Matching implementation found using parameters evaluator: { $Name -eq "TechWintelDeployService" }
Invoking Func: { return "TechWintelDeployService" }
Invoking mock command: Get-VstsEndpoint
Arguments: -Name TechWintelDeployService -Require
Matching implementation found using parameters evaluator: { $Name -eq "TechWintelDeployService" }
Invoking Func: { return $myVar }
Invoking mock command: Get-VstsInput
Arguments: -Name SourcePath -Require
Matching implementation found using parameters evaluator: { $Name -eq "SourcePath" }
Invoking Func: { return $validSourcePackage }
Invoking mock command: Get-VstsInput
Arguments: -Name TargetPath -Require
Matching implementation found using parameters evaluator: { $Name -eq "TargetPath" }
Invoking Func: { return $validApplicationPath }
Invoking mock command: Get-VstsInput
Arguments: -Name AdditionalArguments
Command is stubbed.
Invoking mock command: Get-VstsInput
Arguments: -Name CleanTargetBeforeCopy
Matching implementation found using parameters evaluator: { $Name -eq "CleanTargetBeforeCopy" }
Invoking Func: { return $true }
Importing VSTSLocStrings
Invoking mock command: Test-Path
Arguments: -LiteralPath C:\Windows\Test
Matching implementation found using parameters evaluator: { $LiteralPath -eq $validSourcePackage }
Invoking Func: { return $true }
Invoking mock command: ConvertTo-SecureString
Arguments: Password -AsPlainText -Force
Invoking Func: { return $password }
Invoking mock command: Invoke-Command
Arguments: -ScriptBlock
param (
[string]$fqdn,
[string]$sourcePath,
[string]$targetPath,
[object]$credential,
[string]$cleanTargetBeforeCopy,
[string]$additionalArguments,
[string]$scriptRoot
)
Import-Module "$scriptRoot\..\ps_modules\VstsTaskSdk"
Import-VstsLocStrings -LiteralPath $scriptRoot/Task.json
Write-Verbose "Entering script RobocopyJob.ps1"
Write-Verbose "fqdn = $fqdn"
Write-Verbose "sourcePath = $sourcePath"
Write-Verbose "targetPath = $targetPath"
Write-Verbose "credential = $credential"
Write-Verbose "cleanTargetBeforeCopy = $cleanTargetBeforeCopy"
Write-Verbose "additionalArguments = $additionalArguments"
$sourcePath = $sourcePath.Trim().TrimEnd('\', '/')
$targetPath = $targetPath.Trim().TrimEnd('\', '/')
$isFileCopy = Test-Path -Path $sourcePath -PathType Leaf
$doCleanUp = $cleanTargetBeforeCopy -eq "true"
$sourceDirectory = $sourcePath
$filesToCopy = ""
if($isFileCopy)
{
$sourceDirectory = Split-Path $sourcePath
$filesToCopy = Split-Path $sourcePath -Leaf
}
function ThrowError
{
param(
[string]$errorMessage,
[string]$fqdn
)
$failMessage = (Get-VstsLocString -Key "WFC_CopyingFailedForResource" -ArgumentList $fqdn)
throw "$failMessage`n$errorMessage"
}
function Validate-Null(
[string]$value,
[string]$variableName
)
{
$value = $value.Trim()
if(-not $value)
{
ThrowError -errorMessage (Get-VstsLocString -Key "WFC_ParameterCannotBeNullorEmpty" -ArgumentList $variableName)
}
}
function Validate-Credential(
[object]$credential)
{
if($credential)
{
Validate-Null $credential.UserName "Username"
Validate-Null $credential.Password "Password"
}
else
{
ThrowError -errorMessage (Get-VstsLocString -Key "WFC_ParameterCannotBeNullorEmpty" -ArgumentList "credential")
}
}
function Get-DownLevelLogonName(
[string]$fqdn,
[string]$userName
)
{
if($userName -like '.\*') {
$userName = $userName.replace(".\","\")
$userName = $fqdn+$userName
}
return $userName
}
function Replace-First(
[string]$text,
[string]$search,
[string]$replace
)
{
$pos = $text.IndexOf($search);
if ($pos -le 0)
{
return $text;
}
return $text.Substring(0, $pos) + $replace + $text.Substring($pos + $search.Length);
}
function Get-DestinationNetworkPath(
[string]$targetPath,
[string]$machineShare
)
{
if(-not $machineShare)
{
return $targetPath
}
$targetSpecificPath = Replace-First $targetPath ":" '$'
return [io.path]::Combine($machineShare, $targetSpecificPath)
}
function Get-RoboCopyParameters(
[string]$additionalArguments,
[switch]$fileCopy,
[switch]$clean)
{
$robocopyParameters = "/COPY:DAT"
if(-not $fileCopy.IsPresent)
{
if($clean.IsPresent)
{
$robocopyParameters += " /MIR"
}
else
{
$robocopyParameters += " /E"
}
}
if (-not [string]::IsNullOrWhiteSpace($additionalArguments))
{
$robocopyParameters += " $additionalArguments"
}
return $robocopyParameters.Trim()
}
function Get-MachineShare(
[string]$fqdn,
[string]$targetPath
)
{
if([bool]([uri]$targetPath).IsUnc)
{
return $targetPath
}
if($fqdn)
{
return [IO.Path]::DirectorySeparatorChar + [IO.Path]::DirectorySeparatorChar + $fqdn
}
return ""
}
function Get-NetExeCommand
{
$netExePath = Join-Path -path (get-item env:\windir).value -ChildPath system32\net.exe
if(Test-Path $netExePath)
{
Write-Verbose "Found the net exe path $netExePath. Net command will be $netExePath"
return $netExePath
}
Write-Verbose "Unable to get the path for net.exe. Net command will be 'net'"
return 'net'
}
$machineShare = Get-MachineShare -fqdn $fqdn -targetPath $targetPath
$destinationNetworkPath = Get-DestinationNetworkPath -targetPath $targetPath -machineShare $machineShare
Validate-Credential $credential
$userName = Get-DownLevelLogonName -fqdn $fqdn -userName $($credential.UserName)
$password = $($credential.Password)
$netExeCommand = Get-NetExeCommand
if($machineShare)
{
$command = "$netExeCommand use `"$machineShare`""
if($userName)
{
$command += " /user:`'$userName`' `'$($password -replace "['`]", '$&$&')`'"
}
$command += " 2>&1"
$dtl_mapOut = iex $command
if ($LASTEXITCODE -ne 0)
{
$errorMessage = (Get-VstsLocString -Key "WFC_FailedToConnectToPathWithUser" -ArgumentList $machineShare, $($credential.UserName)) + $dtl_mapOut
ThrowError -errorMessage $errorMessage -fqdn $fqdn
}
}
try
{
if($isFileCopy -and $doCleanUp -and (Test-Path -path $destinationNetworkPath -pathtype container))
{
Get-ChildItem -Path $destinationNetworkPath -Recurse -force | Remove-Item -force -recurse;
$output = Remove-Item -path $destinationNetworkPath -force -recurse 2>&1
$err = $output | ?{$_.gettype().Name -eq "ErrorRecord"}
if($err)
{
Write-Verbose -Verbose "Error occurred while deleting the destination folder: $err"
}
}
$robocopyParameters = Get-RoboCopyParameters -additionalArguments $additionalArguments -fileCopy:$isFileCopy -clean:$doCleanUp
$command = "robocopy `"$sourceDirectory`" `"$destinationNetworkPath`" `"$filesToCopy`" $robocopyParameters"
Invoke-Expression $command
if ($LASTEXITCODE -ge 8)
{
$errorMessage = Get-VstsLocString -Key "WFC_CopyingFailedConsultRobocopyLogsForMoreDetails"
ThrowError -errorMessage $errorMessage -fqdn $fqdn
}
else
{
$message = (Get-VstsLocString -Key "WFC_CopyingRecurivelyFrom0to1MachineSucceed" -ArgumentList $sourcePath, $targetPath, $fqdn)
Write-Output $message
}
}
finally
{
if($machineShare)
{
$dtl_deleteMap = iex "$netExeCommand use `"$machineShare`" /D /Y";
}
}
-ArgumentList System.Object[]
Invoking Func: { }
Leaving D:\a\1\s\Tasks\XXXXTWStaging\WindowsMachineFileCopy.ps1.
Asserting was-called: Invoke-Command
Expected times: 1
Actual times: 1
Executing script D:\a\1\s\Tasks\XXXXTWStaging\Tests\MyFailTest.tests.ps1
Describing MyFailTest
Invoking mock command: Test-Path
Arguments: TestDrive:\
Command is stubbed.
Asserting script block should throw: {
throw "I do nothing interesting."
}
[-] Error occurred in Describe block
778ms
RuntimeException: Actual exception message does not match expected pattern. Expected: Not that message because I want a failure ; Actual: I do nothing interesting.
at Assert-Throws, D:\a\1\s\Tasks\XXXXTWStaging\Tests\lib\TestHelpersModule\PublicFunctions.ps1: line 67
at <ScriptBlock>, D:\a\1\s\Tasks\XXXXTWStaging\Tests\MyFailTest.tests.ps1: line 5
at DescribeImpl, D:\a\_tasks\Pester_31ef0033-64e3-4c55-b888-f446541474a6\6.0.9\4.0.8\Functions\Describe.ps1: line 161
Describing MyPassTest
Invoking mock command: Test-Path
Arguments: TestDrive:\
Command is stubbed.
Asserting script block should throw: {
throw "I will pass"
}
Success. Matched exception message. Pattern: I will pass ; Message: I will pass
Tests completed in 778ms
Tests Passed: 0,
Failed: 1,
Skipped: 0,
Pending: 0,
Inconclusive: 0
Perform operation 'Enumerate CimInstances' with following parameters, ''namespaceName' = root\cimv2,'className' = Win32_OperatingSystem'.
Operation 'Enumerate CimInstances' complete.
##[error]Microsoft.PowerShell.Commands.WriteErrorException: Pester returned errors
Processed: ##vso[task.logissue type=error;]Microsoft.PowerShell.Commands.WriteErrorException: Pester returned errors
##[error]PowerShell script completed with 1 errors.
##[section]Finishing: Pester Test Runner
Evaluating condition for step: 'Publish Test Results Test-Pester.XML'
Evaluating: succeededOrFailed()
Evaluating succeededOrFailed:
=> (Boolean) True
Expanded: True
Result: True
##[section]Starting: Publish Test Results Test-Pester.XML
==============================================================================
Task : Publish Test Results
Description : Publish Test Results to VSTS/TFS
Version : 2.0.1
Author : Microsoft Corporation
Help : [More Information](https://go.microsoft.com/fwlink/?LinkID=613742)
==============================================================================
agent.workFolder=d:\a
loading inputs and endpoints
loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
loading INPUT_MERGETESTRESULTS
loading INPUT_PUBLISHRUNATTACHMENTS
loading INPUT_SEARCHFOLDER
loading INPUT_TESTRESULTSFILES
loading INPUT_TESTRUNNER
loaded 8
testRunner=NUnit
testResultsFiles=Test-Pester.XML
mergeTestResults=true
platform=null
configuration=null
testRunTitle=null
publishRunAttachments=true
searchFolder=d:\a\1\s
testRunner: NUnit
testResultsFiles: Test-Pester.XML
mergeResults: true
platform: null
config: null
testRunTitle: null
publishRunAttachments: true
defaultRoot: 'd:\a\1\s'
findOptions.followSpecifiedSymbolicLink: 'true'
findOptions.followSymbolicLinks: 'true'
matchOptions.debug: 'false'
matchOptions.nobrace: 'true'
matchOptions.noglobstar: 'false'
matchOptions.dot: 'true'
matchOptions.noext: 'false'
matchOptions.nocase: 'true'
matchOptions.nonull: 'false'
matchOptions.matchBase: 'false'
matchOptions.nocomment: 'false'
matchOptions.nonegate: 'false'
matchOptions.flipNegate: 'false'
pattern: 'Test-Pester.XML'
findPath: 'd:\a\1\s\Test-Pester.XML'
statOnly: 'true'
found 1 paths
applying include pattern
adjustedPattern: 'd:\a\1\s\Test-Pester.XML'
1 matches
1 final results
Reading test results from file 'd:\a\1\s\Test-Pester.XML'
Processed: ##vso[results.publish type=NUnit;mergeResults=true;publishRunAttachments=true;resultFiles=d:\a\1\s\Test-Pester.XML;]
task result: Succeeded
Processed: ##vso[task.complete result=Succeeded;]
##[section]Async Command Start: Publish test results
Publishing test results to test run '17948'
Test results remaining: 1. Test run id: 17948
Published Test Run : https://XXXX.visualstudio.com/DevOps/_TestManagement/Runs#runId=17948&_a=runCharts
##[section]Async Command End: Publish test results
##[section]Finishing: Publish Test Results Test-Pester.XML

Answer 1

At the suggestion of a coworker I tried adding "It" blocks around my tests. That seemed to work. I won't mark this as the answer yet as I didn't see it blocks in the example I was working off of. My code now looks like:
[CmdletBinding()]
param()
Describe MyFailTest{
It "This will fail, the patterns don't match"{
Assert-Throws {
throw "I do nothing interesting."
} -MessagePattern "Not that message because I want a failure"
}
}
Describe MyPassTest{
It "This will pass, the patterns do match"{
Assert-Throws {
throw "I will pass"
} -MessagePattern "I will pass"
}
}