Devops Pipeline refusing to run XUnit tests - unit-testing

I have been really getting frustrated with getting unit tests to run in a Azure Devops Pipeline
below is my YAML file contents
trigger:
- main
pool:
vmImage: 'windows-2019'
variables:
solution: '**/FestWise.stock*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
imageRepository: 'festwisestock'
dotNetCoreVrs: '3.1.x'
steps:
- task: UseDotNet#2
inputs:
version: '$(dotNetCoreVrs)'
packageType: 'sdk'
- task: NuGetCommand#2
inputs:
command: 'restore'
restoreSolution: '**/FestWise.stock*.sln'
feedsToUse: 'select'
- task: DotNetCoreCLI#2
displayName: 'Restore project dependencies'
inputs:
command: 'restore'
projects: '**/FestWise.Stock/FestWise.Stock.API/FestWise.StockService*.csproj'
- task: DotNetCoreCLI#2
displayName: 'Build the project - $(buildConfiguration)'
inputs:
command: 'build'
arguments: '--no-restore --configuration $(buildConfiguration)'
projects: '**/FestWise.Stock/FestWise.Stock.API/FestWise.StockService*.csproj'
- task: DotNetCoreCLI#2
displayName: 'Run unit tests - $(buildConfiguration)'
inputs:
command: 'test'
arguments: '--no-build --configuration $(buildConfiguration)'
publishTestResults: true
projects: '**/*StockService.UnitTests.csproj'
the result is as follows:
Starting: Run unit tests - Release
==============================================================================
Task : .NET Core
Description : Build, test, package, or publish a dotnet application, or run a custom dotnet command
Version : 2.187.0
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/build/dotnet-core-cli
==============================================================================
C:\Windows\system32\chcp.com 65001
Active code page: 65001
Info: .NET Core SDK/runtime 2.2 and 3.0 are now End of Life(EOL) and have been removed from all hosted agents. If you're using these SDK/runtimes on hosted agents, kindly upgrade to newer versions which are not EOL, or else use UseDotNet task to install the required version.
C:\hostedtoolcache\windows\dotnet\dotnet.exe test D:\a\1\s\FestWise.Stock\FestWise.StockService.UnitTests\FestWise.StockService.UnitTests.csproj --logger trx --results-directory D:\a\_temp --no-build --configuration Release
##[warning]No test result files were found.
Info: Azure Pipelines hosted agents have been updated and now contain .Net 5.x SDK/Runtime along with the older .Net Core version which are currently lts. Unless you have locked down a SDK version for your project(s), 5.x SDK might be picked up which might have breaking behavior as compared to previous versions. You can learn more about the breaking changes here: https://learn.microsoft.com/en-us/dotnet/core/tools/ and https://learn.microsoft.com/en-us/dotnet/core/compatibility/ . To learn about more such changes and troubleshoot, refer here: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/build/dotnet-core-cli?view=azure-devops#troubleshooting
Finishing: Run unit tests - Release
This is the furthest that i have come, it states it found the correct project but subsequently doesnt run any tests

So this was a drama to get figured out,
for some reason it didnt want to find the build sln from the previous step and thus the --no-build argument made it not work
removing that argument solved it for me
new complete yaml (with some improvements to make it more variable dependent)
trigger:
- main
pool:
vmImage: 'windows-2019'
variables:
solution: '**/FestWise.stock*.sln'
projectPath: '**/FestWise.Stock/FestWise.Stock.API/FestWise.StockService*.csproj'
testProjectPath: '**/*/FestWise.StockService.UnitTests.csproj'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
dotNetCoreVrs: '3.1.x'
steps:
- task: UseDotNet#2
inputs:
version: '$(dotNetCoreVrs)'
packageType: 'sdk'
- task: NuGetCommand#2
inputs:
command: 'restore'
restoreSolution: $(solution)
feedsToUse: 'select'
- task: DotNetCoreCLI#2
displayName: 'Restore project dependencies'
inputs:
command: 'restore'
projects: $(projectPath)
- task: DotNetCoreCLI#2
displayName: 'Build the project - $(buildConfiguration)'
inputs:
command: 'build'
arguments: '--no-restore --configuration $(buildConfiguration)'
projects: $(projectPath)
- task: DotNetCoreCLI#2
inputs:
command: "test"
includeNuGetOrg: true
projects: $(testProjectPath)
publishTestResults: true
displayName: Run the server-side tests

Related

Pipeline: .NetCoreCLI Test Task throws non-zero error

I have used the following yml command for my .Net 5 API project and xUnit Test Project but it throws error and my pipeline is not getting succeeded. Where did I go wrong?
Note: The pipeline is not getting succeded even if the task executed the test cases and showing 15 test cases passed and 2 test cases are filed.
- task: DotNetCoreCLI#2
inputs:
command: 'restore'
projects: '**/GeniusData.Test/GeniusData.Test.csproj'
displayName: 'Restore Projects'
- task: DotNetCoreCLI#2
inputs:
command: test
projects: '**/*Test/*.csproj'
arguments: '--configuration $(buildConfiguration) --collect "Code coverage"'
displayName: 'Test Project'
You're using the DotNetCoreCLI#2 task which will always fail when tests fail. That's by design: failing tests should break the build.

CTest Error on MacOS: Test Executable not found

I am working on a CMake project. For CI, I have decided to use Azure Pipelines. However I am facing a small problem on MacOS in the testing phase. The problem is that MacOS fails to find the test executable, even when it is there.
Earlier my project wasn't getting built up properly. But now the entire pipeline runs successfully (thanks to the Stack Overflow community) except for the minor glitch on MacOS. I have updated my question and now it tells what problem was and how I fixed it so that it may be helpful to others like me who are new to the world of CI.
Edit 1:
Earlier my CMake task wasn't triggering the build process. This was because I was providing not providing any command-line arguments to CMake. All I did was this:
- task: CMake#1
displayName: Generate CMake Cache
inputs:
workingDirectory: build
I assumed that the CMake task would drive the build process automatically, as it wasn't specified clearly in the documentation what that task actually does. This did nothing apart from printing the CMake usage. Then I found out that we have to run the CMake task twice (once for project configuration and then for the actual build) using appropriate command-line arguments.
Edit 2:
This is my updated AzurePipelines.yml file
Azure Pipelines CI
stages:
- stage: Build
displayName: Build
jobs:
- job: RunCMakeTask
displayName: Run CMake Task
strategy:
matrix:
LinuxDebug:
OS: 'Linux'
imageName: 'ubuntu-latest'
BuildConfiguration: 'Debug'
LinuxRelease:
OS: 'Linux'
imageName: 'ubuntu-latest'
BuildConfiguration: 'RelWithDebInfo'
MacOSDebug:
OS: 'MacOS'
imageName: 'macos-latest'
BuildConfiguration: 'Debug'
MacOSRelease:
OS: 'MacOS'
imageName: 'macos-latest'
BuildConfiguration: 'RelWithDebInfo'
WindowsDebug:
OS: 'Windows'
imageName: 'windows-latest'
BuildConfiguration: 'Debug'
WindowsRelease:
OS: 'Windows'
imageName: 'windows-latest'
BuildConfiguration: 'RelWithDebInfo'
pool:
vmImage: $(imageName)
steps:
- script: mkdir $(BuildConfiguration)
displayName: Create Build Directory
workingDirectory: $(Build.SourcesDirectory)
- task: CMake#1
displayName: Generate CMake Cache
inputs:
workingDirectory: $(BuildConfiguration)
cmakeArgs: '-DCMAKE_BUILD_TYPE=$(BuildConfiguration) ..'
- task: CMake#1
displayName: Run Build Process
inputs:
workingDirectory: $(BuildConfiguration)
cmakeArgs: '--build . --config $(BuildConfiguration)'
- task: PublishPipelineArtifact#1
displayName: Publish Build Artifact
inputs:
targetPath: $(BuildConfiguration)
artifactName: '$(OS)$(BuildConfiguration)'
- stage: Test
displayName: Test
dependsOn: Build
jobs:
- job: RunCTestOnWindows
displayName: Run CTest on Windows
variables:
OS: Windows
strategy:
matrix:
Debug:
BuildConfiguration: 'Debug'
Release:
BuildConfiguration: 'RelWithDebInfo'
pool:
vmImage: 'windows-latest'
steps:
- task: DownloadPipelineArtifact#2
displayName: Download Build Artifact
inputs:
artifact: '$(OS)$(BuildConfiguration)'
path: $(Build.SourcesDirectory)/$(BuildConfiguration)
- script: ctest -C $(BuildConfiguration) --output-on-failure
workingDirectory: $(BuildConfiguration)
- job: RunCTestOnUnixBasedSystems
displayName: Run CTest on Unix Based Systems
strategy:
matrix:
LinuxDebug:
OS: 'Linux'
imageName: 'ubuntu-latest'
BuildConfiguration: 'Debug'
LinuxRelease:
OS: 'Linux'
imageName: 'ubuntu-latest'
BuildConfiguration: 'RelWithDebInfo'
MacOSDebug:
OS: 'MacOS'
imageName: 'macos-latest'
BuildConfiguration: 'Debug'
MacOSRelease:
OS: 'MacOS'
imageName: 'macos-latest'
BuildConfiguration: 'RelWithDebInfo'
steps:
- task: DownloadPipelineArtifact#2
displayName: Download Build Artifact
inputs:
artifact: '$(OS)$(BuildConfiguration)'
path: $(Build.SourcesDirectory)/$(BuildConfiguration)
- script: find $(BuildConfiguration)/Tests -type f -name "Test*" ! -name "*.*" ! -exec chmod u+rx {} \;
displayName: Change File Permissions
- script: ctest -C $(BuildConfiguration) --output-on-failure
workingDirectory: $(BuildConfiguration)
The pipeline runs fine on Windows and Linux, but I am facing a small problem on MacOS. On MacOS, ctest fails to find the test executable even when it is there. (If there were any problem in my pipeline or my CMakeLists.txt file, it should also have failed on Windows and Linux)
Edit 3:
And in the Test stage on MacOS, I am getting the error:
Test project /home/vsts/work/1/s/Debug
Start 1: StringOperations_CaseIgnore Could not find executable /Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations
Looked in the following places:
/Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations
/Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations
/Users/runner/work/1/s/Debug/Tests/StringOperations/Debug/TestStringOperations
/Users/runner/work/1/s/Debug/Tests/StringOperations/Debug/TestStringOperations
Debug//Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations
Debug//Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations
Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations
Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations
Users/runner/work/1/s/Debug/Tests/StringOperations/Debug/TestStringOperations
Users/runner/work/1/s/Debug/Tests/StringOperations/Debug/TestStringOperations
Debug/Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations
Debug/Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations
1/1 Test #1: StringOperations_CaseIgnore ......***Not Run 0.00 sec
0% tests passed, 1 tests failed out of 1
Total Test time (real) = 0.00 sec
The following tests FAILED: 1 - StringOperations_CaseIgnore (Not
Run) Unable to find executable:
/Users/runner/work/1/s/Debug/Tests/StringOperations/TestStringOperations
Errors while running CTest
Edit 4:
I have tried to check whether the test executable is actually there or not by using:
ls -l Debug/Tests/StringOperations
And here's the output:
drwxr-xr-x 3 vsts docker 4096 Aug 6 15:05 CMakeFiles
-rw-r--r-- 1 vsts docker 1208 Aug 6 15:05 cmake_install.cmake
-rw-r--r-- 1 vsts docker 642 Aug 6 15:05 CTestTestfile.cmake
-rw-r--r-- 1 vsts docker 9838 Aug 6 15:05 Makefile
-rwxr--r-- 1 vsts docker 1715072 Aug 6 15:05 TestStringOperations
This confirms that the test executable (TestStringOperations) is there at the same place where it was for Windows and Linux, but still the process fails.
Here is the CMakeLists.txt for this executable should you need it:
Set(SRC StringOperations.cpp)
Add_Executable(TestStringOperations ${SRC})
Target_Include_Directories(TestStringOperations PUBLIC
${HEADER_PATH}/StringOperations
)
Target_Link_Libraries(TestStringOperations
PRIVATE ${GTEST_LIBS}
PRIVATE StringOperations
)
Add_Test(NAME StringOperations_CaseIgnore COMMAND TestStringOperations)
I have tried looking for help on this issue on Stack Overflow and some other sites, but their solutions aren't benefitting me.
For example:
CTest can not find executable file and CMake: How to specify directory where ctest should look for executables?
If you need more info, here is my project on GitHub. You can also refer to the pipeline logs at dev.azure.com.
Can you please help me in fixing this problem? Any suggestions regarding the overall implementation of this file are also welcome.
Try running a Publish Pipeline Artifact task after your build. As eluded to in the comments this will publish your build contents and will allow it to be shared across stages.
After doing this then you will also be able to see it as a published artifact in the UI for the pipeline.
# Publish pipeline artifacts
# Publish (upload) a file or directory as a named artifact for the current run
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(Pipeline.Workspace)'
artifactName: # 'drop'
Also another gut check that can be done at the end of the stage is to include a powershell script to see the contents of the directory you are working in:
-powershell: Get-ChildItem -Path 'Insert root path' -recurse

Errors when trying to run an AWSPowerShellModuleScript#1 in Azure devops pipeline

I currently have an Azure Devops pipeline to build and deploy a next.js application via the serverless framework.
Upon reaching the AWSPowerShellModuleScript#1 task I get these errors:
[warning]MSG:UnableToDownload «https:...» «»
[warning]Unable to download the list of available providers. Check
your internet connection.
[warning]Unable to download from URI 'https:...' to ''.
[error]No match was found for the specified search criteria for the
provider 'NuGet'. The package provider requires 'PackageManagement'
and 'Provider' tags. Please check if the specified package has the
tags.
[error]No match was found for the specified search criteria and
module name 'AWSPowerShell'. Try Get-PSRepository to see all
available registered module repositories.
[error]The specified module 'AWSPowerShell' was not loaded because no
valid module file was found in any module directory.
I do have the AWS.ToolKit installed and it's visible when I go to manage extensions within Azure Devops.
My pipeline:
trigger: none
stages:
- stage: develop_build_deploy_stage
pool:
name: Default
demands:
- msbuild
- visualstudio
jobs:
- job: develop_build_deploy_job
steps:
- checkout: self
clean: true
- task: NodeTool#0
displayName: Install Node
inputs:
versionSpec: '12.x'
- script: |
npm install
npx next build
displayName: Install Dependencies and Build
- task: CopyFiles#2
inputs:
Contents: 'build/**'
TargetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
displayName: Publish Artifact
inputs:
pathtoPublish: $(Build.ArtifactStagingDirectory)
artifactName: dev_artifacts
- task: AWSPowerShellModuleScript#1
displayName: Deploy to Lambda#Edge
inputs:
awsCredentials: '###'
regionName: '###'
scriptType: 'inline'
inlineScript: 'npx serverless --package dev_artifacts'
I know I can use the ubuntu vmImage and then make use of the awsShellScript but the build agent I have available to me doesn't support bash.

How do I build a MSVC++ project that uses dependencies from vcpkg through an Azure DevOps pipeline?

I have a C++ project that has cpprestsdk and libpqxx as its dependencies and I'm using vcpkg as my package manager.
I've created an Azure DevOps pipeline that uses the CppBuildTask task to clone and build the dependencies from vcpkg, this is working correctly and all the dependencies are pulled and built successfully, but I'm not sure how to actually build the project using the *.vcxproj file.
I tried using the Visual Studio Build task, but the build fails because it can't find the dependencies that were just downloaded by the CppBuildTask.
What is the correct task to use when trying to build a MSVC++ project with vcpkg on Azure DevOps?
Edit, the pipeline yaml file:
pool:
name: Azure Pipelines
demands:
- msbuild
- visualstudio
steps:
- task: Cache#2
displayName: Cache
inputs:
key: '$(Build.SourcesDirectory)/response_file.txt | 5951e0b42569257f97a5d9ac2d8c5bd4942c417b | x64-windows'
path: '$(Build.SourcesDirectory)/vcpkg'
- task: lucappa.cmake-ninja-vcpkg-tasks.d855c326-b1c0-4d6f-b1c7-440ade6835fb.run-vcpkg#0
displayName: 'Run vcpkg'
inputs:
vcpkgDirectory: '$(Build.SourcesDirectory)/vcpkg'
vcpkgGitCommitId: 5951e0b42569257f97a5d9ac2d8c5bd4942c417b
vcpkgArguments: '#$(Build.SourcesDirectory)/response_file.txt'
cleanAfterBuild: false
- task: VSBuild#1
displayName: 'Build solution TileServer\TileServer.vcxproj'
inputs:
solution: '$(Parameters.solution)'
platform: '$(BuildPlatform)'
configuration: '$(BuildConfiguration)'
msbuildArchitecture: x64
- task: PublishSymbols#2
displayName: 'Publish symbols path'
inputs:
SearchPattern: '**\bin\**\*.pdb'
IndexSources: false
PublishSymbols: false
continueOnError: true
- task: CopyFiles#2
displayName: 'Copy Files to: $(build.artifactstagingdirectory)'
inputs:
Contents: |
**\bin\$(BuildConfiguration)\**
.\Renderer\Styles\Themes\DefaultTheme.json
.\TileServer\glew32.dll
.\TileServer\ReleaseSettings.json
TargetFolder: '$(build.artifactstagingdirectory)'
condition: succeededOrFailed()
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
ArtifactName: TileServer
condition: succeededOrFailed()

Integrating Selenium with Gitlab CI

I have created an automated selenium test script which works perfectly fine.
My task now is to set up Gitlab CI and try to automatically run this selenium script when I make a push to git.
Is it possible to make the selenium script automatically execute and inform the user if the script runs successfully or it fails?
Thank you
How to automatically run Automation Tests on Gitlab Ci with Selenium and specflow with a .net Project ?
If this is something ,you are looking for then .
Here is the core part which is to setup the gitlab-ci.yml file :
Here is how the sample gitlab-ci.yml should look :
image: please give your own docker which can download .net stuff
variables:
DOCKER_DRIVER: overlay2
SOURCE_CODE_DIRECTORY: 'src'
BINARIES_DIRECTORY: 'bin'
OBJECTS_DIRECTORY: 'obj'
NUGET_PACKAGES_DIRECTORY: '.nuget'
stages:
- Build
- Test
before_script:
- 'dotnet restore ${SOURCE_CODE_DIRECTORY}/TestProject.sln --packages ${NUGET_PACKAGES_DIRECTORY}'
Build:
stage: Build
script:
- 'dotnet build $SOURCE_CODE_DIRECTORY/TestProject.sln --no-restore'
except:
- tags
artifacts:
paths:
- '${SOURCE_CODE_DIRECTORY}/*/${BINARIES_DIRECTORY}'
- '${SOURCE_CODE_DIRECTORY}/*/${OBJECTS_DIRECTORY}'
- '${NUGET_PACKAGES_DIRECTORY}'
expire_in: 2 hr
Test:
stage: Test
services:
- selenium/standalone-chrome:latest
script:
- 'export MSBUILDSINGLELOADCONTEXT=1'
- 'export selenium_remote_url=http://selenium__standalone-chrome:4444/wd/hub/'
- 'export PATH=$PATH:${SOURCE_CODE_DIRECTORY}/chromedriver.exe'
- 'dotnet test $SOURCE_CODE_DIRECTORY/ExpressTestProject.sln --no-restore'
artifacts:
paths:
- '${SOURCE_CODE_DIRECTORY}/chromedriver.exe'
- '${SOURCE_CODE_DIRECTORY}/*/${BINARIES_DIRECTORY}'
- '${SOURCE_CODE_DIRECTORY}/*/${OBJECTS_DIRECTORY}'
- '${NUGET_PACKAGES_DIRECTORY}'
Thats it .When you set up your project with this .git-lab-ci.yml ,90 % of your job is done .
The tests will run automatically in Gitlab ,whenever you commit something in your source tree or Tfs.
Thanks