I have created a C# Console App and a C# Unit Test by following this Microsoft Article here:
https://learn.microsoft.com/en-us/visualstudio/test/walkthrough-creating-and-running-unit-tests-for-managed-code?view=vs-2019
This works fine within Visual Studio.
I now want to create an Azure DevOps Pipeline and want it to call this Unit Test as part of the Pipeline. I have added the VsTest Test Assemblies to the Pipeline but I do not know how to make sure that this is actually running the unit test (like when I see it running within Visual Studio).
When I check the results of the build, I can see it has passed all the tests but even though this is marked as passed, there's an error reading:
"Error: System.IO.FileNotFoundException: Could not load file or assembly 'System.Runtime, Version=4.2.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified."
I know this is connected to the version of .Net Core that is being used, but I don't know where in my Pipeline I need to change this?
In order to select the version of .NET Core you want to use, you can use the Use .NET Core task. In YAML code, within a job, I specify the aforementioned task the following way:
- task: UseDotNet#2
displayName: 'Use .NET Core SDK $(dotnetSdkVersion)'
inputs:
packageType: sdk
version: $(dotnetSdkVersion)
In the beginning of the YAML file, you would have to set the dotnetSdkVersion variable that I am using here. You can do it the following way, to set it for .NET Core 3.1.201:
variables:
dotnetSdkVersion: '3.1.201'
Related
I have a Visual Studio .NET Core solution containing Unit Test projects. All the projects have a reference to the Microsoft.NET.Test.Sdk Nuget package is which according to my knowledge is a “must have” for unit tests execution.
I would like to perform static code analysis for unit test code (because it is also a code which is part of our deliverables). However, every time I am running Sonar Analysis as a part of Azure DevOps pipeline – I’m seeing “0” Lines of Code inside unit test *.cs files (and also other files inside unit test project) in the SonarQube report.
I have also found out interesting thing. In case I remove reference to the Microsoft.NET.Test.Sdk Nuget package – static code analysis is performed without any problems and code is visible for Sonar Qube scanner.
Also – the “dotnet build” command does not report any warnings for unit test projects when running on Azure DevOps build agent – even in case it should 😊.
My SonarQube Prepare task configuration pipeline file is very simple and looks like:
- task: SonarQubePrepare#4
displayName: Prepare SonarQube analysis
inputs:
SonarQube: 'SonarQube (XX)'
scannerMode: 'MSBuild'
projectKey: 'XXX-YYY'
projectName: 'XXX-YYY'
extraProperties: |
sonar.sources=src/**/*
Detailed dotnet build command run on Azure DevOps agent looks like:
/usr/bin/dotnet build /home/vsts/work/1/s/src/XXX.IntegrationTests/Tests/XXX.YYY.IntegrationTests.csproj -dl:CentralLogger,"/home/vsts/work/_tasks/DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b/2.181.0/dotnet-build-helpers/Microsoft.TeamFoundation.DistributedTask.MSBuild.Logger.dll"*ForwardingLogger,"/home/vsts/work/_tasks/DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b/2.181.0/dotnet-build-helpers/Microsoft.TeamFoundation.DistributedTask.MSBuild.Logger.dll" --configuration Release
I am wondering what is so special about dotnet core projects with Microsoft.NET.Test.Sdk reference. Is there build process somehow different for them? Is there any way to “force” static code analysis for them?
Any help will be appreciated.
Ok. It occurs this is "by design" SonarQube behavior. Details and methods how to overwrite it may be found on: https://github.com/SonarSource/sonar-scanner-msbuild/wiki/Analysis-of-product-projects-vs.-test-projects
Environment:
.Net Core 2.0
visual Studio Professional 2017
NUnit3
OpenCover (For code coverage)
ReportGenerator(For code coverage report)
Hi Nunit Team,
I am pretty familiar with MS Test framework and have worked on that extensively but was testing how suitable NUnit framework would be and so wanted to do a small POC around that. I created a small classlibrary and a unit test project. Below are the NuGet package been installed:
NUnit 3.7.1
But when I try to run the test cases from console runner like below I get exception saying System.Runtime, Version=4.2.0.0 assembly not found.
C:\Users\UN\.nuget\packages\nunit.consolerunner\3.8.0\tools>nunit3-console.exe D:\Code_Trials\BoMLibrary\BomTest\bin\Debug\netcoreapp2.0\BomTest.dll
Exception detail:
Errors, Failures and Warnings
Invalid : D:\Code_Trials\BoMLibrary\BomTest\bin\Debug\netcoreapp2.0\BomTest.dll
Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.
----> Could not load file or assembly 'System.Runtime, Version=4.2.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
Run Settings
DisposeRunners: True
WorkDirectory: C:\Users\Smukherjee.nuget\packages\nunit.consolerunner\3.8.0\tools
ImageRuntimeVersion: 4.0.30319
ImageTargetFrameworkName: .NETCoreApp,Version=v2.0
ImageRequiresX86: False
ImageRequiresDefaultAppDomainAssemblyResolver: False
NumberOfTestWorkers: 4
Am really not sure why so? No matter what I try not able to pass through this error. Moreover, when I try to install the assembly from Nuget I don't even find a assembly with version 4.2.0.0.
Tried installing the version 4.3.0.0 of System.Runtime but no luck. Can you please help on this.
Strange thing, I have got a sample unit test project (as part of opencover nuget install) which targets .Net Framework 4.5 and have NUnit2.6.7 package installed and when I try to run that using NUnit3 console it works like charm.
Even, installing NUnit3 latest stable version 3.10.1.0 doesn't help either. Also, it's surely not an Opencover issue since opencover console just passes the test run request to NUnit console. So the issue has to be around NUnit.
Let me know if you need more information around this.
Found the answer to this here https://github.com/nunit/nunit/issues/2763
NUnit Console doesn't run .NET Core tests - you need to use dotnet test instead. Take a look at this page:
https://github.com/nunit/docs/wiki/.NET-Core-and-.NET-Standard
I'm building a R# plugin and decided to cover it with tests. However, I noticed, that plugin tests cannot be run using the R# runner if plugin target version is the same as R# version.
For instance, if you try to run tests using JetBrains.ReSharper.SDK.Tests of version 2016.2.20160818.171542, you'll get the following error:
ERROR System.TypeLoadException:
Method 'RunTests' in type 'JetBrains.ReSharper.UnitTestRunner.nUnit26.BuiltInNUnitRunner' from assembly 'JetBrains.ReSharper.UnitTestRunner.nUnit26, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' does not have an implementation.
In previous versions the error was different.
Likely, that happens because R# runner and SDK assemblies make a mix.
Is there any way to configure project/R# to be able to run plugin unit tests using the R# runner? Likely, R# dev team uses their own runner.
Thanks!
Unfortunately, this is a bug in the ReSharper runner in 201.6.2. Something to do with having multiple copies of the same assemblies in the same process - the assemblies from the install, and the assemblies from the bin folder. Something's getting messed up along the way.
You can subscribe to, vote and track this issue: RSRP-460399
The workaround is to install an older version of ReSharper into a Visual Studio "experimental instance" aka "custom hive", and use this instance to develop and test the plugin. The normal instance can be used for other day-to-day development.
You can download an older version of ReSharper here (e.g. 2016.1.2), and in the installer, click Options and enter the name of a custom instance, e.g. "Plugins". Then create a new shortcut to Visual Studio and set the command line to "devenv.exe /rootSuffix Plugins" (or whatever you called the custom hive).
R# and SDK must be either of different major versions, or of binary compatible versions. Unfortunately, within EAP the internal APIs inside the test runner have changed, so R# and SDK from different EAPs are not compatible.
Since running Metro apps headlessly is still a gray area: Running a metro app headlessly, I've recently decided to add a native unit test project to my Windows Metro app in hopes that I can find a way to run these unit tests in an automated fashion on the build server. Basically, I'm looking for something similar to MSTest.exe - a utility which is great for running tests from batch files and/or scripts.
In fact, I've tried using the new version of MSTest.exe that comes with VS11 on a generated test .dll, but it fails with the error:
"Unable to load the test container 'test.dll' or one of its dependencies... Error details: Could not load file or assembly file://test.dll' or one of its dependencies. The Module was expected to contain an assembly manifest."
Does MSTest.exe work with test containers that contain WinRT code? If not, is there a utility that will do what I want?
Edit: I just found out that MSTest does not support running tests on a Metro style app. Found here: http://msdn.microsoft.com/en-us/library/ms253138%28v=vs.110%29.aspx This really is too bad. I'm still hoping there's a utility out there that will work.
After blindly digging through the VS folders, I happened to find a new test runner under:
C:\Program Files\Microsoft Visual Studio
11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe
This utility allows you to execute WinRT unit tests from the command line. It's very similar to MSTest.exe.
There doesn't seem to be any documentation out there for this yet, but at least a help command exists.
If you are executing vstest.console.exe <filename>.dll then your tests do not get executed in appcontainer mode. You need to give <filename>.appx to execute tests in appcontainer mode. More info on how to execute tests for Windows Metro style apps from command line can be found at Running Unit Tests for Windows Metro style apps from Command Line
There is a documentation about (among other things) running unit tests for Windows Phone:
http://msdn.microsoft.com/en-us/library/windowsphone/develop/dn168930(v=vs.105).aspx
It describes also the command line way of doing it using vstest.console.exe.
It also gives a comparison of supported features between unit tests for Windows Phone and WinRT.
One important detail is that unit tests for WinRT cannot be run on a device. This is a pity and relevant to question How to automate non-interactive tests on Microsoft Surface
I'm trying to work out how to run xUnit tests from TFS 2010.
I found some articles on how to achieve that with the older version (http://jonnekats.wordpress.com/2009/05/07/integrate-xunit-tests-into-your-daily-team-build/, http://weblogs.asp.net/mehfuzh/archive/2009/08/25/configuring-team-build-using-xunit.aspx).
These approaches don't work anymore with TFS 2010 because builds are no longer MSBuild files but instead are Workflows with different tasks.
What I would like to achieve is something similar to what I've got on my dev machine:
- build everything
- run the .xunit project file
- check the results
All that I can see on the web are custom build steps which I can't/won't use because I'll have to configure them for each single unit test assembly and they'll probably get completely messed up in TFS.
Any input would be appreciated.
Why dont use "InvokeProcess" in TFS Build workflow and invoke xUnit command line interface with the necessary xUnit project files as parameters?
You can capture the output of the command line by setting InvokeProcess->stdout to write a build message.
Take a look at the xUnit work item Support VSTS Test Runner on codeplex. You can find there a VS2010 extension in Beta version.