I just setup a new project (.net 4.6.2) using xUnit Tests. In order to get unit-testing in VSTS I added the visual-studio runner to the project (using nuget).
Now I'm wondering: how do I get XML-Reports of my testruns, so I can publish those results in VSTS?
Xunit logger can generate xml reports in the xunit v2 format (https://xunit.github.io/docs/format-xml-v2.html).
Add a reference to the Xunit Logger nuget package in test project
Use the following command line in tests
dotnet test --test-adapter-path:. --logger:xunit
Test results are generated in the TestResults directory relative to the test.csproj
A path for the report file can be specified as follows:
dotnet test --test-adapter-path:. --logger:xunit;LogFilePath=loggerFile.xml
loggerFile.xml will be generated in the same directory as test.csproj.
You can use Visual Studio Test task to run XUnit and it can publish the test result to VSTS.
Regarding Core xUnit test, you can generate the trx test result by specifying --configuration $(BuildConfiguration) --logger "trx" arguments in .Net Core Test task and add Publish Test Results task to publish test result (Choose VSTest in Test result format)
Related
using Visual studio 2017 community 15.6.4
Resharper 2018.2 with dotcover 2018.2.3
What I did :
Create a Universal Window UnitTestProject App.
Reference in there, via nuget, the following packages :
Nunit
Nunit3TestAdapter
Then I create a NUnit Test function in the TestClass :
[Test]
public void NunitTest()
{
}
I compile the sln And then ask Resharper to list my tests.
It returns nothing, and there is no Test-shortcut icon in front of my test function.
How can I have Resharper discover the NUnit tests in a UWP Application or library ?
Note : visual studio Test explorer finds and runs the sample MStest function that was created in the Test App. It also displays my NUnit test, but can't run it (error : No test is available in entrypoint\UnitTestProject1.exe. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.)
According to the issue on the NUnit repo on GitHub, NUnit does not support UWP.
I have an issue to automate unit tests. I created a Unit Test Project (with the template for Universal Windows Platform) and I written my test methods. I can execute my tests from the test explorer window by clicking "Execute all" button. But my issue appears when I try to execute my tests with MSTest from command line. The command returns "No test to execute."
Here is the command I use:
"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\MSTest.exe" /resultsfile:"C:\UnitTests\Results.trx" /testcontainer:C:\UnitTestProject\bin\x86\Release\UnitTestProject.exe /nologo
Thanks
Samuel
There is a detailed guide to running UWP Unit tests from command line here - Running Universal Windows unit tests from the command line . The problem is that you UWP EXE files cannot be run directly as usual Windows apps, but run in a sandbox, which means you use the APPX package to first deploy and install the test app and then run the actual tests.
For the life of me I can't get unit testing working in Visual Studio 2017 from the new msbuild-based netcoreapp1.0 xunit project template.
The requirement is for unit tests to work both inside Visual Studio (for the devs) and from dotnet test on the CLI for the automated build process however, I can't get either working consistently.
Here is what I have tried:
In an existing solution, create a new project and select .NET Core > xUnit Test Project.
Build project from Visual Studio, default test appears and runs successfully, now run dotnet test from powershell prompt, get:
> dotnet test
Test run for D:\...\bin\Debug\netcoreapp1.0\MyProj.dll(.NETCoreApp,Version=v1.0)
dotnet exec needs a managed .dll or .exe extension. The application specified was 'C:\Program'
Or dotnet test with csproj file:
> dotnet test MyProject.csproj
(same error as above)
> dotnet test ..\MySolution.sln
Couldn't find a project to run test from. Ensure a project exists in D:\...
Or pass the path to the project
If I add the xunit.runner.console or xunit.runner.msbuild nuget packages, it stops the unit tests working from inside Visual Studio.
How do I get both working at the same time?
Thanks!
The bug you're hitting is present in Preview 3 and fixed in Preview 4. They didn't escape the command line when executing it, and since dotnet.exe is installed into C:\Program Files\dotnet by default, it always fails.
If you want to continue to use Preview 3, the simplest work-around is to edit your system PATH environment variable, and replace C:\Program Files\dotnet with C:\Progra~1\dotnet.
I know this isn't a very good answer, but dotnet-test-xunit only support project.json files. VS 2017 forces you to switch to csproj files.
I found this on xunit twitter feed:
If you're trying use #xunit in VS2017 RC w/ .NET Core, remove dotnet-test-xunit and use xunit.runner.visualstudio 2.2 beta 4 instead.
With the latest RC.3 I was having issues with the tests not being discovered, and found out that when you run the built-in Test Explorer it says in the output that Microsoft.DotNet.InternalAbstractions 1.0.0 is missing. This was also issue in the previous versions of .NET Core, and the solution is the same, install the package from Nuget.
I'm trying to setup TeamCity 9.1.7 on a server and I'm encountering an issue when trying to add a Testing step.
Runner Type: Visual Studio Tests
Test engine type: VSTest
Test engine version: VSTest 2015
Test file names: C2.Tests\bin\Debug\C2.Tests.dll
Target platform: x86
When I added this Step, I was prompted to add a Config parameter:
Name: teamcity.dotnet.vstest.14.0
Value: C:\Program Files (x86)\MSBuild\14.0\Bin\MSBuild.exe
When I run all my steps I get an error in the Unit Testing step:
[14:23:17][Step 3/3] VSTest report watcher [14:23:17][VSTest report
watcher] No reports found for paths: [14:23:17][VSTest report watcher]
C:\BuildAgent\work\d28aa71801c772cb\TestResults*.trx [14:23:17][Step
3/3] Step Unit Testing (Visual Studio Tests) failed
I've had to do several things differently on this setup (on server) then when I was testing locally on my machine. For example setting up the 2015 Build Tools. I'm not sure where to look to correct this issue.
Please advise.
From the documentation:
The Visual Studio Tests runner integrates MSTest runner and VSTest console runner. Support for both frameworks enables TeamCity to execute tests and automatically import their test results.
The Visual Studio Test Runner requires Visual Studio Test Agent or Microsoft Visual Studio installed on the build agent.
https://confluence.jetbrains.com/display/TCD10/Visual+Studio+Tests
So you can use VSTests perfectly fine from version 10.
For VSTests, you need to specify a different directory for the config parameter:
teamcity.dotnet.vstest.14.0: C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe
This file will be available after installing the test agent (or visual studio).
If you still get the error then it means there is something wrong with the tests you have specified, you probably didn't reference to the assemblies correctly, make sure the path and .dll files exist.
There are two problems in the current configuration:
You are using as TestEngine the VSTest that will behave differently than the MsBuild
When you run the Test using the MSBuild this will not generate the .trx files that the Report watcher needs to display the execution results.
So in order to fix your problem you need to change the Test Engine to [MSTest]
This link may help you understand the capabilities of each Test Engine
Choose and configure a test runner
If this helps anyone, in my case this exact error was generated when I wrongly identified the DLL file containing the tests. Instead of "IntegrationTests.dll" I wrote "ItegrationTests.dll". You can identify this issue by looking into the build log where team city says:
Command line params:
and then lists paths to all test DLL files. If it's empty, it means the files you have specified in the build step have not been found.
If you're using VS Test runner, you'll see these lines telling you what happened:
[Step 3/3] No test source files were specified.
[Step 3/3] Process exited with code 1
[Step 3/3] VSTest execution failure
If you're using MS Test runner, you'll see these lines instead:
[Step 3/3] Please specify tests to run, or specify the /publish switch to publish results.
[Step 3/3] For switch syntax, type "MSTest /help"
[Step 3/3] Process exited with code 1
[Step 3/3] MSTest execution failure
Try building the tests using the MSBuild step before the testing step - it seems if it can't find the DLLs containing your tests you get the "No reports found" error
I have a project which builds and runs tests in Visual Studio Online. I recently added SpecFlow the the project and wrote a bunch of scenarios, some of which run against a LocalDb database. I was half expecting the tests to fail on VSO, but when I pushed the code up it ran all of my tests and they all passed. When I checked the logs it seems that my Specflow tests were not run.
Is there something I have to do to get the specflow custom tool which generates the actual unit tests to run as part of the build, so that VSO will find the tests after it has built?
Can I run what are effectively integration tests in VSO, creating and destroying databases in my tests?
Ok so this was a bit of a D'oh moment. VSOnline wasn't running my tests because Specflow was building NUnit test. Changing it to build MSTest tests meant that the generated files containing the tests had the correct attributes and VSOnline recognized them and so happily built and ran them.
And using the LocalDb during the tests was fine as well, all worked like a dream.