I have an issue to automate unit tests. I created a Unit Test Project (with the template for Universal Windows Platform) and I written my test methods. I can execute my tests from the test explorer window by clicking "Execute all" button. But my issue appears when I try to execute my tests with MSTest from command line. The command returns "No test to execute."
Here is the command I use:
"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\MSTest.exe" /resultsfile:"C:\UnitTests\Results.trx" /testcontainer:C:\UnitTestProject\bin\x86\Release\UnitTestProject.exe /nologo
Thanks
Samuel
There is a detailed guide to running UWP Unit tests from command line here - Running Universal Windows unit tests from the command line . The problem is that you UWP EXE files cannot be run directly as usual Windows apps, but run in a sandbox, which means you use the APPX package to first deploy and install the test app and then run the actual tests.
Related
I'm working on a project with multiple Unit Tests. I have a visual studio .sln file with around 10 XXPrj in it. Those projects are made with Google Test. Everything works well if I want to run them using Visual Studio 2019, I can build and run the unit tests.
I would like to know what is the best way to run them an automated way with commandline. Purpose is to then integrate this commandline stuff in a jenkins to have everything automated.
Build
Building a Visual Studio solution/project through the command line is done with msbuild.exe. It works best to add the path of MSBuild to the PATH environment variable.
MSBuild is usually installed somewhere in the Visual Studio folders. E.g. on my machine the path is as follows:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin\MSBuild.exe
Build the solution containing all your projects as follows:
msbuild.exe Example.sln
# Or if you want to build a release version or add additional arguments
msbuild.exe Example.sln /property:Configuration=Debug
See MSBuild CLI Docs for more options.
Side Note: Jenkins has a msbuild plugin you can use with a build step called "Build a Visual Studio project or solution using MSBuild" (Important: this does not install MSBuild, it only provides a GUI to use MSBuild in a build plan).
Run Tests
To run the tests you have two options:
Run each project's executable in your build pipeline and the executable's exit code will indicate the success/failure of the unit tests for that project. However, you will need to call each executable separately; or
Use the vstest.console.exe in combination with the Google Test Adapter
You can use the Google Test Adapter the same way in which Visual Studio uses it when you click Test -> Run -> All tests to discover & execute tests in your projects.
In my environment, vstest.console.exe is located here:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\Extensions\TestPlatform\vstest.console.exe
You also need to provide the path to the test adaptor. Then execute all the tests as follows:
# Assuming vstext.console.exe is included in the PATH
# and the current working directory is the relevant project executable
# output folder:
vstest.console.exe Project1.exe Project2.exe Project3.exe /TestAdapterPath:"<path to adapter>"
Once again the path is hidden somewhere in the Visual Studio Folders. I found it through searching for GoogleTestAdapter.TestAdapter.dll. On my machine it is located in:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\Extensions\drknwe51.xnq
Conclusion
Thus, an automated way to build and run GoogleTest unit tests, which are split over multiple projects, with the commandline can be performed in these two steps:
Build the solution/projects using msbuild.exe
Run the tests using vtest.console.exe in combination with the Google Test Adapter
For the life of me I can't get unit testing working in Visual Studio 2017 from the new msbuild-based netcoreapp1.0 xunit project template.
The requirement is for unit tests to work both inside Visual Studio (for the devs) and from dotnet test on the CLI for the automated build process however, I can't get either working consistently.
Here is what I have tried:
In an existing solution, create a new project and select .NET Core > xUnit Test Project.
Build project from Visual Studio, default test appears and runs successfully, now run dotnet test from powershell prompt, get:
> dotnet test
Test run for D:\...\bin\Debug\netcoreapp1.0\MyProj.dll(.NETCoreApp,Version=v1.0)
dotnet exec needs a managed .dll or .exe extension. The application specified was 'C:\Program'
Or dotnet test with csproj file:
> dotnet test MyProject.csproj
(same error as above)
> dotnet test ..\MySolution.sln
Couldn't find a project to run test from. Ensure a project exists in D:\...
Or pass the path to the project
If I add the xunit.runner.console or xunit.runner.msbuild nuget packages, it stops the unit tests working from inside Visual Studio.
How do I get both working at the same time?
Thanks!
The bug you're hitting is present in Preview 3 and fixed in Preview 4. They didn't escape the command line when executing it, and since dotnet.exe is installed into C:\Program Files\dotnet by default, it always fails.
If you want to continue to use Preview 3, the simplest work-around is to edit your system PATH environment variable, and replace C:\Program Files\dotnet with C:\Progra~1\dotnet.
I know this isn't a very good answer, but dotnet-test-xunit only support project.json files. VS 2017 forces you to switch to csproj files.
I found this on xunit twitter feed:
If you're trying use #xunit in VS2017 RC w/ .NET Core, remove dotnet-test-xunit and use xunit.runner.visualstudio 2.2 beta 4 instead.
With the latest RC.3 I was having issues with the tests not being discovered, and found out that when you run the built-in Test Explorer it says in the output that Microsoft.DotNet.InternalAbstractions 1.0.0 is missing. This was also issue in the previous versions of .NET Core, and the solution is the same, install the package from Nuget.
I'm using Visual Studio 2013, and I have a legacy solution that I need to modify, containing a a console application and a unit test project.
I want to run the console application project, and while it's running I also want to run one of my unit tests.
I run my console application, but when I go to the test explorer to run my unit test, the option is grayed out.
why is this, and how can I run my unit test?
You don't mention why you want to do this, so I'll just answer blind.
If you run your console app without debugging, then you can run your unit tests at the same time.
To run without debugging, use Ctrl+F5 (DEBUG -> Start Without Debugging).
Alternatively, open your console app's project folder, go to the bin\Debug folder and double-click on <your_app>.exe.
FWIW it doesn't sound like a very good idea to do this. Unit tests are meant to be runnable completely separately from your application.
I'm using TFS, it works fine. Now I want to add tests and code coverage. So I create a Local.testsettings. Running tests on my computer runs fine.
Configuration of the TFS Build is as follows:
- Automated test with test assembly file specification and the same test settings file (as recommended, instead of using a .vsmdi file)
When launching a build with TFS, my tests don't get executed :
Run MSTest for Test Assemblies
The MSTestActivity was invoked without a value for Platform or Flavor. The values Mixed Platforms and Debug were used.
C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe /nologo /usestderr /testSettings:"C:\Builds\1\MyProject\Sources\MyProject\Main\Source\MyProject\Local.testsettings" /searchpathroot:"C:\Builds\1\MyProject\Binaries" /resultsfileroot:"C:\Builds\1\MyProject\TestResults" /testcontainer:"C:\Builds\1\MyProject\Binaries\MyProject.Tests.Module1.dll" /maxpriority:"1" /minpriority:"1" /publish:"http://tfsserver:8080/tfs/Test" /publishbuild:"vstfs:///Build/Build/433" /teamproject:"MyProject" /platform:"Mixed Platforms" /flavor:"Debug"
Loading C:\Builds\1\MyProject\Sources\MyProject\Main\Source\MyProject\Local.testsettings...
Loading C:\Builds\1\MyProject\Binaries\MyProject.Tests.Module1.dll...
Starting execution...
No tests to execute.
There are no results to be published.
So it finds the assembly, but MSTest doesn't actually runs the tests.
Any hint is welcome.
Looking at this:
/maxpriority:"1" /minpriority:"1"
It looks like your priority criteria may be the problem.
I installed the newest Gallio release 3.2.603 on a box without Visual Studio that will be our CI server. The NAnt script successfully built the .Net 4.0 projects but when it tries to run MSTest tests the following error occurs.
[gallio] Gallio NAnt Task - Version 3.2 build 601
[gallio] [error] Assembly XXXXXXXXXXXXXX
[gallio] Cannot run tests because the MSTest executable was not found
Am I wrong to expect that Gallio is able to run MSTest tests without the MSTest executable or any other Visual Studio components installed?
Ok, I got an answer from Gallio support that it actually requires MSTest.exe to be present on the box. I'll test the package that Aseem suggested whether it provides enough stuff to run the tests.
Update: I installed Microsoft Visual Studio Test Agent 2010 from the package. I had to add the string value InstallDir to [HKLM\SOFTWARE\Microsoft\VisualStudio\10.0] and set the value to something like
“C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE”
Now my CI box runs those tests perfectly.
I dont know anything about Gallio, but just wanted to share that with visual studio 2010, you dont really need visual studio to run MSTest tests. You can simply install test agent (a very light-weight sku) which additionally installs MSTest. This test agent installer can be installed from here.
Confirmed this worked for me. Running Sonar on a box with VS2010. Sonar could not find MSTest until I added this registry value.