I have a simple application and Nunit unit test project with 2 tests to test that app.
I've managed to have unit tests discovered and run locally in Visual studio 2013 but when trying to build application and run these unit tests on TFS 2013 test runner does not find tests ( both projects are built successfully.
No test found. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.
I've added "NUnit adapter with framework" package to Unit tests project according to answer in this question and committed changes to the TFS.
Unit tests projects is successfully built but still no tests are found. The Test dll fits the **\*Test*.dll naming pattern as configured in the Build Definition.
Why no tests are found? How I can make sure Test Runner is aware of my NUnit test adapter?
If you go to the build, and look in diagnostics (from the web portal), there is a Run VS Test Runner section.
In the same log, there is a Run MSBuild section, it should say "Successfully installed xxxx" where xxx is the name of your adapter package.
Something like:
Check to see that the test runner is successfully restored.
Found the reason - build agent folder in source settings of Build definition was not set correctly - instead of setting it somewhere under build agent working folder I set it outside of it that's why unit tests were not found..
I discovered this using Build process activity log (which was saved in drop folder in my case).this log contains very detailed information (much more than you can see in Visual studio) - there I found in which location test runner is looking for unit tests and then understood what i did wrong.
That's what I saw in log:
Run VS Test Runner00:00:00
Inputs
TestSpecs: BuildParameter[] Array
Enabled: True
ConfigurationsToTest: String[] Array
OutDir:
There were no matches for the search pattern C:\Builds\11\XXXX\Application -Gated\bin\**\*Test*.dll
Hope this will be helpful to someone else.
thanks to everyone who responded to my question!
Related
I'm in the process of moving from on-premise TFS to Visual Studio Online (VSTS) and I'm trying to create a new CI build definition as our local one is xaml based.
For our server based tests I'm using a Visual Studio Test task it it's running fine.
I also need to run Silverlight tests (as we currently have in the local xaml build). Currently I can run my SL unit tests from the command line and output a .trx test results file using the following command:
"C:\Program Files (x86)\StatLight\statlight.exe" -x .\path-to-silverlight.xap --ReportOutputFileType:TRX -r=.\statlight-results.trx
I can also run this using a CommandLine task in vsts (I'm already using a local build agent instead of a hosted one), but I don't know how to:
Get it to fail the build if there's a failing test.
Get the test results into the Test output tab in vsts.
I'll also need to do something similar to fail a build based on js unit tests as we migrate away from Silverlight, so the solution for the SL tests may help with running js tests also.
See if there is a switch that causes the command line to return the number of failed tests. If so you can use the return code to fail your build. Any value other than zero will fail build.
As for test results just add a publish test results task and point to the generated trx file. That is it.
I am working on a web app, solution in VS2013 with several project in it, one of them is used for unit tests - Ms Unit test framework. We have TFS 2013 set up to build the solution after each check in and trying to run the unit tests after each build.
From some reason unit tests are not executed after build, but also there is no error indicating that something broken or not found. I tried lot of things regarding editing the build definition - change the 'Test assembly file specification' to ***test*.dll, to use the exact dll name, changed the Target platform ...etc.
If I look into the log file I can see this:
Run VS Test Runner 00:00:15 Inputs TestSpecs: BuildParameter[] Array
Enabled: True ConfigurationsToTest: OutDir: The RunTestsActivity was
invoked without a value for Platform or Flavor. The values Any CPU and
Debug were used.
Test Run Completed. 0 tests executed.
What should I take a look, or what could be the problem ?
I'm using tfs 2010 and I want to create a tfsbuild defintion that runs unit tests.
I created unit tests and it works perfectly on my local machine.
I created a build definition, enabled tests, opened automated tests and put the vsmdi file.
the build succeed but it does not run any tests. In the build log file I don't find any information about the number of tests ran, it just tell me no test results ,besides the TestResults Directory is empty.
Has any one an idea.
I finally found the Problem. The build platform is x86 but for the test project it's AnyCPU, that's why tfsbuild did not execute tests.
Since running Metro apps headlessly is still a gray area: Running a metro app headlessly, I've recently decided to add a native unit test project to my Windows Metro app in hopes that I can find a way to run these unit tests in an automated fashion on the build server. Basically, I'm looking for something similar to MSTest.exe - a utility which is great for running tests from batch files and/or scripts.
In fact, I've tried using the new version of MSTest.exe that comes with VS11 on a generated test .dll, but it fails with the error:
"Unable to load the test container 'test.dll' or one of its dependencies... Error details: Could not load file or assembly file://test.dll' or one of its dependencies. The Module was expected to contain an assembly manifest."
Does MSTest.exe work with test containers that contain WinRT code? If not, is there a utility that will do what I want?
Edit: I just found out that MSTest does not support running tests on a Metro style app. Found here: http://msdn.microsoft.com/en-us/library/ms253138%28v=vs.110%29.aspx This really is too bad. I'm still hoping there's a utility out there that will work.
After blindly digging through the VS folders, I happened to find a new test runner under:
C:\Program Files\Microsoft Visual Studio
11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe
This utility allows you to execute WinRT unit tests from the command line. It's very similar to MSTest.exe.
There doesn't seem to be any documentation out there for this yet, but at least a help command exists.
If you are executing vstest.console.exe <filename>.dll then your tests do not get executed in appcontainer mode. You need to give <filename>.appx to execute tests in appcontainer mode. More info on how to execute tests for Windows Metro style apps from command line can be found at Running Unit Tests for Windows Metro style apps from Command Line
There is a documentation about (among other things) running unit tests for Windows Phone:
http://msdn.microsoft.com/en-us/library/windowsphone/develop/dn168930(v=vs.105).aspx
It describes also the command line way of doing it using vstest.console.exe.
It also gives a comparison of supported features between unit tests for Windows Phone and WinRT.
One important detail is that unit tests for WinRT cannot be run on a device. This is a pity and relevant to question How to automate non-interactive tests on Microsoft Surface
I'm trying to work out how to run xUnit tests from TFS 2010.
I found some articles on how to achieve that with the older version (http://jonnekats.wordpress.com/2009/05/07/integrate-xunit-tests-into-your-daily-team-build/, http://weblogs.asp.net/mehfuzh/archive/2009/08/25/configuring-team-build-using-xunit.aspx).
These approaches don't work anymore with TFS 2010 because builds are no longer MSBuild files but instead are Workflows with different tasks.
What I would like to achieve is something similar to what I've got on my dev machine:
- build everything
- run the .xunit project file
- check the results
All that I can see on the web are custom build steps which I can't/won't use because I'll have to configure them for each single unit test assembly and they'll probably get completely messed up in TFS.
Any input would be appreciated.
Why dont use "InvokeProcess" in TFS Build workflow and invoke xUnit command line interface with the necessary xUnit project files as parameters?
You can capture the output of the command line by setting InvokeProcess->stdout to write a build message.
Take a look at the xUnit work item Support VSTS Test Runner on codeplex. You can find there a VS2010 extension in Beta version.