I am attempting to get the Resharper test runner to recognize my MSTest unit tests via Gallio.
I have the following installed:
VSTS 2005 8.0.50727.762
Resharper 4.1
Gallio 3.0.0.285
I am also running Windows XP x64.
The unit test options only shows NUnit as being available. I am thinking that I must have some versioning wrong. Can someone point me in the right direction? Am I barking up the wrong tree and this is only works in VS2k8?
UPDATE: Well I updated Gallio to GallioBundle-3.0.4.385-Setup and it now shows up in the unit test options for R#. But I get the following error when running tests in either R# or Icarus:
Failures Cannot
run tests because the
MSTest executable was not found
Thanks
I'm not sure if this applies to your question, but the latest news on the Gallio site states :
Gallio and ReSharper 4.0
A few people have asked when Gallio will
support the final release of ReSharper
4.0. Unfortunately there were some last-minute breaking changes between
ReSharper 4.0 Beta and the Final
release. So the ReSharper plugin in
Gallio Alpha 3 Update 2 will not work
anymore. No matter... we plan to
release another update of Gallio later
this week.
Update: just saw this applied to an earlier version than you were using so disregard that.
It looks like this is a Gallio problem. It appears to only support the version of MSTest that comes with VS2k8. The XML format for vsmdi has changed between versions.
ReSharper 4.5 supports MSTest out of the box.
There's also a MSTest ReSharper plugin that was recently released:
http://www.sneal.net/blog/2009/01/24/MSTestReSharper4Plugin.aspx
I've been able to use it successfully with ReSharper 4.1 on Visual Studio 2005.
You do not need resharper to support standalone mstest installation. I explain how to make a standalone mstest installation without the VS in my post here - http://www.shunra.com/shunrablog/index.php/2009/04/running-mstest-without-visual-studio
Related
My code has NUnit tests. My colleague is able to run the tests from within Visual Studio on their desktop, however I can't run them on mine. When I try to use Test Explorer and "Run All", it just builds the project. Other options, such as Run Tests and Debug Tests also give the same result.
I have NUnit referenced in my project, why can't I run the tests from within Visual Studio?
Out of the box, Visual Studio doesn't have support for running tests built for the NUnit framework. As has been said in the the comments, your colleague probably has some kind of add-in like Resharper installed which adds in support for running Nunit tests.
If you don't have a license for whatever tool they're using, then you can still have visual studio integrated support for running/debugging NUnit tests by installing the NUnit Test Adapter. This is a free plugin that you can install into your solution via NuGet.
He may also be running the MS test adapter which you can get from the nunit site
I added the "xUnit.net runner for Visual Studio" v0.99.8 via Extensions Manager, but when I open the Test Explorer window, it does not seem to pick up any of my unit tests. Also, the Resharper 9 EAP does which is the only version of Resharper that supports VS2015 does seem yet to have the plugin for xUnit Test Runner.
How then, can I run xUnit Unit Tests in VS2015 Preview?
You can find the answer here: http://blogs.msdn.com/b/webdev/archive/2014/11/12/announcing-asp-net-features-in-visual-studio-2015-preview-and-vs2013-update-4.aspx
Visual Studio supports running and debugging for ASP.NET 5 xUnit tests through test explorer. All you need to do is add the xUnit dependencies and test commands to the test project's project.json file, as shown below (NOTE: To install the xUnit packages you will need to add https://www.myget.org/F/aspnetvnext/api/v2 as a NuGet package source):
"dependencies": {
"Xunit.KRunner": "1.0.0-beta1"
},
"commands": {
"test": "Xunit.KRunner"
},
If anyone is asking how to add https://www.myget.org/F/aspnetvnext/api/v2 as a NuGet package source...
here are the steps:
In Visual Studio 2015 Preview go to Tools -> Options -> NuGet Package Manager -> Package Sources
Click the Plus (Add) button at the top (see image below)
Enter the Name and Source like in the image below (NOTE: be sure to click the Update button after entering the Name and Source)
Happy coding!
You need to add reference to these 3 nuget packages:
"xunit": "2.1.0.0-beta1-build2945",
"xunit.runner.aspnet": "2.1.0.0-beta1-build60",
"xunit.runner.visualstudio": "2.1.0.0-beta1-build1051"
Check this article for more info:
http://blog.developers.ba/unit-integration-testing-in-asp-net-5-and-visual-studio-2015-using-xunit-net/
With visual studio 2015 RC I could not get tests to work with the following:
"xunit": "2.1.0-beta2-*",
"xunit.runner.dnx": "2.1.0-beta2-*"
But tests started showing up in the Test Explorer after changing the casing of the first letter "x" to "X"
"Xunit": "2.1.0-beta2-*",
"Xunit.runner.dnx": "2.1.0-beta2-*"
When I change it back to lower case it fails. I have not yet found a way to get these tests to show up in the latest version of Resharper at this time.
Here is the post I followed to get this far and everything worked except for the casing for some reason:
http://xunit.github.io/docs/getting-started-dnx.html
I know this will be out of date soon, but hopefully it helps someone in the mean time :)
Regarding managing / running the tests through test explorer, it is available through nuget.
I had to install package xunit.runner.visualstudio version 0.99.9-build1021 in the test projects.
After building the solution, the tests showed up fine and I was able to run them.
BTW, this was for a windows service but should work for ASP.NET/Web projects.
first of all, I've never used unit tests with ReSharper before, so I don't know how to debug it. The problem is that if run MS unit tests by VS test explorer everything works fine, but if I run it through resharper's Unit Test Sessions it just does nothing and stays in a pending state, even with just one test starting with Assert.Fail. I am seeing no load on CPU or anything.
I am using ReSharper 8.0.14.856 with Visual Studio 11.0.50727.1 . Searches bring me many results with older versions, but I guess those should work fine?
As I see you use VS2012 RTM, but according to http://www.jetbrains.com/resharper/download/index.html -> "System requirements", you need to have at least Visual Studio 2012 Update 3 (there were some changes in MSTest API between RTM and Update 3, ReSharper 8 supports the latest ones). So, please install Update 3 to your VS2012 installation and you will be able to run MSTest tests.
I have a xUnit.net Test as follows:
static class MyTestClass
{
[Fact]
static void MyTestMethod()
{
}
}
The xUnit plugin for VS 2012 says:
No tests found to run.
TestDriven.net runs it fine but mentions something about Ad hoc:
1 passed, 0 failed, 0 skipped (see 'Task List'), took 0.47 seconds (Ad hoc)
TeamCity, xunit.gui.exe and xunit.console.exe and Visual Studio also can't find TestMethod
(I've got xunit.runner.visualstudio installed and VS is seeing some tests.)
What gives?
TL;DR your Test Classes must be public (but your Test Methods can be private and/or static)
For reasons of efficiency, the xUnit authors have opted to not use BindingFlags.NonPublic when searching for Test Classes in the runner (the MSIL metadata tables don't index private(/internal) classes to the same degree hence there is a significant performance difference in the relative efficiency that Reflection can thus achieve).
As a result of the above, the fact that your class is private means it doesn't get picked up.
The fact that the Test Method is private and static is fine - xUnit by design since 1.0 has supported both those aspects.
Note that the Visual Studio xUnit Runner extension, xunit.console.exe (and the GUI), the xunit MSBuild task, Resharper and CodeRush are all consistent in honouring this (although arguably they [especially the latter two] could do more to flag when a Test Class (i.e. class [potentially indirectly] containing Fact-derived annoations) is private).
The reason TestDriven.net runs your test is that the Author of TestDriven.net has put great effort into making it Just Work. It internally uses a special Test Runner wrapper/shim (termed the Adhoc Runner) to run your test. Be aware that the method is actually not being run via the xUnit.net runner and hence any attributes you put on your test that have side effects will not be triggered.
Notably NUnit (and I'm pretty sure MSTest) do use private reflection [and hence pick up tests in private classes] which is probably why it never seemed an important thing for you to worry about before.
Note: A side effect / trick enabled by this is that you can make a Test Class private as a quick way of Skipping all tests in a Test Class [and any nested classes]. (Sadly the cases on this planet of this being used unintentionally vastly outnumber the intentional cases of this though!)
This answer is for VS 2013, but the steps are essentially the same for VS 2012. This applies for running via ReSharper's unit test functionality.
Install the xUnit.net runner for Visual Studio 2013 (be careful running Visual Studio as an Administrator, or the tests may not run when running the IDE as a non-Admin):
a. From within Visual Studio 2013, go to Tools -> Extensions and Updates -> Online
b. Search for xUnit.net runner for Visual Studio 2012 and 2013
c. Then download (install) it. If upgrading to VS 2013 from VS 2012, it is suggested that this be uninstalled, and then re-installed.
d. Restart Visual Studio.
If ReSharper is installed, install the xUnit.net test runner plugin :
(NOTE: Since ReSharper 2016.1, xunit support is built in to ReSharper, meaning the xunit plugin is no longer required.)
a. In Visual Studio 2013, Navigate: Resharper -> Extension Manager.
b. On the left, select Online.
c. Search for “xunit.net”. Select the “xUnit.net Test Support”. Click Install.
d. Restart Visual Studio 2013.
“Clean” the solution
a. In the IDE, in Solution Explorer, right-click the solution, and choose “Clean”.
b. Re-compile.
c. Now, when right-clicking a [Fact] attribute, select Resharper’s “Run Unit Tests” (as opposed to the default “Run Tests”)
Troubleshooting running with XUnit:
If problems running the [Fact] tests with XUnit persist, it might
be necessary to manually remove the xUnit package from any/all of the
following folders (review content for the xunit DLLs, then delete
xUnit folder if found):
C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Extensions\
C:\Users\<username>\AppData\Local\Microsoft\VisualStudio\12.0\Extensions\
As for ReSharper, try un-installing and re-installing the
xunitcontrib library (xUnit.net Test Support). I have noticed once
when un-installing, some error messages flashing past. I grabbed a
screen-shot at one point, and it listed:
Access to the path
C:\Users\<username>\AppData\Local\JetBrains\ReSharper\vAny\packages\xunitcontrib.1.3.0\ReSharper\v8.1\plugins\xunit.dll
is denied.
... and the same for the other DLLs in that directory
To resolve this, delete the C:\Users\<username>\AppData\Local\JetBrains\ReSharper\vAny\packages\xunitcontrib.1.3.0\ directory after uninstalling from Visual Studio, then run Visual Studio as a non-administrator, and re-install via ReSharper (Resharper -> Extension Manager)
From http://xunit.github.io/docs/getting-started-desktop.html#run-tests-visualstudio:
If you're having problems discovering or running tests, you may be a
victim of a corrupted runner cache inside Visual Studio. To clear this
cache, shut down all instances of Visual Studio, then delete the
folder %TEMP%\VisualStudioTestExplorerExtensions. Also make sure your
project is only linked against a single version of the Visual Studio
runner NuGet package (xunit.runner.visualstudio).
I had the same issue in VS2017 RC, .NET core 1.1 project. Updating xunit.runner worked for me,
Install-Package xunit.runner.visualstudio
(As referred to by #Kyle in the comments on the other answer) The same No tests found to run message can result from using NuGet to get xUnit.dll and ending up with version 2.0.0 (which is currently marked as prerelease as some core functionality like discovering of v1 tests etc. has yet to be implemented in that branch).
The resolution in this case is to select Stable Only versions (as opposed to Include Prerelease) in the NuGet package manager.
I've been having this issue with .NET Core for a while now where a test class or a test method is not being discovered. The following fix works for me:
Open a command prompt window.
Change to the project directory.
Build the project running the following command:
dotnet build
NOTE: Building from Visual Studio.NET will not work! <<<<<<<<<<< IMPORTANT!
Run the tests: Test --> Run --> Test All - CTRL+R +A (this will discover the new test(s) - but not run the the new test(s).
Run the tests again.
In my case, in order to see any tests, I had to complete the following steps:
(All installed through NuGet Package Manager)
Install xUnit v2.0.50727
Install xUnit.extensions v2.0.50727
Navigate to the following link and follow the steps outlined in the documentation: http://xunit.github.io/docs/running-tests-in-vs.html
I'm using Visual Studio 2013 Premium. (Resharper NOT installed)
For me, the combination of my test class and test method names were too long; xUnit appears to have some cap on this combination.
Shortening the name of just the test method allowed xUnit to discover that single test. Shortening the name of the entire class allowed xUnit to discover all tests in the class.
Threshold of class name + method name appears to be 172 characters.
My problem was that I updated xunit.runner.visualstudio to version 2.4.5. However, the project I am working for, is for .NET Standard 2.0. Therefore, I had to downgrade to version 2.4.3 of xunit.runner.visualstudio, since it supports ".NET 2.0 or later". But since version 2.4.4, ".NET Core 3.1 or later" is supported.
I installed the newest Gallio release 3.2.603 on a box without Visual Studio that will be our CI server. The NAnt script successfully built the .Net 4.0 projects but when it tries to run MSTest tests the following error occurs.
[gallio] Gallio NAnt Task - Version 3.2 build 601
[gallio] [error] Assembly XXXXXXXXXXXXXX
[gallio] Cannot run tests because the MSTest executable was not found
Am I wrong to expect that Gallio is able to run MSTest tests without the MSTest executable or any other Visual Studio components installed?
Ok, I got an answer from Gallio support that it actually requires MSTest.exe to be present on the box. I'll test the package that Aseem suggested whether it provides enough stuff to run the tests.
Update: I installed Microsoft Visual Studio Test Agent 2010 from the package. I had to add the string value InstallDir to [HKLM\SOFTWARE\Microsoft\VisualStudio\10.0] and set the value to something like
“C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE”
Now my CI box runs those tests perfectly.
I dont know anything about Gallio, but just wanted to share that with visual studio 2010, you dont really need visual studio to run MSTest tests. You can simply install test agent (a very light-weight sku) which additionally installs MSTest. This test agent installer can be installed from here.
Confirmed this worked for me. Running Sonar on a box with VS2010. Sonar could not find MSTest until I added this registry value.