Running NUnit tests with Gallio - unit-testing

I'm trying to write some NUnit tests for a project. I add a new Class Library to the solution. Is this the correct project type?
I added a reference:
using NunitFramework;
and added my tests.
I have Gallio installed, but I'm unsure about whether it's configured properly. I'm new to using this, and I'm hoping someone can tell me how you actually run the tests.
It was my understanding you could right click and select run all tests. Alternatively I thought you could click on the [Test] attribute and click run or debug.
Alternatively, how do I load tests into the Test Results window to run debug them?

Basically it's correct. Your unit test should look like this:
using System;
using NUnit.Framework;
namespace Demo
{
[TestFixture]
public class SampleTestFixture
{
[Test]
public void SampleTest()
{
// my test here...
}
}
}
Compile your test project (a class library is OK) in debug mode. Now you can run your test suite within any runner that supports NUnit:
Any Gallio test runner (Echo, PoSh snap-in, Icarus, etc.) There is a nice tutorial explaining how to use the Gallio test runners here. The examples use MbUnit instead of NUnit, but it's all the same.
The built-in NUnit GUI or console runner.
The built-in VS test runner (interfaced by Gallio) if your version of Visual Studio includes the MS testing tools. You should be able to see your tests in the test tool window, but there is a nasty trick to make VS recognize your project as a test project if you did not create it initially as such.
You can also use TestDriven.Net which is lightweight and fast test runner that comes as an add-in for VS. IMHO, it's certainly the most simple and efficient way to run your unit tests directly from the VS IDE.

Related

VS 2015 with NUnit does not discover test cases

I am trying to use NUnit for unit testing in my code base. When I try to run my test cases using ReSharper, I get:
"System.IO.FileNotFoundException: Could not load file or assembly
'ServicesTests.dll' or one of its dependencies.
ServicesTests is my class lib for my unit tests.
I have also tried to install the NUnit Test Adapter as described here but do not see my test cases in the Test Explorer.
Any ideas on what might be happening? Below are images of my class library from the solution explorer, the test explorer, ReSharper screen.
I was able to test using Nunit3 and VS2015 with Test Explorer after I changed the type of class library I was creating. Initially I was creating a Class Library (Package) and when I switched to Class Library it worked (see image below). I am still not able to use Resharper since I am using Nunit3 beta and that is currently unsupported by Resharper.
I have a similar issue also. The best answer I've come across to solve it is try and rebuild the solution (cleans and builds), or just resort to MS Test. It's a pain to convert the tests though...
Hope this helps.
EDIT:
My NUnit runs, but no tests "exist", despite having a project, classes, and methods all correct.
ReSharper does not support running unit tests in DNX projects https://youtrack.jetbrains.com/issue/RSRP-446679.
I know this is an old question but I had a similar issue recently in VS 2017. Use the Fusion Log Viewer (aka "Assembly Binding Log Viewer") to see what assembly is failing to load:

Running BOOST unit tests from Visual Studio 2010

I am just starting to work with QuantLib project. There are quite a few BOOST unit tests defined, hence the question: is there an easy way to run these tests selectively from Visual Studio (and see results)?
I've only used unit testing in .Net languages so far, and there you can either use the built-in test runner if you use MsTest, or some add-on (I used Resharper and TestDriven.NET) for other frameworks. There you can choose which tests you run, and you can run them with the debugger attached (which is unfortunately useful sometimes). Are there similar add-ons for BOOST tests in unmanaged C++?
Yes you can!
If your boost unit test project is defined correctly, then you can run the entire test project from VS10 debugger by clicking
right click on project --> Debug --> Start new instance
This will run the tests inside VS framework which will give you all the debugger goodies.
For running specific test suites/cases, just edit the project command line arguments under
right click on project --> Properties --> General --> Debugging
and add the parameters according to the links in the other answers
you can run any test case or test suite via command line. running tests by name
it even allows wildcards.
ReSharper C++ supports discovery and selective running/debugging of Boost tests (with Boost 1.59 and later).
I'm using the Boost Unit Test Adapter. It's free and has a very nice output. It can also list the time the single tests consumed and you can start single tests and group of tests.
You can download it here:
https://visualstudiogallery.msdn.microsoft.com/5f4ae1bd-b769-410e-8238-fb30beda987f?SRC=VSIDE

How to write a simple WP7 unit test using xUnitContrib?

I'm working from the XunitContrib codeplex page and toward the bottom it lists these steps
For Windows Phone 7
Follow along with this blog post
Create a Windows Phone application
Add references to:
Microsoft.Silverlight.Testing.dll (Silverlight 3 version - included in release)
xunit-silverlight-wp7
xunit.extensions-silverlight-wp7
xunitcontrib.runner.silverlight.toolkit-wp7
Visual Studio may display warnings about including Silverlight 3 assemblies. Ignore it, these are the right files
Add [Fact] based tests and run the app (note- the blog post mentioned isn't using fact based tests so I'm even more confused ...)
But after I add the above mentioned dll's and start with something like the below ... resharper and I can't seem to hookup the testing harness enough to compile. Has anyone actually wired up a unit test with xunit for WP7?
public class MyFirstWp7Test
{
[Fact]
public void Can_Run_Test_For_WP7()
{
var x = "hello world";
Assert.Equal("hello world", x);
}
}
You can't use resharper to run Silverlight or Windows Phone tests, unfortunately. The xunitcontrib resharper runner is for the desktop clr only (newer builds ignore files in a silverlight/wp project so you don't even see the unit test marker in the editor).
You need to actually deploy to the device or emulator, run your application and drive the tests through the UI of the Silverlight Unit Testing Framework. You can filter by tag, test class or test method name, but it all has to be in the phone's environment.
I finally found a way to test-drive my app on the WP7 platform using NUnit and ReSharper.
I did a short screencast on my blog for a full "how-to" of sorts
http://toranbillups.com/blog/archive/2011/07/24/Test-Driving-My-Windows-Phone-7-App

Is there an MS Test Runner that is faster

MS Test is killing me.
It is so slow compared to NUnit. I am stuck with it because I need to be able to get Test Results into TFS easy. (Plus it works better with Pex and I am using that too).
But I would really like it to go faster. Even just a bit faster would be nice.
Has anyone made a test runner for MS Test tests that goes faster than the one embedded in Visual Studio?
The Gallio platform provides a test adapter for MSTest. You can then run your tests with any runner supporting Gallio, including built-in runners such as Echo or Icarus, and 3rd party runners such as TD.Net or R#.
The non-GUI test runners are usually slightly faster. Thus you may want to try Echo or the PoSh snap-in.
You should be able to integrate NUnit into TFS with some custom build tasks (which some people have done). If you really wanted to get crazy you could transform NUnit test results into .TRX files and import them directly using the MSTEST command line options
TestDriven.Net is probably the fastest one.
Then goes msTest runner embedded into Visual Studio.
I found that ReSharper test runner is slower than built-in, especially if the solution is big.
At the time I was researching, ReSharper seemed to do a full rebuild of the solution, where msTest was just building changed projects.
So your best choice would be TestDriven.Net
Albacore has an MSTest task.
https://github.com/derickbailey/Albacore/wiki/MSTest-Task
You could write a simple rake file that built your project and then ran the MSTest task.
My recommendation is that, initially, you write a project to only run the albacore MSTest task and time how long that takes.
PS: I wouldn't use Gallio because, in my experience, that test runner takes 6 seconds longer than TestDriven.net to run NUnit tests.
PS 2: My real recommendation is to ditch MSTest for xUnit.net or NUnit

Running VSTS tests without mstest.exe

From reasons I won't get into, all our unit tests are using the VSTS test framework. I now want to create an MSBuild script that runs the tests, but I don't want to use mstest.exe from various reasons (it's slower, requires Visual Studio installation everywhere, I need to maintain testrunconfig, etc.)
I've seen that TestDriven.net and TeamCity are able to run VSTS tests 'NUnit style', without using mstest.exe. Are you aware of any standalone command line utility that does this?
You can execute Team System Tests (MSTest) in NUnit if you use a special NUnit Addin that recognizes the MS Test Attributes (TestClass, etc).
Exact Magic Software has an open-source "test-adapter" that can do this.
UPDATE: I've reworked Exact Magic's Msts NUnit Adapter for NUnit 2.5.2.
Download here: http://snippetware.googlecode.com/files/ExactMagic.MstsAdapter.zip
Read more about it here: http://www.bryancook.net/2009/10/mstest-nunit-adapter.html
It seems like TeamCity is simply leveraging Gallio to run VS tests. Gallio appears to have msbuild integration and sounds perfect but after a closer look it seems that it would require a VS install just like MSTest as it appears to depend on MS exes:
The plugin enable condition was not satisfied: '${process:DEVENV.EXE} or
${process:VSTESTHOST.EXE} or
${process:QTAGENT.EXE} or
${process:QTAGENT32.EXE} or
${process:QTDCAGENT.EXE} or
${process:QTDCAGENT32.EXE}'.
Host process exited with code: 0
That being said it sounds like at least one person has got it working:
Christoph De Baene - Running MSTest without Visual Studio
It is possible to run MSTests without installing Visual Studio. See how-do-i-use-mstest-without-visual-studio.
I did this so that I could run my tests as part of my CI process. (I am using CC.NET for my CI solution).
I am in a similar situation as you, in that I want to use TestDriven.NET to get code coverage stats. But, I am running into problems. My first problem is that I am using AssemblyInitialize attributes to initialize a database connection. This isn't supported by NUnit so about half of my tests fail whereas they run fine under MSTest.
So, it seems that translating tests from one test framework to another has pitfalls. If you are aware of that, then go forth, but it might be better to try and keep consistent on one test framework.
We run VSTS tests using msbuild TestToolsTask on a Cruise Control server. This does not use the MSTEST executable -- the condition you ask for -- but does use a variety of TFS dependencies.
Note that we are migrating tests off of the VSTS test framework for NUnit, mostly because we can create extensions for NUnit to perform useful tasks.