Is there actually a good way to unit test MonoTouch projects using NUnit and the MonoDevelop test runner?
I know there is the official MonoTouch unit tests project type, but running tests within the simulator isn't the way I want to go. For now I want to run tests with the MonoDevelop test runner, later everything should work with Jenkins (CI).
I know the limitations about UI specific code, so everything I want to test has nothing to do with MonoTouch itself, it's all about business logic placed within separete projects.
By adding tests to MonoTouch Library type projects, I am getting System.IO.FileNotFoundException's as described here: http://ben.phegan.name/index.php/2011/02/28/monotouch-and-unit-testing/
By using a separate NUnit test project, I can't reference my system under test, because its project type is of type MonoTouch library project, which, of course, has an incompatible target framework (vMonoTouch).
So, there isn't any real alternative to Touch.Unit, is it?
Is there actually a good way to unit test MonoTouch projects using NUnit
Touch.Unit
and the MonoDevelop test runner?
Not really. MonoTouch projects depends on monotouch.dll which needs to execute under iOS (not OSX). So there's a need for the runner to execute on the simulator or devices.
Now there's a few misconceptions in your question:
later everything should work with Jenkins (CI).
Touch.Unit is already used with continuous builds / integration servers (as long as they are running OSX) using both the iOS simulator and/or devices. Details are available here.
I know the limitations about UI specific code,
Touch.Unit is not about UI testing. In fact it's pretty bad at UI testing (but that's beside the point).
Touch.Unit is a test runner that execute on iOS. That allows you to use MonoTouch / iOS API inside your own tests (it could be UIKit, but it could be StoreKit, GameKit, *Kit, any Foundation class... it's a quite large world).
So, there isn't any real alternative to Touch.Unit, is it?
Yes. If your business logic is well isolated and does not depend on monotouch.dll then you should be able to build it either as:
a non-MonoTouch project (different project, same sources), i.e. linked with the regular framework; or
link to the sources from within your unit test assembly (which links to the regular framework);
That classic nunit test assembly would then be a regular framework project and will be able to run from the default NUnit runner or from within MonoDevelop unit test runner.
Related
I'm coming from a native iOS / Android development background and I'm trying to understand the tooling around Xamarin Unit Testing using the Command Line.
From my point of view there are two types of code that you want to Unit Test:
Plain Old C# Code - with no dependencies to any iOS / Android framework - so it shouldn't need an iOS / Android emulator to run on
Code that depends on iOS / Android frameworks that needs to run on a device / emulator
The official Xamarin documentation mentions NUnitLite / Touch.Unit but it doesn't mention any support around Command Line. I did found an example though, but it's not clear to me if this is a tool that's officially supported by Xamarin. Also it seems that you can run tests only on the emulator/device using that tool.
Another example I've found around refers to xUnit.net - it seems that you can also run tests without an emulator / device, and that you can also run them on an emulator / device - however in that specific blogpost it's not documented how you do that.
So my question is: How should I approach Xamarin Unit Testing and what tools do you recommend using so I can have Command Line support in my CI.
Thank you
The most popular unit testing frameworks used with Xamarin are NUnit and XUnit. They are both similar to JUnit.
Usually a Xamarin cross-platform app uses a Portable Class Library (PCL) project where the platform agnostic (shared) code sits: business logic, model, view models, service etc. This code is unit tested in a seperate pcl or.net45 test project which references the source project and nunit/xunit.
To run the nunit/xunit unit tests you need to run the corresponding test runner and point it tou your test assembly. Both nunit and xundit feature console runners which can be parameterized at will from your command line (see links).
Feel free to chose either nunit or xunit. I like them both.
You might also have platform specific unit tests (which depend on the android/ios/uwp sdks) and that have to be run on a device. These tests can also be created with nunit or xunit and run with nunit device runner or xunit device runner. Basically what will happen here is you add an android/ios app project for testing which references nunit/junit, contains your device specific tests and links to your shared tests and can run them both on the device.
There is also the layer of coded UI tests where NUnit, Xamarin UITest and Specflow might be of use. Im guessing this part is beyond the scope of your question.
But then again you are coming form Android and are used with gradle. Well Xamarin and .net does not have gradle but it has Cake. I use it to automate all my project builds/tests/ci/deployments etc.
Cake (C# Make) is a cross platform build automation system with a C# DSL to do things like compiling code, copy files/folders, running unit tests, compress files and build NuGet packages.
Your Cake script can look something like this:
Task("Run-Unit-Tests")
.IsDependentOn("Build")
.Does(() =>
{
NUnit("./src/**/bin/" + configuration + "/*.Tests.dll");
});
Task("Build")
.Does(() =>
{
DotNetBuild("YourAndroid.csproj");
DotNetBuild("YourCoreTests.csproj");
...
}
);
Cake comes with a bootstrapper file either (ps1 - powershell for windows or sh for mac) which downloads all the tools you need to run your script (cake itself, nuget, nunit/xunit runner etc).
Your command line/CI can run it like this:
./build.sh -Target Run-Unit-Tests
Unit testing in painful on Xamarin.
Don't have huge experience in Unit Testing mobile project, but if you want to test the app I would recommend this integration test approach:
1) For any calculation type functionality (you called it "Plain Old C#") use NUnit (it is supported by mono).
However I cannot come up with an example of such code, as heavy calculations should be done on server side. And there you do separate Unit tests
2) UITests(NUnit again) can be done to prove app is working and UI interaction call needed behavior.
Xamarin also providing TestCloud where app could be tested on many devices, but it is paid for Service. As Alternative you can setup build server like Jenkins to do this job for you.
Anyway - it is my view on how this could be done and hope it answered a bit on your question.
All I can read about unit testing on a Xamarin Android project is about tests run on devices. But what I am looking for are business logic tests I can run pretty close to my IDE (Xamarin Studio/Visual Studio), meaning in my IDE.
Isn't there a way to do it?
If you have business logic that you can test which would not require you to run in the context of an Android device/emulator, you can use a more typical unit testing scenario. Create a C# library project and add your favorite unit testing library to it. Xamarin Studio currently supports NUnit if you want to use the test runner built into the IDE.
There is a blog post here that should help. There is a more generic NUnit tutorial on this blog post.
Open your solution and right click to choose Add > Add new Project. On the now open window click on .NET beneath the Others section. There you see NUnit library project.
This works for method testing. Unfortunately you can not test your Android project that way as far as I know.
I have a Windows 8 Store App with two projects within the solution - one project exclusively for tests. I have added NUnit and NUnit Test Adapter (https://www.nuget.org/packages/NUnitTestAdapter/1.0.0) for this through nuget.
My tests are detected - but I cannot run or debug them. Looking at the test output window, I get the following error:
Could not find test executor with URI 'executor://nunittestexecutor/'. Make sure that the test executor is installed and supports .net runtime version 4.0.30319.34003.
I am not using resharper and I am using Visual Studio Professional 2012 Update 3 on Windows 8.1. References for the test project are as follows:
Based on my comments above, below is a workaround. Unfortunately at this stage, Windows 8.1 store Unit Test project types, using NUnit extension wouldn't work due to the different .NET targets. I tried with different Test Unit Adapters including an NUnitTestAdapterWithFramework.
It seems that the issue you haveing was occurring with standard .NET libraries targeting NUnit test adapter but the above NUnitTestAdapterWithFramework must have fixed those issue. See the Q & A section of the NUnitTestExtension
But unfortunatly it seems that this still of an issue that hasn't been fixed for Win8 Store App type Unit Testing. Pretty sure xUnit.NET also not compatible yet with different .NET target types (i,e WinRT)
So what are the options?
a. For your group, you can change them to use MSTest framework. Outcome - Problem Solved no issues.
b. Workaround "linked project". Outcome - Can't *guarantee** but this should also work.
With option 'b'
In your comment you mentioned.
but I'm still not sure what it does or how to implement a 'linked
project', do you have any more information on this? Also, as this is
for a group university project, I was hoping i wouldn't have to force
too many workarounds
When you think about it, it is not really hard work around. It is simple and I'm sure your group would be able to apply this workaround easily.
Please follow the below steps.
Create a separate class library in your solution (you can target .NET framework 4).
Then add NUnit assemblies and the NUnit test adapter as usual.
Right click on this project and select 'Add' then 'Existing Item'
Select the Win8 Store Unit Test project and locate the Unit Test file you want to add. When you add the file, make sure you select 'Add As a Link' button. Please see below.
Now rebuild the solution, close and re-open the UnitTest explorer and you should be able to run those tests.
*The reason I said can't guaranteed. I haven't really written Unit tests against Win8 App. So if your SUT (System Under Test) require special configuration it might cause issues. But I'm not sure.
Finally creating a link files are not that hard if everything works you can continue to do this until NUnit has the support for Win8 Unit Testing. Or the other option is simply change all your Unit Tests to use MSTest framework if possible.
I am just starting to work with QuantLib project. There are quite a few BOOST unit tests defined, hence the question: is there an easy way to run these tests selectively from Visual Studio (and see results)?
I've only used unit testing in .Net languages so far, and there you can either use the built-in test runner if you use MsTest, or some add-on (I used Resharper and TestDriven.NET) for other frameworks. There you can choose which tests you run, and you can run them with the debugger attached (which is unfortunately useful sometimes). Are there similar add-ons for BOOST tests in unmanaged C++?
Yes you can!
If your boost unit test project is defined correctly, then you can run the entire test project from VS10 debugger by clicking
right click on project --> Debug --> Start new instance
This will run the tests inside VS framework which will give you all the debugger goodies.
For running specific test suites/cases, just edit the project command line arguments under
right click on project --> Properties --> General --> Debugging
and add the parameters according to the links in the other answers
you can run any test case or test suite via command line. running tests by name
it even allows wildcards.
ReSharper C++ supports discovery and selective running/debugging of Boost tests (with Boost 1.59 and later).
I'm using the Boost Unit Test Adapter. It's free and has a very nice output. It can also list the time the single tests consumed and you can start single tests and group of tests.
You can download it here:
https://visualstudiogallery.msdn.microsoft.com/5f4ae1bd-b769-410e-8238-fb30beda987f?SRC=VSIDE
Currently I am experimenting with Mono, and creating a small testproject. When I tried to write some unittests I recognized that there is no default unittest framework like in VS. I decided to go with nUnit (but mbUnit, XUnit,...) would be fine too.
My problem is, that I was not able to find a testrunner that executes my tests and displays the testresults.
Is there a good tool available (currently I am using Monodevelop 2.4)? It is not a requirement tha it is integrated within monodevelop, so an external tool would be good enough.
NUnit is fine. I am using it with mono and works perfectly. For test runner you can use
nunit-console [input]
or the gui one
nunit-gui
These apps can be probably installed from your distribution repository (Ubuntu has it for sure) or manually installed using binaries from NUnit site. As you have written, you used to use built in VS tests. If you are using resharper, nunit will integrate as well, I also find it very clean.
I'd like to add that from the gui runner is winforms application and does not look (and feel ;)) too good on my OS. Nevertheless works fine.