All I can read about unit testing on a Xamarin Android project is about tests run on devices. But what I am looking for are business logic tests I can run pretty close to my IDE (Xamarin Studio/Visual Studio), meaning in my IDE.
Isn't there a way to do it?
If you have business logic that you can test which would not require you to run in the context of an Android device/emulator, you can use a more typical unit testing scenario. Create a C# library project and add your favorite unit testing library to it. Xamarin Studio currently supports NUnit if you want to use the test runner built into the IDE.
There is a blog post here that should help. There is a more generic NUnit tutorial on this blog post.
Open your solution and right click to choose Add > Add new Project. On the now open window click on .NET beneath the Others section. There you see NUnit library project.
This works for method testing. Unfortunately you can not test your Android project that way as far as I know.
Related
I'm coming from a native iOS / Android development background and I'm trying to understand the tooling around Xamarin Unit Testing using the Command Line.
From my point of view there are two types of code that you want to Unit Test:
Plain Old C# Code - with no dependencies to any iOS / Android framework - so it shouldn't need an iOS / Android emulator to run on
Code that depends on iOS / Android frameworks that needs to run on a device / emulator
The official Xamarin documentation mentions NUnitLite / Touch.Unit but it doesn't mention any support around Command Line. I did found an example though, but it's not clear to me if this is a tool that's officially supported by Xamarin. Also it seems that you can run tests only on the emulator/device using that tool.
Another example I've found around refers to xUnit.net - it seems that you can also run tests without an emulator / device, and that you can also run them on an emulator / device - however in that specific blogpost it's not documented how you do that.
So my question is: How should I approach Xamarin Unit Testing and what tools do you recommend using so I can have Command Line support in my CI.
Thank you
The most popular unit testing frameworks used with Xamarin are NUnit and XUnit. They are both similar to JUnit.
Usually a Xamarin cross-platform app uses a Portable Class Library (PCL) project where the platform agnostic (shared) code sits: business logic, model, view models, service etc. This code is unit tested in a seperate pcl or.net45 test project which references the source project and nunit/xunit.
To run the nunit/xunit unit tests you need to run the corresponding test runner and point it tou your test assembly. Both nunit and xundit feature console runners which can be parameterized at will from your command line (see links).
Feel free to chose either nunit or xunit. I like them both.
You might also have platform specific unit tests (which depend on the android/ios/uwp sdks) and that have to be run on a device. These tests can also be created with nunit or xunit and run with nunit device runner or xunit device runner. Basically what will happen here is you add an android/ios app project for testing which references nunit/junit, contains your device specific tests and links to your shared tests and can run them both on the device.
There is also the layer of coded UI tests where NUnit, Xamarin UITest and Specflow might be of use. Im guessing this part is beyond the scope of your question.
But then again you are coming form Android and are used with gradle. Well Xamarin and .net does not have gradle but it has Cake. I use it to automate all my project builds/tests/ci/deployments etc.
Cake (C# Make) is a cross platform build automation system with a C# DSL to do things like compiling code, copy files/folders, running unit tests, compress files and build NuGet packages.
Your Cake script can look something like this:
Task("Run-Unit-Tests")
.IsDependentOn("Build")
.Does(() =>
{
NUnit("./src/**/bin/" + configuration + "/*.Tests.dll");
});
Task("Build")
.Does(() =>
{
DotNetBuild("YourAndroid.csproj");
DotNetBuild("YourCoreTests.csproj");
...
}
);
Cake comes with a bootstrapper file either (ps1 - powershell for windows or sh for mac) which downloads all the tools you need to run your script (cake itself, nuget, nunit/xunit runner etc).
Your command line/CI can run it like this:
./build.sh -Target Run-Unit-Tests
Unit testing in painful on Xamarin.
Don't have huge experience in Unit Testing mobile project, but if you want to test the app I would recommend this integration test approach:
1) For any calculation type functionality (you called it "Plain Old C#") use NUnit (it is supported by mono).
However I cannot come up with an example of such code, as heavy calculations should be done on server side. And there you do separate Unit tests
2) UITests(NUnit again) can be done to prove app is working and UI interaction call needed behavior.
Xamarin also providing TestCloud where app could be tested on many devices, but it is paid for Service. As Alternative you can setup build server like Jenkins to do this job for you.
Anyway - it is my view on how this could be done and hope it answered a bit on your question.
I have a Windows 8 Store App with two projects within the solution - one project exclusively for tests. I have added NUnit and NUnit Test Adapter (https://www.nuget.org/packages/NUnitTestAdapter/1.0.0) for this through nuget.
My tests are detected - but I cannot run or debug them. Looking at the test output window, I get the following error:
Could not find test executor with URI 'executor://nunittestexecutor/'. Make sure that the test executor is installed and supports .net runtime version 4.0.30319.34003.
I am not using resharper and I am using Visual Studio Professional 2012 Update 3 on Windows 8.1. References for the test project are as follows:
Based on my comments above, below is a workaround. Unfortunately at this stage, Windows 8.1 store Unit Test project types, using NUnit extension wouldn't work due to the different .NET targets. I tried with different Test Unit Adapters including an NUnitTestAdapterWithFramework.
It seems that the issue you haveing was occurring with standard .NET libraries targeting NUnit test adapter but the above NUnitTestAdapterWithFramework must have fixed those issue. See the Q & A section of the NUnitTestExtension
But unfortunatly it seems that this still of an issue that hasn't been fixed for Win8 Store App type Unit Testing. Pretty sure xUnit.NET also not compatible yet with different .NET target types (i,e WinRT)
So what are the options?
a. For your group, you can change them to use MSTest framework. Outcome - Problem Solved no issues.
b. Workaround "linked project". Outcome - Can't *guarantee** but this should also work.
With option 'b'
In your comment you mentioned.
but I'm still not sure what it does or how to implement a 'linked
project', do you have any more information on this? Also, as this is
for a group university project, I was hoping i wouldn't have to force
too many workarounds
When you think about it, it is not really hard work around. It is simple and I'm sure your group would be able to apply this workaround easily.
Please follow the below steps.
Create a separate class library in your solution (you can target .NET framework 4).
Then add NUnit assemblies and the NUnit test adapter as usual.
Right click on this project and select 'Add' then 'Existing Item'
Select the Win8 Store Unit Test project and locate the Unit Test file you want to add. When you add the file, make sure you select 'Add As a Link' button. Please see below.
Now rebuild the solution, close and re-open the UnitTest explorer and you should be able to run those tests.
*The reason I said can't guaranteed. I haven't really written Unit tests against Win8 App. So if your SUT (System Under Test) require special configuration it might cause issues. But I'm not sure.
Finally creating a link files are not that hard if everything works you can continue to do this until NUnit has the support for Win8 Unit Testing. Or the other option is simply change all your Unit Tests to use MSTest framework if possible.
Is there actually a good way to unit test MonoTouch projects using NUnit and the MonoDevelop test runner?
I know there is the official MonoTouch unit tests project type, but running tests within the simulator isn't the way I want to go. For now I want to run tests with the MonoDevelop test runner, later everything should work with Jenkins (CI).
I know the limitations about UI specific code, so everything I want to test has nothing to do with MonoTouch itself, it's all about business logic placed within separete projects.
By adding tests to MonoTouch Library type projects, I am getting System.IO.FileNotFoundException's as described here: http://ben.phegan.name/index.php/2011/02/28/monotouch-and-unit-testing/
By using a separate NUnit test project, I can't reference my system under test, because its project type is of type MonoTouch library project, which, of course, has an incompatible target framework (vMonoTouch).
So, there isn't any real alternative to Touch.Unit, is it?
Is there actually a good way to unit test MonoTouch projects using NUnit
Touch.Unit
and the MonoDevelop test runner?
Not really. MonoTouch projects depends on monotouch.dll which needs to execute under iOS (not OSX). So there's a need for the runner to execute on the simulator or devices.
Now there's a few misconceptions in your question:
later everything should work with Jenkins (CI).
Touch.Unit is already used with continuous builds / integration servers (as long as they are running OSX) using both the iOS simulator and/or devices. Details are available here.
I know the limitations about UI specific code,
Touch.Unit is not about UI testing. In fact it's pretty bad at UI testing (but that's beside the point).
Touch.Unit is a test runner that execute on iOS. That allows you to use MonoTouch / iOS API inside your own tests (it could be UIKit, but it could be StoreKit, GameKit, *Kit, any Foundation class... it's a quite large world).
So, there isn't any real alternative to Touch.Unit, is it?
Yes. If your business logic is well isolated and does not depend on monotouch.dll then you should be able to build it either as:
a non-MonoTouch project (different project, same sources), i.e. linked with the regular framework; or
link to the sources from within your unit test assembly (which links to the regular framework);
That classic nunit test assembly would then be a regular framework project and will be able to run from the default NUnit runner or from within MonoDevelop unit test runner.
I'm working from the XunitContrib codeplex page and toward the bottom it lists these steps
For Windows Phone 7
Follow along with this blog post
Create a Windows Phone application
Add references to:
Microsoft.Silverlight.Testing.dll (Silverlight 3 version - included in release)
xunit-silverlight-wp7
xunit.extensions-silverlight-wp7
xunitcontrib.runner.silverlight.toolkit-wp7
Visual Studio may display warnings about including Silverlight 3 assemblies. Ignore it, these are the right files
Add [Fact] based tests and run the app (note- the blog post mentioned isn't using fact based tests so I'm even more confused ...)
But after I add the above mentioned dll's and start with something like the below ... resharper and I can't seem to hookup the testing harness enough to compile. Has anyone actually wired up a unit test with xunit for WP7?
public class MyFirstWp7Test
{
[Fact]
public void Can_Run_Test_For_WP7()
{
var x = "hello world";
Assert.Equal("hello world", x);
}
}
You can't use resharper to run Silverlight or Windows Phone tests, unfortunately. The xunitcontrib resharper runner is for the desktop clr only (newer builds ignore files in a silverlight/wp project so you don't even see the unit test marker in the editor).
You need to actually deploy to the device or emulator, run your application and drive the tests through the UI of the Silverlight Unit Testing Framework. You can filter by tag, test class or test method name, but it all has to be in the phone's environment.
I finally found a way to test-drive my app on the WP7 platform using NUnit and ReSharper.
I did a short screencast on my blog for a full "how-to" of sorts
http://toranbillups.com/blog/archive/2011/07/24/Test-Driving-My-Windows-Phone-7-App
I'm developing in vs2008, c#, .net 3.5.
I downloaded NUnit 2.5 win / msi version. I have created a [TestFixture] Class containing several [Test] methods.
How do I run the tests?
When I run a NUnit demo solution from Ed Ames, his test .cs files have an icon in the grey column to the left of the code (same place where breakpoints, bookmarks, etc show up.) Clicking on the icon gives me an option to run the tests.
That icon is not showing up in my projects. Is there a property I need to set?
Also, the documentation refers to a NUnit GUI that can be used to run the tests. A GUI doesnt seem to have downloaded in the msi version of NUnit. Is there a separate download for the Gui?
Thanks for all your help guys.
I am using resharper, forgot to mention it.
Actually, tests were not running (no tests found in file) because my [TestFixture] class was not Public. Changed it to Public and all my tests showed up.
Also, thanks for your help finding the Gui. I was looking for an exe with GUI in the filename. But I will use Resharper to run the tests. Now that I have found them!
The demo you saw most likely had a VS plug in (Resharper, TestDriven.NET, etc.), which doesn't come with NUnit. However, NUnit installs with a GUI. Go to the folder that you installed NUnit and you will find it there.
The program is called nunit.exe and it's in the bin folder.
Once you start it, you'll need to go to File -> Open Project and find the DLL you built in Visual Studio
You can use the nUnit runner (GUI - nunit.exe in the bin folder, Console - nunit-console-x86.exe) which comes with nUnit I believe.
Alternatively, if you have resharper (http://www.jetbrains.com/resharper/) installed, it has a test runner.
There is also TestDriven.Net (http://www.testdriven.net/)
I haven't seen this demo, but it sounds very much like he is using Resharper.
If you ran the msi installer, it will have installed the NUnit GUI runner. If you browse your start menu its likely under Programs -> Nunit. From the GUI runner you can load your test assembly by navigating to the bin directory of your project.
You have to open the dll with 1) Nunit command line or 2) Nunit GUI