Defining the order in which tests are displayed and run in Visual Studio - unit-testing

Using Visual Studio 2019 and trying out the MSTest, NUnit and xUnit testing frameworks with C#.
Say I have two methods:
// MSTest
[TestClass]
public class MyTests_MSTest
{
[TestMethod]
public void WantedToRunThisFirst()
{
// Arrange, Act, Assert...
}
[TestMethod]
public void ButThisRunsFirst()
{
// Arrange, Act, Assert...
}
}
Have tried the equivalent tests and syntax in all test frameworks (i.e. MSTest, NUnit and xUnit). Tests are always displayed in the Visual Studio Test Explorer section in alphabetical order (i.e. with ButThisRunsFirst at the top). And when the tests are run, the alphabetic order is used again and ButThisRunsFirst runs before WantedToRunThisFirst.
Tests should be independent so the run order shouldn't really matter, but having the tests listed and run to match the order in the code would be nice.
Is it possible to configure the display and/or run order of unit test methods?

Related

Microsoft Native Unit Test for C++ - where is the Logger output going?

Trying out C++ unit testing in Visual Studio 2019. Have added a C++ Native Unit Test Project unit test project to my solution. Then added the following Logger messages:
TEST_CLASS_INITIALIZE(ClassInitialize)
{
Logger::WriteMessage("Setup on test class. Where will this appear?");
}
// ...
TEST_METHOD(TestMethod1)
{
Logger::WriteMessage("Test method. Where is this message going??");
}
The tests are picked up by the Visual Studio Test Explorer and run successfully. Breakpoints are also hit if the tests are debugged. According to the Microsoft documentation on Logger, output should appear in the Visual Studio Output Window, but the messages are not being displayed.
Where is Logger sending its' output?
Although the MS documentation states "The logger class contains static methods to write to the Output Window" this no longer seems to be the case.
The output from the Logger Class can now be seen in the Test Detail Summary, after the unit tests have been run.
Output from the Logger::WriteMessage that is contained in TEST_METHOD(TestMethod1) will appear in the Test Detail Summary for that given unit test.
Output from the Logger::WriteMessage that is contained in TEST_CLASS_INITIALIZE(ClassInitialize) will appear in the first test method that is executed during that test run.

Build and Execute Xamarin.iOS Unit Test from Command Line

I have been trying to run the iOS unit test from command line. I am using Xamarin Studio on Mac for now, I Created a iOS Unit test project as follows
Add New Project --> iOS --> Tests --> Unit Test App
And added a simple Unit test class and its code snippets as shown below:
using System;
using NUnit.Framework;
namespace iOSUnitTest
{
[TestFixture]
public class iOSTestSample
{
public iOSTestSample()
{
}
[Test]
public void MySampleTest() {
Assert.True(true);
}
[Test]
public void MyFailerTest()
{
Assert.False(true);
}
}
}
From Xamarin Studio, I am able to run the application which deploys a app into simulator and execute the test cases. I am trying to automate it through a script.
Till now, I am able to get the Unit Test project build and install the app into the running simulator.
I am not sure how to automate the unit test execution once the app is installed.
This blog post (and the ones linked( might be a bit out-dated but it shows you how to automate unit testing (both simulator and device builds) with the tools that ships with Xamarin.iOS.

Cannot find element for dotnet test id: error in Resharper 2016.3

I just downloaded Resharper 2016.3 EAP 4 to check out the Unit test functionality with .NET Core. But when I run all unit tests, I get this error:
Cannot find element for dotnet test id:
MvcMovieTests.SimpleTests.TestMethodPassing
Cannot find element for dotnet test id:
MvcMovieTests.SimpleTests.TestMethodFailing
Here are my simple unit tests:
[TestClass]
public class SimpleTests
{
[TestMethod]
public void TestMethodPassing()
{
Microsoft.VisualStudio.TestTools.UnitTesting.Assert.IsTrue(true);
}
[TestMethod]
public void TestMethodFailing()
{
Microsoft.VisualStudio.TestTools.UnitTesting.Assert.IsTrue(false);
}
}
When I run all unit tests with the MSTEST Test Explorer, they run properly and I see the results. But with Resharper 2016.3 I get the two errors above showing up in the Unit Test Sessions window in Visual Studio 2015 Community.
Clearing up the ReSharper caches fixed it up for me. Open the Environment | General page of ReSharper options. Click Clear caches. More information here.

NUnit TestCaseAttribute causing AmbiguousMatchException with NUnit VS Adapter

I wrote a bunch of unit tests that utilized the TestCaseAttribute. These tests run great on my local machine when I use the ReSharper unit test runner. Unfortunately, when I run the tests through Visual Studio with the NUnit VS Adapter 2.0.0.0 I get the following output:
------ Run test started ------
NUnit VS Adapter 2.0.0.0 executing tests is started
Loading tests from D:\Projects\Ever\WebApp\Ever.UnitTests\bin\Debug\Ever.UnitTests.dll
Exception System.Reflection.AmbiguousMatchException,
Exception thrown executing tests in
D:\Projects\Ever\WebApp\Ever.UnitTests\bin\Debug\Ever.UnitTests.dll
NUnit VS Adapter 2.0.0.0 executing tests is finished
========== Run test finished: 0 run (0:00:00.8290488) ==========
We use the Visual Studio Online hosted build server for our build, and that relies on the test adapter to run our NUnit unit tests. This means I need to figure out a way to make this work with the attribute (much preferred) or I have to work around this limitation.
Do I have to abandon the use of the TestCaseAttribute because MSTest doesn't support parameterized tests1,2?
After further debuging and testing I've concluded that the TestCaseAttribute isn't the cause of the issue. I'm answering my own question instead of deleting1 in case anyone else falls into the same trap that I did.
The TestCaseAttribute works properly as you can see with the following tests. These tests run perfectly well via the VS Test Adapter and the ReSharper test runner.
[TestFixture]
public class SimpleReproAttempt
{
[Test]
[TestCase(true, false)]
[TestCase(false, true)]
public void DoesNotReproduceIssue(bool a, bool b)
{
Assert.IsTrue(a || b);
}
[Test]
[TestCase("", false, true)]
[TestCase(null, true, false)]
public void DoesNotReproduceIssue(string a, bool b, bool c)
{
Assert.IsTrue(b || c);
Assert.IsNullOrEmpty(a);
}
}
The issue seems to be present only in tests that have an overloaded method with at least one of the overloads using async/await.
1: Editing my question based on this information would turn this into a chameleon question, and a chameleon via self answer is discouraged so I dismissed that option as well.

How to run tests in Nemerle project

How do I run tests which will test my nemerle code. So for example, I've a Calculator class and a CalculatorTests class in a nemerle project. I have already added a reference to nunit using package manager ("install-package nunit"). Now NUnit is available in nemerle project.
After writing following code
[TestFixture]
class CalculatorTests
{
[Test]
MyTest() : void
{
def result = Calculator().Add( 1 );
Assert.AreEqual( 2, result );
}
}
I tried to use TestDriven.net visual studio add-in to run the test but couldn't able to. Can someone tell me how to run tests in nemerle or do i have to write code to run all tests when executing a console app?
Maybe it caused by NUnit runs under .Net 2.0 runtime. Try to set a runtime version in the NUnit command line.