Trying to expose additional information when using xUnit Assert.Throws - unit-testing

I'm just setting up some first unit tests, and I can't quite see how I'm trying to achieve (with my current test structure) can be done, which means I'm not sure whether my approach to the tests is incorrect, or it's just a limitation on xUnit.
I'm testing my MVC Controllers, and want to ensure that they all provide a ArgumentNullException if they are constructed passing null across as the arguments (they get resolved via Castle in the real world).
So, I've a private field on the Test class:
private IEnumerable<Type> ControllerTypes = typeof(MyBaseController).Assembly.GetTypes().Where(t => IsController(t));
Then, my test method:
[Fact]
public void EnsureControllersThrowIfProvidedWithNull() {
foreach (var controller in ControllerTypes) {
var ctrs = GetConstructorsForType(controller);
if (null == ctrs || !ctrs.Any()) { //if the controller has no constructors, that's fine, we just skip over it
continue;
}
var ctr = ctrs.ElementAt(0);
var ctrParamsAsNull = ctr.GetParameters().Select(p => (object)null);
Assert.Throws<ArgumentNullException>(() => {
ctr.Invoke(ctrParamsAsNull.ToArray());
});
}
}
So this is all working fine, I run the test runner, and one of my Controllers doesn't throw an ArgumentNullException when passed null, great, my test fails, but I don't know which controller it was, from the given output.
I do know how I can debug through the test to see which it is that fails, and can manually go through all my controllers to check which it is, but it would be useful to know which controller it was that failed.
Or am I just using a unit test wrong here?
(Side note, there's another test which ensures there's only 1 public constructor for each controller, so I can be sure I'm targeting the correct constructor when this fires, as long as that first test passed).
Thanks
Note:
There's a flaw in the logic for the test, which means it doesn't fully cover what I was expecting it too, as long as it throws an ArgumentNullException for at least 1 of the arguments, then it will pass the test, which isn't right. However as the arguments are interfaces I can't instantiate a new instance of them. So anyone looking to copy the code for the test, I wouldn't do so. Not looking for a solution to that issue here.

Assert.Throws is only helper method that executes delegate inside try catch block. You don't have to use it and you can replace it with your own implementation. Something like:
[Fact]
public void EnsureControllersThrowIfProvidedWithNull() {
foreach (var controller in ControllerTypes) {
var ctrs = GetConstructorsForType(controller);
if (null == ctrs || !ctrs.Any()) { //if the controller has no constructors, that's fine, we just skip over it
continue;
}
var ctr = ctrs.ElementAt(0);
var ctrParamsAsNull = ctr.GetParameters().Select(p => (object)null);
book ok = false;
try
{
ctr.Invoke(ctrParamsAsNull.ToArray());
}
catch(ArgumentNullException)
{
//you get exception you expected so continue
ok = true;
}
if(!ok)
{
// you didn't get exception so throw your own exception with message that contains controller type name
throw new Exception(String.Format("Ctor on type {0} did not throw ArgumentNullException",controller.Name);
}
}
}
This is only as idea to work on. You can refactor that inside your own static assertion method...

Related

Moq Verify not matching on invocation

Before starting, I have used Moq to mock things in unit tests for years. This should be a simple mock verify, but for whatever reason moq is not matching the invocation on the Mock when it occurs. I've manually tested, it is hit. I've debugged the test and compared actual vs. expected values (they match), I've scoured SO and its multitudes of people doing blatantly wrong things and I cannot figure out why this isn't working. Help appreciated.
The unit test is a very simple test of checking if BulkInsert is called at the end of a void returning function. Sample code:
Code:
public interface IDependencyService
{
void BulkInsert(IList<T> items);
}
public class MyServiceClass
{
private readonly IDependencyService _service;
/* ctor and all that jazz */
public void Run()
{
/* do things to the data */
_service.BulkInsert(items); // where items is an IList<T>
}
}
Test:
public class ServiceTests
{
[Fact]
public void ServiceRun_Calls_DependencyBulkInsert()
{
var dependencyMock = new Mock<IDependencyService>();
List<T> expected = /* somehow build expected values */
dependencyMock
.Setup(mock => mock.BulkInsert(It.IsAny<IList<T>>()));
var sut = new MyServiceClass(dependencyMock.Object);
sut.Run();
dependencyMock.Verify(mock => mock.BulkInsert(expected), Times.Once());
}
}
Error message:
Expected invocation on the mock once, but was 0 times: mock => mock.BulkInsert([ThresholdCheck])
Performed invocations:
Mock<IThresholdCheckHandler:1> (mock):
IThresholdCheckHandler.GetQueuedChecks()
IThresholdCheckHandler.BulkInsert([ThresholdCheck])
If I change expected to It.IsAny<T>() in the Verify call, test passes. This leads me to believe that maybe somehow the objects passed from expected are somehow different from the objects generated when running the program. However as stated I've went through with the debugger and manually compared every value in the actual list to the expected list of values in the test and they are exactly the same.
This then leads me to believe that I'm just a stupid bipedal monkey clicking at a keyboard and that the problem is right in front of me and I'm just not seeing it. Any help or set of eyes is appreciated.
As per NKosi's comment, Verify operates via. reference when using reference types. I was confused as I'd used Verify in the past for simple things like ints and strings but was unaware of that quirk. One would assume with Verify that it would check the equivalency of actual vs. expected, but no.
Regardless then, as per Quercus's comment I adjusted my test to this in order to continue with my day:
public class ServiceTests
{
[Fact]
public void ServiceRun_Calls_DependencyBulkInsert()
{
var dependencyMock = new Mock<IDependencyService>();
List<T> actual = new List<T>();
List<T> expected = /* somehow build expected values */
dependencyMock
.Setup(mock => mock.BulkInsert(It.IsAny<IList<T>>()))
.Callback<List<T>>(l => actual = l);
var sut = new MyServiceClass(dependencyMock.Object);
sut.Run();
actual.Should().BeEquivalentTo(expected);
}
}
and this solution works for me. thanks to you both for helping me realize my mistake.

Spock - How to work with repeated interactions

For few test cases I'm trying to follow a DRY principle, where only the interactions are different with same test case conditions. I'm not able to find a way to implement multiple methods in the interaction { } block.
As mentioned in http://spockframework.org/spock/docs/1.3/interaction_based_testing.html#_explicit_interaction_blocks, I'm using interaction { } in the then: block like below:
Java Code:
// legacy code (still running on EJB 1.0 framework, and no dependency injection involved)
// can't alter java code base
public voidGetData() {
DataService ds = new DataService();
ds = ds.findByOffset(5);
Long len = ds.getOffset() // happy path scenario; missing a null check
// other code
}
// other varieties of same code:
public voidGetData2() {
ItemEJB tmpItem = new ItemEJB();
ItemEJB item = tmpItem.findByOffset(5);
if(null != item) {
Long len = item.getOffset();
// other code
}
}
public voidGetData3() {
ItemEJB item = new ItemEJB().findByOffset(5);
if(null != item) {
Long len = item.getOffset();
// other code
}
}
Spock Test:
def "test scene1"() {
given: "a task"
// other code ommitted
DataService mockObj = Mock(DataService)
when: "take action"
// code omitted
then: "action response"
interaction {
verifyNoDataScenario() // How to add verifyErrorScenario() interaction to the list?
}
}
private verifyDataScenario() {
1 * mockObj.findByOffset(5) >> mockObj // the findByOffset() returns an object, so mapped to same mock instance
1 * mockObj.getOffset() >> 200
}
private verifyErrorScenario() {
1 * mockObj.findByOffset(5) >> null // the findByOffset() returns null
0 * mockObj.getOffset() >> 200 // this won't be executed, and should ie expected to throw NPE
}
The interaction closure doesn't accept more than one method call. I'm not sure if it's design limitation. I believe more can be done in the closure than just mentioning the method name. I also tried interpolating the mockObj as a variable and use data pipe / data table, but since it's referring the same mock instance, it's not working. I'll post that as a separate question.
I ended up repeating the test case twice just to invoke different interaction methods. Down the line I see more scenarios, and wanted to avoid copy & paste approach. Appreciate any pointers to achieve this.
Update:
Modified shared java code as the earlier DataService name was confusing.
As there's no DI involved, and I didn't find a way to mock method variables, so I mock them using PowerMockito, e.g. PowerMockito.whenNew(DataService.class).withNoArguments().thenReturn(mockObj)
Your application code looks very strange. Is the programming style in your legacy application really that bad? First a DataService object is created with a no-arguments constructor, just to be overwritten in the next step by calling a method on that instance which again returns a DataService object. What kind of programmer creates code like that? Or did you just make up some pseudo code which does not have much in common with your real application? Please explain.
As for your test code, it also does not make sense because you instantiate DataService mockObj as a local variable in your feature method (test method), which means that in your helper method mockObj cannot be accessed. So either you need to pass the object as a parameter to the helper methods or you need to make it a field in your test class.
Last, but not least, your local mock object is never injected into the class under test because, as I said in the first paragraph, the DataService object in getData() is also a local variable. Unless your application code is compeletely fake, there is no way to inject the mock because getData() does not have any method parameter and the DataService object is not a field which could be set via setter method or constructor. Thus, you can create as many mocks as you want, the application will never have any knowledge of them. So your stubbing findByOffset(long offset) (why don't you show the code of that method?) has no effect whatsoever.
Bottom line: Please provide an example reflecting the structure of your real code, both application and test code. The snippets you provide do not make any sense, unfortunately. I am trying to help, but like this I cannot.
Update:
In my comments I mentioned refactoring your legacy code for testability by adding a constructor, setter method or an overloaded getData method with an additional parameter. Here is an example of what I mean:
Dummy helper class:
package de.scrum_master.stackoverflow.q58470315;
public class DataService {
private long offset;
public DataService(long offset) {
this.offset = offset;
}
public DataService() {}
public DataService findByOffset(long offset) {
return new DataService(offset);
}
public long getOffset() {
return offset;
}
#Override
public String toString() {
return "DataService{" +
"offset=" + offset +
'}';
}
}
Subject under test:
Let me add a private DataService member with a setter in order to make the object injectable. I am also adding a check if the ds member has been injected or not. If not, the code will behave like before in production and create a new object by itself.
package de.scrum_master.stackoverflow.q58470315;
public class ToBeTestedWithInteractions {
private DataService ds;
public void setDataService(DataService ds) {
this.ds = ds;
}
// legacy code; can't alter
public void getData() {
if (ds == null)
ds = new DataService();
ds = ds.findByOffset(5);
Long len = ds.getOffset();
}
}
Spock test:
Now let us test both the normal and the error scenario. Actually I think you should break it down into two smaller feature methods, but as you seem to wish to test everything (IMO too much) in one method, you can also do that via two distinct pairs of when-then blocks. You do not need to explicitly declare any interaction blocks in order to do so.
package de.scrum_master.stackoverflow.q58470315
import spock.lang.Specification
class RepeatedInteractionsTest extends Specification {
def "test scene1"() {
given: "subject under test with injected mock"
ToBeTestedWithInteractions subjectUnderTest = new ToBeTestedWithInteractions()
DataService dataService = Mock()
subjectUnderTest.dataService = dataService
when: "getting data"
subjectUnderTest.getData()
then: "no error, normal return values"
noExceptionThrown()
1 * dataService.findByOffset(5) >> dataService
1 * dataService.getOffset() >> 200
when: "getting data"
subjectUnderTest.getData()
then: "NPE, only first method called"
thrown NullPointerException
1 * dataService.findByOffset(5) >> null
0 * dataService.getOffset()
}
}
Please also note that testing for exceptions thrown or not thrown adds value to the test, the interaction testing just checks internal legacy code behaviour, which has little to no value.

Asserting a property has been set in a mocked class

I'm using the MockingContainer<T> to automatically set up my dependencies. How do I assert that a property on one of those dependencies gets set?
[SetUp]
public void SetUp()
{
//arrange
_baseUrl = "http://baseUrl";
_container = new MockingContainer<ApiInteractionService>();
_container.Arrange<IConfigService>(m => m.BaseUrl).Returns(_baseUrl);
_uut = _container.Instance;
}
The following fails with 0 calls, which makes sense since I believe it's looking at the Getter, not the Setter. So how do I assert that the Setter was called by the unit under test?
[Test]
public void BaseUrlSet()
{
//act
var _ = _uut.MakeRequest((InitialRequest) Arg.AnyObject);
//assert
_container.Assert<IRestService>(m => m.BaseUrl, Occurs.Once());
}
Per the documentation (located at JustMock Docs for anyone who isn't familiar but wishes to try assisting) it appears I should be using Mock.ArrangeSet(lambda), however I cannot seem to figure out how to get that syntax to work in relation to MockingContainer<T>.
If worse comes to worse, I can just NOT use MockingContainer<T>, but I'd prefer to not have to refactor my test suite just to accommodate one specific unit test.
Not that it's really relevant to the question, but in the off chance anyone needs it, here is a stub of ApiInteractionService
public ApiInteractionService(IRestService restService, IConfigService configService)
{
_restService = restService;
_restService.BaseUrl = configService.BaseUrl;
}
public string MakeRequest(InitialRequest initialRequest)
{
return _restService.Post(initialRequest);
}
Why not simply assert that BaseUrl has the correct value at the end of the test?
var baseUrl = _container.Get<IRestService>().BaseUrl;
Assert.AreEqual(baseUrl, _baseUrl);
As suggested in the comments, _container.Assert<IRestService>(m => m.BaseUrl == _baseUrl) will not work. MockingContainer<T>.Assert asserts an expectation, it's not just asserting truth like regular asserts. The correct syntax would have been:
_container.AssertSet<IRestService>(restService => restService.BaseUrl = _baseUrl, Occurs.Once());
but, oddly, there is no AssertSet method on the container.

value generation in pex parmeterized unit test

I'm building unit testing using Pex. my problem is not all code branches are being tested, Pex keep generating parameter values that fails the same condition which make all the code after that condition not to run.
my method goes something like this:
public void SetUp(DbSyncScopeDescription SyncScopeDesc, BasicInfo info, string dbContext = "MyDBContext")
{
// <pex>
Contracts validation
// </pex>
string localDbConnStr = string.Empty;
//this condition never get a parameter that results in true
if (IsContextExist(dbContext))
{
localDbConnStr = ConfigurationManager.ConnectionStrings[dbContext + "Context"].ConnectionString;
}
else
{
throw new MissingFieldException("dbcontext does not exist");
}
// This part is never being reached
ProvisionLocalScope(SyncScopeDesc, info.FarmId, localDbConnStr);
info.Tables = GetSyncTablesAsSyncTableInfo(SyncScopeDesc);
AdminOrm.Create(info.ToORM(), String.Format("name={0}AdminEntities", dbContext));
}
I wonder if its possible to tell Pex to pass that test so all the code will be reached.
if this is not possible is it possible to make Pex take the default values of the function parameters for one of the tests (i think this will be a good feature if its not present).
Thank you

Moq tests using ExpectSet() with It.Is<T>() aren't behaving as... expected

I've isolated the behaviour into the following test case. I'd be grateful to anyone who can tell me how to expect/verify a property set for a List<T> property - it appears there's something going on inside It.Is<T>(predicate) that isn't making a whole lot of sense to me right now. Sample code will run as a console app from VS2008 - you'll need to add a reference to Moq 2.6 (I'm on 2.6.1014.1) - please try uncommenting the different ExpectSet statements to see what's happening...
using System;
using Moq;
using System.Collections.Generic;
namespace MoqDemo {
public interface IView {
List<string> Names { get; set; }
}
public class Controller {
private IView view;
public Controller(IView view) {
this.view = view;
}
public void PopulateView() {
List<string> names = new List<string>() { "Hugh", "Pugh", "Barney McGrew" };
view.Names = names;
}
public class MyApp {
public static void Main() {
Mock<IView> mockView = new Mock<IView>();
// This works - and the expectation is verifiable.
mockView.ExpectSet(mv => mv.Names);
// None of the following can be verified.
// mockView.ExpectSet(mv => mv.Names, It.Is<Object>(o => o != null));
// mockView.ExpectSet(mv => mv.Names, It.Is<List<string>>(names => names.Count == 3));
// mockView.ExpectSet(mv => mv.Names, It.IsAny<IList<String>>());
Controller controller = new Controller(mockView.Object);
controller.PopulateView();
try {
mockView.VerifyAll();
Console.WriteLine("Verified OK!");
} catch (MockException ex) {
Console.WriteLine("Verification failed!");
Console.WriteLine(ex.Message);
}
Console.ReadKey(false);
}
}
}
}
I'm not using the very latest version of Moq, so I don't have an overload of ExpectSet that takes two parameters, but I've had some success with this pattern:
mockView.ExpectSet(mv => mv.Names).Callback(n => Assert.That(n != null));
The Assert (from NUnit) call in the callback will throw an exception if the value assigned to .Names doesn't match the predicate. It does make it hard to trace when a test fails, though. I agree that the ability to pass an It.Is or It.IsAny as the second parameter would be handy.
The second parameter of ExpectSet() is the value you're expecting. You can't use It.Is<T> in this case as there's no overload that takes a predicate - though it would be nice ;) Here's a (simplified) excerpt from your sample, illustrating the use of a value:
var mockView = new Mock<IView>();
var list = new List<string> { "Hugh", "Pugh", "Barney McGrew" };
mockView.ExpectSet(mv => mv.Names, list);
mockView.Object.Names = list;
Hope that helps.
Edit: fixed typo.
BTW, It.Is is not supported on ExpectSet. Your code compiles just because they are regular method invocations when used as values (as opposed to expressions), whereas when used in an Expect expression they are pre-processed by Moq and given specific meaning (rather than the null/default value that all It.Is members actually return).
You could use the stub behavior on the given property (mockView.Stub(mv => mv.Names)) and later assert directly for its value after execution.
Moq doesn't provide an overload receiving It.IsAny as it's effectively the same as calling ExpectSet without passing an expected value ;)