I'm converting a webtest that loops 100 times (fires the request and validates the response 100 times) to NUnit. It seems that the [Repeat] attribute only works for [Test], not [TestCase] or any other decoration/attribute. What is the cleanest way to do this? I want to fire that one test case with those parameters 100 times, and I'd like to do it without adding a loopCount parameter to my test case and nesting my act and assert sections in a for loop.
The below code only runs once.
[TestCase("arg1", "arg2"), Repeat(100)]
public void testing(string arg1, string arg2)
{
//Arrange
//Act
var response = RequestSender.SendGetRequest();
//Assert
AssertStuff(arg1, arg2);
}
From the online NUnit documentation for Repeat, it would appear that this behaviour is deliberate.
Notes:
It is not currently possible to use RepeatAttribute on a TestFixture or any other type of test suite. Only single tests may be repeated.
Since a parameterized test method represents a suite, RepeatAttribute is ignored when it appears on such a method.
With the assumption that TestCase is obviously regarded as a parameterized test. The loop looks inevitable, without some additional customization?
What you could do instead is use use a test case source and feed it an enumerable of 100 repeated items:
private static readonly IEnumerable<Tuple<string, string>> _oneHundredCases =
Enumerable.Range(0, 100)
.Select(_ => Tuple.Create("arg1", "arg2"));
[TestCaseSource(nameof(_oneHundredCases))]
public void testing(Tuple<string, string> theArgs)
{
...
AssertStuff(theArgs.Item1, theArgs.Item2);
}
But for the same price, you may as well vary the arguments a bit?, e.g.
_oneHundredCases =
Enumerable.Range(0, 100)
.Select(n => Tuple.Create(n.ToString(), $"arg{n}"));
Related
I'm trying to cover the following:
I'm using the following test code:
public function test_it_deletes_a_patient()
{
// ...
$cacheKey = vsprintf('%s.%s', [$this->doctorUser->id, 'backoffice.stats.patientsTotalCount']);
Cache::shouldReceive('has')->with($cacheKey)->once()->andReturn(false);
Cache::shouldReceive('increment')->with($cacheKey, -1)->once()->andReturn(true);
$response = $this->json('DELETE', route('patients.destroy', $this->patient), ['confirmation' => 'ELIMINAR']);
// ...
}
That triggers the following controller code:
public function destroy(Patient $patient, Request $request)
{
$this->authorize('delete', $patient);
$confirmation = $request->get('confirmation');
if ($confirmation != 'ELIMINAR') {
return response()->json(['success' => false]);
}
logger()->info("Deleting Patient Profile PATIENT_ID:[{$patient->id}]");
$patient->delete();
$this->updatePatientsCount(-1);
return response()->json(['success' => true]);
}
protected function updatePatientsCount($amount = 1)
{
$key = vsprintf('%s.%s', [auth()->user()->id, 'backoffice.stats.patientsTotalCount']);
if (Cache::has($key)) { // I want to mock for testing this
Cache::increment($key, $amount); // I want to mock for testing this
}
}
After test run I get:
alariva#trinsic:~/fimedi$ t --filter=test_it_deletes_a_patient
PHPUnit 7.3.1 by Sebastian Bergmann and contributors.
F 1 / 1 (100%)
Time: 6.53 seconds, Memory: 26.00MB
There was 1 failure:
1) Tests\Browser\Backoffice\PatientsTest::test_it_deletes_a_patient
Unable to find JSON fragment
["success":true]
within
[{"exception":"Mockery\\Exception\\NoMatchingExpectationException","file":"\/home\/alariva\/fimedi\/vendor\/mockery\/mockery\/library\/Mockery\/ExpectationDirector.php","line":92,"message":"No matching handler found for Mockery_0_Illuminate_Cache_CacheManager::has('2056e535e689ab723b3f44831b488f05f7fb8b90'). Either the method was unexpected or its arguments matched no expected argument list for this method\n\n","trace":[{"class":"App\\Http\\Middleware\\Language","file":"\/home\/alariva\/fimedi\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php","function":"handle","line":151,"type":"->"},{"class":"Barryvdh\\Debugbar\\Middleware\\InjectDebugbar","file":"\/home\/alariva\/fimedi\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php","function":"handle","line":151,"type":"->"},{"class":"Illuminate\\Auth\\Middleware\\Authenticate","file":"\/home\/alariva\/fimedi\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php","function":"handle","line":151,"type":"->"},{"class":"Illuminate\\Cookie\\Middleware\\AddQueuedCookiesToResponse","file":"\/home\/alariva\/fimedi\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php","function":"handle","line":151,"type":"->"},{"class":"Illuminate\\Cookie\\Middleware\\EncryptCookies","file":"\/home\/alariva\/fimedi\/vendor\/laravel\/framework\/src\/Illuminate\/Pipeline\/Pipeline.php","function":"handle","line":151,"type":"->"},{"class":"Il
What I interpret after a couple of tests, is that it looks like once I mock Cache it is being called by some middlewares before reaching the tested block, so since those called methods are not mocked, the test fails because it does not know what to answer for those middleware calls.
Imagine I could successfully mock all the calls before getting to the tested codeblock, I would be able to make it reach. But that's not the way to go over it.
How can I mock Cache and avoid failure due to previous Cache calls that I'm not testing?
EDIT: I realized after getting to a solution that this is a misleading question. My actual need was:
How can I successfully cover those lines?
Sidenote: if I try to disable middlewares ($this->withoutMiddleware();) I get an AccessDeniedHttpException
alariva#trinsic:~/fimedi$ t --filter=test_it_deletes_a_patient
PHPUnit 7.3.1 by Sebastian Bergmann and contributors.
F 1 / 1 (100%)
Time: 12.95 seconds, Memory: 24.00MB
There was 1 failure:
1) Tests\Browser\Backoffice\PatientsTest::test_it_deletes_a_patient
Unable to find JSON fragment
["success":true]
within
[{"exception":"Symfony\\Component\\HttpKernel\\Exception\\AccessDeniedHttpException","file":"\/home\/alariva\/fimedi\/vendor\/laravel\/framework\/src\/Illuminate\/Foundation\/Exceptions\/Handler.php","line":201,"message":"This action is unauthorized.","trace":[{"class":"App\\Exceptions\\Handler","file":"\/home\/alariva\/fimedi\/vendor\/laravel\/framework\/src\/Illuminate\/Routing\/Pipeline.php","function":"render","line":83,"type":"->"},{"class":"Illuminate\\Foundation\\Exceptions\\Handler","file":"\/home\/alariva\/fimedi\/app\/Exceptions\/Handler.php","function":"render","line":65,"type":"->"},{"class":"Illuminate\\Foundation\\Exceptions\\Handler","file":
Maybe I can cherry-pick middlewares to disable?
I managed to cover the controller's method by encapsulating the custom Cache operation into a macro, so as to get the benefits of spliting into code units.
I moved my code into a macro (in the boot() of a service provider):
Cache::macro('incrementExisting', function($key, $amount) {
if (Cache::has($key)) {
Cache::increment($key, $amount);
}
return $this;
});
I refactored to use the macro
protected function updatePatientsCount($amount = 1)
{
$key = vsprintf('%s.%s', [auth()->user()->id, 'backoffice.stats.patientsTotalCount']);
Cache::incrementExisting($key, $amount);
}
I could get the desired coverage while I can still test the refactored code with unit testing.
Update I
Regarding the concern of handling many calls that are not mocked, I just learned from Adam Wathan that there exists shouldIgnoreMissing() and that would allow to use the Mocking approach for this case.
Update II
Write your tests first. When doing so it gets easier to avoid hard-to-test code.
The title is self explanatory but I'll elaborate.
I'm using FakeItEasy framework where I have the option to check the number of times a method have been invoked.
A.CallTo(() => foo.Bar()).MustHaveHappened();
But by doing this I'm testing the internal behavior of my code. I'm not testing a returned value or a state change which is what a good test should do.
so my question is this: is it a good practice to test the number of times a method have been called?
You should care about the number of times a dependency was invoked in many cases.
For example, suppose your have a service that inserts some data into some database by invoking a web service.
public interface IDataInserter
{
void Insert(Data[] data);
}
And assume that in some cases you need to insert some 10000 items. But the web service cannot handle such volume of data in a single call. So you decide to create a decorator that will split the data into multiple chunks and send each chunk in a single request.
public class SplittingDecorator : IDataInserter
{
private readonly IDataInserter m_DataInserter;
public SplittingDecorator(IDataInserter data_inserter)
{
m_DataInserter = data_inserter;
}
public void Insert(Data[] data)
{
var chunks =
data
.Select((d, i) => new {d, i})
.GroupBy(x => x.i/50)
.Select(x => x.Select(y => y.d).ToArray())
.ToList();
foreach (var chunk in chunks)
{
m_DataInserter.Insert(chunk);
}
}
}
When you want to test the SplittingDecorator class, you will create a mock for the data_inserter constructor argument.
In such test, you need to assert that the mocked IDataInserter was invoked some X times when you invoke SplittingDecorator.Insert with a data of size Y.
For example, if the data size (length of the data array) is 160 items, you want to check that the mock was invoked 4 times.
I'm trying out TDD on a greenfield hobby app in ASP.NET MVC, and have started to get test methods such as the following:
[Test]
public void Index_GetRequest_ShouldReturnPopulatedIndexViewModel()
{
var controller = new EmployeeController();
controller.EmployeeService = GetPrePopulatedEmployeeService();
var actionResult = (ViewResult)controller.Index();
var employeeIndexViewModel = (EmployeeIndexViewModel)actionResult.ViewData.Model;
EmployeeDetailsViewModel employeeViewModel = employeeIndexViewModel.Items[0];
Assert.AreEqual(1, employeeViewModel.ID);
Assert.AreEqual("Neil Barnwell", employeeViewModel.Name);
Assert.AreEqual("ABC123", employeeViewModel.PayrollNumber);
}
Now I'm aware that ideally tests will only have one Assert.xxx() call, but does that mean I should refactor the above to separate tests with names such as:
Index_GetRequest_ShouldReturnPopulatedIndexViewModelWithCorrectID
Index_GetRequest_ShouldReturnPopulatedIndexViewModelWithCorrectName
Index_GetRequest_ShouldReturnPopulatedIndexViewModelWithCorrectPayrollNumber
...where the majority of the test is duplicated code (which therefore is being tested more than once and violates the "keep tests fast" advice)? That seems to be taking it to the extreme to me, so if I'm right as I am, what is the real-world meaning of the "one assert per test" advice?
It seems just as extreme to me, which is why I a also write multiple asserts per test. I already have >500 tests, writing just one assert per test would blow this up to at least 2500 and my tests would take over 10 minutes to run.
Since a good rest runner (such as Resharper's) lets you see the line where the test failed very quickly, you should still be able to figure out why a test failed with little effort. If you don't mind the extra effort, you can also add an assert description ("asserting payroll number correct"), so that you can even see this without even looking at the source code. With that, there is very little reason left to just one assert per test.
In his book The Art of Unit Testing, Roy Osherove talks about this subject. He too is in favour of testing only one fact in a unit test, but he makes the point that that doesn't always mean only one assertion. In this case, you are testing that given a GetRequest, the Index method ShouldReturnPopulatedIndexViewModel. It seems to me that a populated view model should contain an ID, a Name, and a PayrollNumber so asserting on all of these things in this test is perfectly sensible.
However, if you really want to split out the assertions (say for example if you are testing various aspects which require similar setup but are not logically the same thing), then you could do it like this without too much effort:
[Test]
public void Index_GetRequest_ShouldReturnPopulatedIndexViewModel()
{
var employeeDetailsViewModel = SetupFor_Index_GetRequest();
Assert.AreEqual(1, employeeDetailsViewModel.ID);
}
[Test]
public void Index_GetRequest_ShouldReturnPopulatedIndexViewModel()
{
var employeeDetailsViewModel = SetupFor_Index_GetRequest();
Assert.AreEqual("Neil Barnwell", employeeDetailsViewModel.Name);
}
[Test]
public void Index_GetRequest_ShouldReturnPopulatedIndexViewModel()
{
var employeeDetailsViewModel = SetupFor_Index_GetRequest();
Assert.AreEqual("ABC123", employeeDetailsViewModel.PayrollNumber);
}
private EmployeeDetailsViewModel SetupFor_Index_GetRequest()
{
var controller = new EmployeeController();
controller.EmployeeService = GetPrePopulatedEmployeeService();
var actionResult = (ViewResult)controller.Index();
var employeeIndexViewModel = (EmployeeIndexViewModel)actionResult.ViewData.Model;
var employeeDetailsViewModel = employeeIndexViewModel.Items[0];
return employeeDetailsViewModel;
}
It could also be argued that since these tests require the same setup, they should get their own fixture and have a single [SetUp] method. There's a downside to that approach though. It can lead to lots more unit test classes than actual, real classes which might be undesirable.
I use a helper class to contain the asserts. This keeps the test methods tidy and focused on what they're actually trying to establish. It looks something like:
public static class MvcAssert
{
public static void IsViewResult(ActionResult actionResult)
{
Assert.IsInstanceOfType<ViewResult>(actionResult);
}
public static void IsViewResult<TModel>(ActionResult actionResult, TModel model)
{
Assert.IsInstanceOfType<ViewResult>(actionResult);
Assert.AreSame(model, ((ViewResult) actionResult).ViewData.Model);
}
public static void IsViewResult<TModel>(ActionResult actionResult, Func<TModel, bool> modelValidator)
where TModel : class
{
Assert.IsInstanceOfType<ViewResult>(actionResult);
Assert.IsTrue(modelValidator(((ViewResult) actionResult).ViewData.Model as TModel));
}
public static void IsRedirectToRouteResult(ActionResult actionResult, string action)
{
var redirectToRouteResult = actionResult as RedirectToRouteResult;
Assert.IsNotNull(redirectToRouteResult);
Assert.AreEqual(action, redirectToRouteResult.RouteValues["action"]);
}
}
If I have a method that calls itself under a certain condition, is it possible to write a test to verify the behavior? I'd love to see an example, I don't care about the mock framework or language. I'm using RhinoMocks in C# so I'm curious if it is a missing feature of the framework, or if I'm misunderstanding something fundamental, or if it is just an impossibility.
a method that calls itself under a certain condition, is it possible to write a test to verify the behavior?
Yes. However, if you need to test recursion you better separate the entry point into the recursion and the recursion step for testing purposes.
Anyway, here is the example how to test it if you cannot do that. You don't really need any mocking:
// Class under test
public class Factorial
{
public virtual int Calculate(int number)
{
if (number < 2)
return 1
return Calculate(number-1) * number;
}
}
// The helper class to test the recursion
public class FactorialTester : Factorial
{
public int NumberOfCalls { get; set; }
public override int Calculate(int number)
{
NumberOfCalls++;
return base.Calculate(number)
}
}
// Testing
[Test]
public void IsCalledAtLeastOnce()
{
var tester = new FactorialTester();
tester.Calculate(1);
Assert.GreaterOrEqual(1, tester.NumberOfCalls );
}
[Test]
public void IsCalled3TimesForNumber3()
{
var tester = new FactorialTester();
tester.Calculate(3);
Assert.AreEqual(3, tester.NumberOfCalls );
}
Assuming you want to do something like get the filename from a complete path, for example:
c:/windows/awesome/lol.cs -> lol.cs
c:/windows/awesome/yeah/lol.cs -> lol.cs
lol.cs -> lol.cs
and you have:
public getFilename(String original) {
var stripped = original;
while(hasSlashes(stripped)) {
stripped = stripped.substringAfterFirstSlash();
}
return stripped;
}
and you want to write:
public getFilename(String original) {
if(hasSlashes(original)) {
return getFilename(original.substringAfterFirstSlash());
}
return original;
}
Recursion here is an implementation detail and should not be tested for. You really want to be able to switch between the two implementations and verify that they produce the same result: both produce lol.cs for the three examples above.
That being said, because you are recursing by name, rather than saying thisMethod.again() etc., in Ruby you can alias the original method to a new name, redefine the method with the old name, invoke the new name and check whether you end up in the newly defined method.
def blah
puts "in blah"
blah
end
alias blah2 blah
def blah
puts "new blah"
end
blah2
You're misunderstanding the purpose of mock objects. Mocks (in the Mockist sense) are used to test behavioral interactions with dependencies of the system under test.
So, for instance, you might have something like this:
interface IMailOrder
{
void OrderExplosives();
}
class Coyote
{
public Coyote(IMailOrder mailOrder) {}
public void CatchDinner() {}
}
Coyote depends on IMailOrder. In production code, an instance of Coyote would be passed an instance of Acme, which implements IMailOrder. (This might be done through manual Dependency Injection or via a DI framework.)
You want to test method CatchDinner and verify that it calls OrderExplosives. To do so, you:
Create a mock object that implements IMailOrder and create an instance of Coyote (the system under test) by passing the mock object to its constructor. (Arrange)
Call CatchDinner. (Act)
Ask the mock object to verify that a given expectation (OrderExplosives called) was met. (Assert)
When you setup the expectations on the mock object may depend on your mocking (isolation) framework.
If the class or method you're testing has no external dependencies, you don't need (or want) to use mock objects for that set of tests. It doesn't matter if the method is recursive or not.
You generally want to test boundary conditions, so you might test a call that should not be recursive, a call with a single recursive call, and a deeply-recursive call. (miaubiz has a good point about recursion being an implementation detail, though.)
EDIT: By "call" in the last paragraph I meant a call with parameters or object state that would trigger a given recursion depth. I'd also recommend reading The Art of Unit Testing.
EDIT 2: Example test code using Moq:
var mockMailOrder = new Mock<IMailOrder>();
var wily = new Coyote(mockMailOrder.Object);
wily.CatchDinner();
mockMailOrder.Verify(x => x.OrderExplosives());
There isn't anything to monitor stack depth/number of (recursive) function calls in any mocking framework I'm aware of. However, unit testing that the proper mocked pre-conditions provide the correct outputs should be the same as mocking a non-recursive function.
Infinite recursion that leads to a stack overflow you'll have to debug separately, but unit tests and mocks have never gotten rid of that need in the first place.
Here's my 'peasant' approach (in Python, tested, see the comments for the rationale)
Note that implementation detail "exposure" is out of question here, since what you are testing is the underlying architecture which happens to be utilized by the "top-level" code. So, testing it is legitimate and well-behaved (I also hope, it's what you have in mind).
The code (the main idea is to go from a single but "untestable" recursive function to an equivalent pair of recursively dependent (and thus testable) functions):
def factorial(n):
"""Everyone knows this functions contract:)
Internally designed to use 'factorial_impl' (hence recursion)."""
return factorial_impl(n, factorial_impl)
def factorial_impl(n, fct=factorial):
"""This function's contract is
to return 'n*fct(n-1)' for n > 1, or '1' otherwise.
'fct' must be a function both taking and returning 'int'"""
return n*fct(n - 1) if n > 1 else 1
The test:
import unittest
class TestFactorial(unittest.TestCase):
def test_impl(self):
"""Test the 'factorial_impl' function,
'wiring' it to a specially constructed 'fct'"""
def fct(n):
"""To be 'injected'
as a 'factorial_impl''s 'fct' parameter"""
# Use a simple number, which will 'show' itself
# in the 'factorial_impl' return value.
return 100
# Here we must get '1'.
self.assertEqual(factorial_impl(1, fct), 1)
# Here we must get 'n*100', note the ease of testing:)
self.assertEqual(factorial_impl(2, fct), 2*100)
self.assertEqual(factorial_impl(3, fct), 3*100)
def test(self):
"""Test the 'factorial' function"""
self.assertEqual(factorial(1), 1)
self.assertEqual(factorial(2), 2)
self.assertEqual(factorial(3), 6)
The output:
Finding files...
['...py'] ... done
Importing test modules ... done.
Test the 'factorial' function ... ok
Test the 'factorial_impl' function, ... ok
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK
Is it possible to mock a stub/mock's object member call without having to define that as a stub, and also set the return value as all seperate verbose lines?
Example:
[TestMethod]
public void AssignedPermissions_AssociateExists_ReturnsEdit_Rhino()
{
//Arrange
var fakeConfiguration = MockRepository.GenerateStub<IDomainControllerConfiguration>();
var fakeAssociateRepository = MockRepository.GenerateStub<IAssociateRepository>();
fakeConfiguration.Stub(x => x.AssociateRepository).Return(fakeAssociateRepository);
fakeAssociateRepository.Stub(x=>x.GetAssociatesByRole(null,false,null)).IgnoreArguments()
.Return(new IAssociate[]{MockRepository.GenerateStub<IAssociate>()});
var domain = new DomainController(fakeConfiguration);
const AssignedPermission expected = AssignedPermission.Edit;
//Act
AssignedPermission actual = domain.AssignedPermissions();
//Assert
Assert.AreEqual(expected, actual);
}
Are all those temporary variables necessary just to stub out nested method calls?
I've never used the functionality, so I'm not 100% certain that this will work, but theoretically Rhino mocks supports "recursive mocking", which should allow you to at least cut out the fakeAssociateRepository by doing something like this:
var fakeConfiguration = MockRepository.GenerateStub<IDomainControllerConfiguration>();
fakeConfiguration.Stub(x => x.AssociateRepository.GetAssociatesByRole(null,false,null))
.IgnoreArguments()
.Return(new IAssociate[]{MockRepository.GenerateStub<IAssociate>()});
var domain = new DomainController(fakeConfiguration);
(note: code not tested, or even compiled)
Just wanted to share my input on this, since I just spent the last few hours wrestling with it. The answer posted above by Alconja absolutely works, but if you plan to use it for "AssertWasCalled" type of assertion, it does not assert the way I expected it to. It seems that the AssertWasCalled methods tried to assert the "get accessor" associated with the "nested" object.
For instance, if you wanted to do this:
fakeconfiguration.AssertWasCalled(x => x.AssociateRepository.GetAssociatesByRole(null, false, null));
You would get an exception such as
System.InvalidOperationException : Previous method 'IDomainControllerConfiguration.get_AssociateRepository();' requires a return value or an exception to throw.
Because the AssertWasCalled is asserting the get-accessor of the AssociateRepository property, rather than the GetAssociatesByRole() method. In the end, for my case I had to use the OP's methodology of creating mutliple stubs.