I'm trying to uss the Akka BehaviorTestKit to verify that an anonymous actor of a specific type is spawned.
I tried to use:
testKit.expectEffectType[SpawnedAnonymous[MyActor.Request]]
When I did this, I got an AssertionError because there were other effects that came before the the one I'm testing for. It seems that expectEffectType() only looks at the effects in the order that they occurred. I want to ignore any other previous effects and only care that the one I'm testing for happened at some point. Relying on specific other prior effects would make the test brittle.
So, I then switched gears and tried the following:
val effects: Seq[Effect] = testKit.retrieveAllEffects()
assert(!effects.filter(effect => effect.isInstanceOf[SpawnedAnonymous[Behavior[MyActor.Request]]]).isEmpty)
This turned out to not be a valid test. It always succeeds, regardless of the Behavior type that I check for, due to JVM generic type erasure.
How can I verify that an anonymous actor of a specific type was spawned at any point of time?
Yes, it's problematic. You have 2 choices.
val spawned = effects.find{case _: SpawnedAnonymous[_] => true ...}
spawned.ref ! someMessageThatWillHaveAKnownEffect
Or... as it says in https://doc.akka.io/docs/akka/current/typed/testing-sync.html
Interactions with other actors must be stubbed.
It's not explicit, but part of the meaning to me is that you can't leave behaviors be defined by the actor under test. So instead of
ctx.spawn(Behaviors.supervise[MyActor.Request](Behavior(args)).onFailure(...)), I'll instead construct MyActor with an
trait MyActorConstructors {
def requestHandler(args): Behavior[MyActor.Request]
}
and then call ctx.spawn(constructors.requestHandler(args))
When you construct MyActor you pass in the standard implementations, and from your test you pass in mock actors.
That way you can easily determine effects.contains(SpawnedAnonymous(myMockedRequestHandler))
It's a bit of a hassle to set up, but it allows you to explicitly perform synchronous behavior testing (which is what BehaviorTestKit is primarily for), and also gives you a simplified testing point for the standard behaviour implementations in isolation. IdiomaticMockito (from mockito-scala) makes it nicer to deal with as well.
Related
I am trying to wrap my head around the practice of unit testing. I have read that unit tests should not be dependent on the implementations.
The way I understand this is, for example, if we have a function int addTwoNumbers (int a, int b) we should be testing whether, for example addition return correct results (i.e. addTwoNumbers (2, 2) == 4)) and we should not care whether, for example, addTwoNumbers calls operator + just once - it might as well use bit manipulation for this.
It seemed reasonable for me and - in my opinion - offers a decent decoupling between the tests and the code.
Enter the mocking frameworks. As far I can tell from reading their documentation, their functionality amounts to
(a) generating objects that implement placeholder functionality of some interfaces of base classes (stubbing) and
(b) checking whether that functionality was called according to expectations set forth by the tester.
I have no problems with (a) - I understand that we sometimes need to hard-code some functionality of external dependencies for the testing. I do not understand though, why should we check whether the tested code called mock's functionality in a way expected by the tester.
Shouldn't we be interested only in what a tested method returns or how it modifies its out arguments and not really care about its implementation details? Does the verification functionality of the mocking frameworks not introduce tight coupling between tested and testing code?
I do not understand though, why should we check whether the tested code called mock's functionality in a way expected by the tester
Because the contract of a method is not always to return something or to modify its arguments. Sometimes the contract of the method is (or includes) to have side effects. Think for example of the following method:
void notifyServerOfError(error: string) {
this.http.post('/api/errors', {
error: error,
ip: myIpAddress
});
}
This method doesn't return anything. It doesn't modify its arguments. Its sole responsibility is to send a specific object, containing specific details, to a specific URL. And unit-testing this method should thus verify that the contract is respected.
A good way to do that is to mock the http dependency and to check that, when this method is called, its post() method indeed being called with the correct URL and the correct data.
I am sure this is a silly question but I simply do not know - suppose I have a method that is expected to do nothing for certain inputs. Like, value < 10, print chars, otherwise do nothing.
Basically, the unit test would pass for the actual method or for one with just an empty body. How do I write a failing test for such an instance?
Your unit tests assertions should be based on unit observable effects — return value or (unfortunately) side effects. Sometimes you do want to test behavior instead of result with help of mocks, but this makes tests fragile.
If you have a value returned everything is simple, you just check that value satisfies some known postconditions.
In case of side effects, you are forced to setup proper test environment (some global state for example), and then check how it was changed. For your concrete example you may redirect output and verify later that something was written to it on values less than 10 and nothing was written otherwise.
So generally you still want to check that method did nothing, because it's a separate execution flow.
I know that many mocking libraries let the programmer check that a method was called only once. But why is that useful?
Also, why is it useful to verify the parameters to a mock's method?
When doing unit tests you are testing a isolated method: all other methods are supposed to work correctly, and you test only that your method behaves in the expected (specified...) way.
But in many occasions the expected way implies calling methods of classes you depend on (via dependency injection, if you want to do unit testing).
For these reason you need to assure that this calls are really done... and of course they are called with the expected parameters.
Example:
In your real application you have a repository class that stores all your changes in the database (and does only this!). But for unit test your "business" class (where all your business rules are defined), you should mock that "repository" class: then you must check that this mocked class receives the correct updated data.
In general, the answers to both of those questions are the same.
It's useful if the requirements of the unit/method you're testing specify that that behavior is required. If that behavior is required, then that's what you need to verify is actually happening.
If it's important to ensure that a particular method is only called once, then you can do that. If it doesn't matter that a method is called more than once, then don't test for it explicitly. Note that the default for the Mockito "verify" method is "times(1)", which means that it confirms that the method was called once and only once.
Concerning the second question, you might want to verify the parameters if it's entirely possible the method could be called with different parameters, and you don't want to count those occurrences, you only care about a specific set of parameter values.
I have started using moq for mocking. Can someone explain me the concept of strict and non-strict mocks? How can they can be used in moq?
edit:
in which scenario do we use which type of mock?
I'm not sure about moq specifically, but here's how strict mocks work in Rhino. I declare that I expect a call to foo.Bar on my object foo:
foo.Expect(f => f.Bar()).Returns(5);
If the calling code does
foo.Bar();
then I'm fine because the expectations are exactly met.
However, if the calling code is:
foo.Quux(12);
foo.Bar();
then my expectation failed because I did not explicitly expect a call to foo.Quux.
To summarize, a strict mock will fail immediately if anything differs from the expectations. On the other hand, a non-strict mock (or a stub) will gladly "ignore" the call to foo.Quux and it should return a default(T) for the return type T of foo.Quux.
The creator of Rhino recommends that you avoid strict mocks (and prefer stubs) because you generally don't want your test to fail when receiving an unexpected call as above. It makes refactoring your code much more difficult when you have to fix dozens of test that relied on the exact original behavior.
Ever come across Given / When / Then?
Given a context
When I perform some events
Then an outcome should occur
This pattern appears in BDD's scenarios, and is also relevant for unit tests.
If you're setting up context, you're going to use the information which that context provides. For instance, if you're looking up something by Id, that's context. If it doesn't exist, the test won't run. In this case, you want to use a NiceMock or a Stub or whatever - Moq's default way of running.
If you want to verify an outcome, you can use Moq's verify. In this case, you want to record the relevant interactions. Fortunately, this is also Moq's default way of running. It won't complain if something happens that you weren't interested in for that test.
StrictMock is there for when you want no unexpected interactions to occur. It's how old-style mocking frameworks used to run. If you're doing BDD-style examples, you probably won't want this. It has a tendency to make tests a bit brittle and harder to read than if you separate the aspects of behaviour you're interested in. You have to set up expectations for both the context and the outcome, for all outcomes which will occur, regardless of whether they're of interest or not.
For instance, if you're testing a controller and mocking out both your validator and your repository, and you want to verify that you've saved your object, with a strict mock you also have to verify that you've validated the object first. I prefer to see those two aspects of behaviour in separate examples, because it makes it easier for me to understand the value and behaviour of the controller.
In the last four years I haven't found a single example which required the use of a strict mock - either it was an outcome I wanted to verify (even if I verify the number of times it's called) or a context for which I can tell if I respond correctly to the information provided. So in answer to your question:
non-strict mock: usually
strict mock: preferably never
NB: I am strongly biased towards BDD, so hard-core TDDers may disagree with me, and it will be right for the way that they are working.
Here's a good article.
I usually end up having something like this
public class TestThis {
private final Collaborator1 collaborator1;
private final Collaborator2 collaborator2;
private final Collaborator2 collaborator3;
TestThis(Collaborator1 collaborator1, Collaborator2 collaborator2, Collaborator3 collaborator3) {
this.collaborator1 = collaborator1;
this.collaborator2 = collaborator2;
this.collaborator3 = collaborator3;
}
public Login login(String username) {
User user = collaborator1.getUser(username);
collaborator2.notify(user);
return collaborator3.login(user);
}
}
...and I use Strict mocks for the 3 collaborators to test login(username). I don't see how Strict Mocks should never be used.
I have a simple convention:
Use strict mocks when the system under test (SUT) is delegating the call to the underlying mocked layer without really modifying or applying any business logic to the arguments passed to itself.
Use loose mocks when the SUT applies business logic to the arguments passed to itself and passes on some derived/modified values to the mocked layer.
For eg:
Lets say we have database provider StudentDAL which has two methods:
Data access interface looks something like below:
public Student GetStudentById(int id);
public IList<Student> GetStudents(int ageFilter, int classId);
The implementation which consumes this DAL looks like below:
public Student FindStudent(int id)
{
//StudentDAL dependency injected
return StudentDAL.GetStudentById(id);
//Use strict mock to test this
}
public IList<Student> GetStudentsForClass(StudentListRequest studentListRequest)
{
//StudentDAL dependency injected
//age filter is derived from the request and then passed on to the underlying layer
int ageFilter = DateTime.Now.Year - studentListRequest.DateOfBirthFilter.Year;
return StudentDAL.GetStudents(ageFilter , studentListRequest.ClassId)
//Use loose mock and use verify api of MOQ to make sure that the age filter is correctly passed on.
}
I think understand the definition of State / Interaction based testing (read the Fowler thing, etc). I found that I started state based but have been doing more interaction based and I'm getting a bit confused on how to test certain things.
I have a controller in MVC and an action calls a service to deny a package:
public ActionResult Deny(int id)
{
service.DenyPackage(id);
return RedirectToAction("List");
}
This seems clear to me. Provide a mock service, verify it was called correctly, done.
Now, I have an action for a view that lets the user associate a certificate with a package:
public ActionResult Upload(int id)
{
var package = packageRepository.GetPackage(id);
var certificates = certificateRepository.GetAllCertificates();
var view = new PackageUploadViewModel(package, certificates);
return View(view);
}
This one I'm a bit stumped on. I'm doing Spec style tests (possibly incorrectly) so to test this method I have a class and then two tests: verify the package repository was called, verify the certificate repository was called. I actually want a third to test to verify that the constructor was called but have no idea how to do that! I'm get the impression this is completely wrong.
So for state based testing I would pass in the id and then test the ActionResult's view. Okay, that makes sense. But wouldn't I have a test on the PackageUploadViewModel constructor? So if I have a test on the constructor, then part of me would just want to verify that I call the constructor and that the action return matches what the constructor returns.
Now, another option I can think of is I have a PackageUploadViewModelBuilder (or something equally dumbly named) that has dependency on the two repositories and then I just pass the id to a CreateViewModel method or something. I could then mock this object, verify everything, and be happy. But ... well ... it seems extravagant. I'm making something simple ... not simple. Plus, controller.action(id) returning builder.create(id) seems like adding a layer for no reason (the controller is responsible for building view models.. right?)
I dunno... I'm thinking more state based testing is necessary, but I'm afraid if I start testing return values then if Method A can get called in 8 different contexts I'm going to have a test explosion with a lot of repetition. I had been using interaction based testing to pass some of those contexts to Method B so that all I have to do is verify Method A called Method B and I have Method B tested so Method A can just trust that those contexts are handled. So interaction based testing is building this hierarchy of tests but state based testing is going to flatten it out some.
I have no idea if that made any sense.
Wow, this is long ...
I think Roy Osherove recently twitted that as a rule of thumb, your tests should be 95 percent state-based and 5 percent interaction-based. I agree.
What matters most is that your API does what you want it to, and that is what you need to test. If you test the mechanics of how it achieves what it needs to do, you are very likely to end up with Overspecified Tests, which will bite you when it comes to maintainability.
In most cases, you can design your API so that state-based testing is the natural choice, because that is just so much easier.
To examine your Upload example: Does it matter that GetPackage and GetAllCertificates was called? Is that really the expected outcome of the Upload method?
I would guess not. My guess is that the purpose of the Upload method - it's very reason for existing - is to populate and serve the correct View.
So state-based testing would examine the returned ViewResult and its ViewModel and verify that it has all the correct values.
Sure, as the code stands right now, you will need to provide Test Doubles for packageRepository and certificateRepository, because otherwise exceptions will be thrown, but it doesn't look like it is important in itself that the repository methods are being called.
If you use Stubs instead of Mocks for your repositories, your tests are no longer tied to internal implementation details. If you later on decide to change the implementation of the Upload method to use cached instances of packages (or whatever), the Stub will not be called, but that's okay because it's not important anyway - what is important is that the returned View contains the expected data.
This is much more preferrable than having the test break even if all the returned data is as it should be.
Interestingly, your Deny example looks like a prime example where interaction-based testing is still warranted, because it is only by examining Indirect Outputs that you can verify that the method performed the correct action (the DenyPackage method returns void).
All this, and more, is explained very well in the excellent book xUnit Test Patterns.
The question to ask is "if this code worked, how could I tell?" That might mean testing some interactions or some state, it depends on what's important.
In your first test, the Deny changes the world outside the target class. It requires a collaboration from a service, so testing an interaction makes sense. In your second test, you're making queries on the neighbours (not changing anything outside the target class), so stubbing them makes more sense.
That's why we have a heuristic of "Stub Queries, Mock Actions" in http://www.mockobjects.com/book