I have a quite complicated method for which I want to test the behavior (using Mockito and JUnit). This method takes an object (let's call its type State) as input, and should take a few different state variables into account for deciding its output.
As an example, considering the following specification (s is a mock of the State class):
If s.varOne is set, return its value.
Else, if s.varTwo is set, return that instead.
Else, if s.varThree is set, call s.update(s.varThree) and then return s.varOne, which will now have a value (even though it didn't at stage 1.)
Else, throw an error.
In order to test case 3 properly, I would like to set up the s object so that s.varOne and s.varTwo are both unset to begin with, but if (and only if!) the sut calls s.update(s.varThree), then after that s.varOne returns something.
Is there a good way to setup this behavior in Mockito?
I have considered setting up some chain of return values for s.varOne, and then verifying that the order of the calls corresponds to the order of the outputs (as well as that the return value of the sut is correct, of course), but this feels dirty; if I then change the method to calculate its return value in some other way, which calls s.varOne a different number of times but doesn't change its output, then the test will fail even though the functionality is the same.
My ideal solution is a way where I can add some "delayed" setup for the mock object, which is run when the sut calls the s.update() method, but I can't figure out a way to accomplish that.
You have a couple of options to mock a state change here, a good option and a better option. The best option, as tieTYT notes above, is to just to untangle the general contract of State: Does it really make sense for State to be mutable, and to have self-mutating methods that aren't simple setters?
The good option (sometimes) is to create a disposable subclass—or, better, an interface implementation—of State.
#Test public void yourTest() {
SystemUnderTest sut = createSystemUnderTest();
State state = new State() {
#Override public void update() {
this.varOne = 42;
}
}
// rest of your test
}
At that point, you may not need Mockito at all. Be warned, though: This can get a little tricky if State has side-effects or is otherwise difficult to instantiate. You could extract an interface, which would require you to wrap your fields in getters; you could also make State a named abstract class and then pass mock(MyAbstractState.class, CALLS_REAL_METHODS), but that gets particularly hairy when you consider that no initializer actually runs on that fake instance, and consequently the fields are all zero or null. If it's not simple, don't waste your time forcing a square peg into a round hole.
A more-common pattern for mocks is to use a custom Answer, in which you can execute arbitrary code at the expense of type safety. Here's an example, assuming update is void and varThree is an integer:
#Test public void yourTest() {
SystemUnderTest sut = createSystemUnderTest();
final State s = Mockito.mock(State.class);
doAnswer(new Answer<Void>() {
#Override public Void answer(InvocationOnMock invocation) {
int actualVarThree = (int) invocation.getArguments()[0];
assertEquals(EXPECTED_VAR_THREE, actualVarThree);
s.varOne = actualVarThree;
return null;
}
}).when(s).update(anyInt());
// rest of your test
}
Note that the array of arguments is an Object[], so the cast is necessary and slightly-dangerous, but then you can make all sorts of assertions and modifications synchronously when your mocked method is called.
Hope that helps!
Related
I have this problem. I have a method in a class which gives me some return value. This method also makes changes to the state of the instance of that class. I need to unit test them both, so I made the test cases for the return value and copied them and just changed the assert from testing the return value to testing the change in representation (the only way for me to test this class is to test its representation). However, this does not seem like a very good way to deal with this. If I ever have changes in the functionality of the class, I will have to change the Act in the AAA in both test cases. How do I go about testing this? Is there some pattern to do that? Having both asserts in one test would mean I would be testing two things in one test case. In some languages (like JavaScript), I know I can make an array of tuples with the functionality, the assert for the return value and the assert for the representation, but I'm not sure a) how much better that is over some copy-pasting and b) what to do in other languages (e.g. in C# I imagine I'd have to make some classes with the asserts and include them in test cases).
Edit: For example:
class A {
state;
foo(animal, foodAmount) {
let returnResult = {};
//does things with animal and foodAmount
//which things change BOTH returnResult and the state variable
return returnResult;
}
bar() {
let stateToReturn = "";
//state is used to change the stateToReturn variable
return stateToReturn;
}
}
Here I will test function foo with a) an animal that does not exist currently, b) an animal that exists where I will add a non-zero amount of food to make sure the amount of food changes, c) two different animals to make sure they do not interfere with each other's food amounts, etc. I will do all these tests and make sure the returnResult variable is correct. The problem is that I need to make sure it does not only affect the returnResult variable, but also the state. The bar function is the only way for me to see the inner state. How would I go about testing the foo function? Would I write all tests twice (once to check foo returns the correct result and once to check foo changes the state correctly)?
Even with your example it's still a bit vague. The general and short answer to your question is: You should test for behavior, not methods. If that requires you to make assertions on the result of multiple methods in one test that's perfectly fine.
You shouldn't, however, expose the internal state of your class just to make it testable. It's better to test against the public api of your class. After all, that's how the production code interacts with it.
If a method changes the internal state but there is nothing in the public api that makes the change visible, how do you test it? You probably have a hidden concept in your class that should be extracted into its own class with its own public api. You can test the new class independently and the interaction between the two classes. This leads to smaller and more focused tests.
I'm not sure if this answers your question sufficiently but maybe there's some food for thought.
If method A's single role is to call method B, should I write a test that verifies method B is called when I call method A? Or is this a waste?
EDIT: I am editing to add some context. Here is my class:
module PaidGigs
class UserValue
def initialize(user)
#user = user
end
def default_bid(multiplier = 3.5)
PaidGigs::UserValue.cpm_value(#user.instagram_follower_count, multiplier)
end
def bid_value_including_markup(user_bid, multiplier = 3)
user_bid + PaidGigs::UserValue.cpm_value(#user.instagram_follower_count, multiplier, 0)
end
def self.cpm_value(base_count, multiplier, rounder = -1)
((base_count.to_f / 1000) * multiplier).round(rounder)
end
end
end
Should I write a test that verifies '#default_bid' calls '.cpm_value' with the proper arguments? Is this a waste of time, or is there value in this?
Don't test that one method within a method calls another. This will lead to a more fragile test, and create a barrier (although a small one) to refactoring. Your test should not care about the internals that produce the result, only that it's correct.
The answer changes if the method is delegating to another class - then you absolutely need to test, in a unit test, that the delegation occurs corectly by mocking the delegate.
There's value in methods that call others within the class: the name of the methods communicates the purpose of the method (if it's named well) and that's good for the human readers of your class. Well-named methods (and everything else) is hugely powerful, and largely underutilized. The names of low-level methods within a class can create a mini-DSL (Domain Specific Language) within the class, helping the reader to quickly understand what a high-level method is doing without taking the time to dig into the details.
What you're asking about is does it make sense to explicitly test everything your object does, or do you want to implicitly test it.
This is mostly a matter of opinion.
Personally, I see no value in writing that test, because eventually you should be writing a test that mocks out the return value of the method and verifying that function your testing actually mutates the mocked value in such a way. This way you implicitly test that you're calling the function - if you weren't using the method to get your end result, then the value would not align to your expected result.
Edit: A code example in something a bit more readable:
Calculator.cs
public class Calculator {
private IAddingService _addingService;
public Calculator(IAddingService addingService) {
_addingService = addingService;
}
public AddNumbers(int valueOne, int valueTwo) {
return _addingService(valueOne, valueTwo);
}
}
CalculatorTests.cs
public class CalculatorTests {
public void test_adding_numbers() {
var addingService = new Mock<IAddingService>()
addingService.Setup(service => service.Add(1,2)).Returns(2);
var calculator = new Calculator(addingService.Object);
var result = calculator.Add(1,2);
Assert.That(result, Is.EqualTo(2));
}
}
In this example, I've implicitly tested that adding service is the way that we add things because there's no action that allows calculator to determine this on its own.
I don't see why you wouldn't test it. It's easy to do and partially prevents further refactoring of this method to break something.
I'll start my question with short example:
SomeResult DoSomething(input)
{
var a = svc1.getA(input);
if (condition with a)
{
var b = svc2.getB(a);
if (cond with b)
{
var c = svc3.getC(b);
if (cond with c)
{
}
else
{
}
}
else
{
}
}
else
{
}
}
I believe the idea is clear here. We have complex branching logic where conditions depend on interim results returned by injected services.
When we want the part of cond with c we have to mock svc1 and svc2 and svc3.
To apppear at cond with b we have to mock svc1 and svc2.
Thus we replay all upper parts of execution path every time we go level deeper. Guess how it is usually done? Bingo, copy-paste!
We have bunches of unit tests where most of the lines are occupied by objects' (a,b,c...) initialization and services mocking. When a, b or c are objects with tens of properties all this looks like a real hell. Tiny change in cond with a can easily break 20 tests simultaneously.
I insist on having some notion of "jump strait to the place I want to test".
What if we changed the code like that:
SomeResult DoSomething(input)
{
var a = svc1.getA(input);
if (condition with a)
{
var b = svc2.getB(a);
if (cond with b)
{
ProcessBLikeThis(b);
}
else
{
}
}
else
{
}
}
Then we could test ProcessBLikeThis separately from unrelated logic.
Yet for it to be testable it must be public. Moreover, as we want to have tests verifying that ProcessBLikeThis was called with the given argument depending on cond with b we either need to use isolator or make ProcessBLikeThis to be a method of some interface.
However, there is no other necessity for such granular design besides DRY-adherent testability.
So I'd appreciate some guidance here how to design and test such methods.
Addition:
I also forget to mention that my teammates are strongly against putting initialization logic in reusable methods as they see no strict border line between what can be put there and what can not and expect that some day someone will extend the code and break tests logic. They prefer copy paste as a mean of isolation.
my teammates are strongly against putting initialization logic in reusable methods as they see no strict borderline between what can be put there and what can not and expect that some day someone will extend the code and break test logic. They prefer copy / paste as a mean of isolation.
If your team wants to repeat themselves and you don't want them to, then that discussion needs to take place and a consensus formed so that everybody works with the same goals in mind. There are arguments for repeating some test setup code, usually down to readability, however, this can usually be overcome by naming any methods and variables sensibly so that their usage is obvious.
The argument that tests can't reuse code because somebody might change the shared code and break tests is a bit of a null argument. If somebody did change the shared logic and a bunch of tests didn't break you would have a larger problem. As you've said, the more likely scenario is that a small change in the production code will result in a bunch of tests failing. If the tests don't share relevant setup code, then the fix is likely to be blindly copy/pasted into each of the tests to make them work.
That said, the usual way to simplify testing is to create a different level of indirection so you're testing less. With the code you've posted, one approach might be to separate the flow logic from the action logic.
You might end up with code something like this (the names would obviously need to be tailored to your situation):
interface ISomeActioner {
bool IsTriggered( SomeStateProvider state);
SomeResult TriggeredAction(SomeStateProvider state);
SomeResult UntriggeredAction(SomeStateProvider state);
}
SomeResult DoSomething(input) {
SomeResult result = Unknown;
foreach(var actioner in _someActions) {
if(IsTriggered(/* some state provider */)) {
result = actioner.TriggeredAction(/* some state provider */);
} else {
result = actioner.UntriggeredAction(/* some state provider */);
}
if(result != Unknown) break;
}
return result;
}
You then end up implementing several classes that implement the ISomeActioner interface. Each one of these classes is straightforward; it checks the state from the state provider and returns a flag to indicate which of its other functions should be called. These classes can be tested in isolation to make sure that each public method does what is expected, by setting up the SomeStateProvider to the appropriate state before calling each of its methods.
An ordered list of these classes would then need to be injected into the class containing the DoSomething method. This allows you to use mocked instances of the interface when testing the DomeSomething method, which effectively becomes testing of a for loop.
Since the datastructure of my application domain is becoming pretty complex as of late, I started reading up on mock objects. Soon a simple question came to my mind, but the answer has proven to be quite the headache so far. So here goes:
We have a class 'Foo' with 'bar' as one of its methods:
class Foo {
public String bar(int i){
if(i == 1) return "arrr!";
}
}
And we have a class Pirate calling Foo.bar(1); in one of its methods:
class Pirate {
public String yell(){
Foo foo = new Foo();
return foo.bar(1);
}
Now we mock the Foo class in the unit test of the Pirate class because Foo happens to have a plethora of other dependencies:
#Test
public void returnsPirateString() {
Pirate blackBeard = new Pirate();
Foo fooMock = mock(Foo.class);
fooMock.expectAndReturn("bar",1,"arrr!"); //expects 'bar' function to be called once and returns "arrr!"
assertEquals(blackBeard.yell(),"arrr!");
}
What happens now, is that if we refactor method bar to return null instead of "arrr!", our test will keep running happily while our program does not work the way we want it to. This may result in a possible debugging nightmare.
Using a mockist approach instead of the classical testing approach to unit testing, most of the time all "helper" objects get mocked, and only the tested object remains unmocked, so the previously stated problem can occur quite often as well.
What can be done to prevent this problem while mocking?
In your test, you are testing the yell() method of Pirate class which uses Foo. So you have to mock the behavior of Foo's bar method.
To make sure your bar method is functioning correctly, you need another test case to test the bar method of Foo.
#Test
public void testBar() {
//make sure bar retrun "arrr"!
}
Now if your bar method returns null, this test case will fail!
You should be testing the 'helper object' in isolation as well. Once both of those are covered and tested, then you can be sure that both interact with each other in the expected way.
Changing your 'helper object' is something that should be done with tests against that helper object to confirm it still behaves as expected.
If you are concerned about the specific runtime behavior of the combination of helper and primary class, then you should use integration tests, or some other test at a higher level, to assert the two work together as expected.
The test returnsPirateString is not a false positive - it's testing what happens when a Pirate's Foo instance returns 'arrr!'
In other words, when you're testing Pirate.yell, it doesn't matter what Foo.bar returns, unless it creates a special boundary condition (and you should probably already have a test that documents what yell does when Foo returns null).
Pirate.yell is not responsible for guaranteeing any particular return value for Foo.bar, so its unit tests should not expect any particular return values.You should even make a point of changing your test to use something other than the current return value of Foo.bar.
I'm using TypeMock Isolater to mock up some objects for some unit tests - attempting to use the AAA api (so the Isolate calls).
I have a straightforward singleton class where you call a static GetInstance(), which then returns an instance of the class. I thought it would be a simple matter of mocking that up, but I'm running into a very frustrating problem ! I can't seem to make GetInstance() return my mocked object correctly with my expected calls set.
I've tried:
using MST projects (using the Accessor classes) to assign a mocked object directly to the instance variable (faking the object using Memers.MustSpecifyReturnValues, and Isolate.WhenCalled using WithExactArguments to set expectations), but for some reason the mocked object always returns null (and no exceptions).
Mocking Singleton.GetInstance() to return the mocked object. This returns a mocked object which needs WhenCalled set, but now the Isolate.WhenCalled calls I make seem to do nothing on the fake object - so all calls throw an unexpected call exception.
I've also tried mocking the actual method call (eg Singleton.GetInstance().Test()), which will work for the call to that method, but all other calls to other methods on the singleton return null rather then throw an exception as I want it to (because this seems to automatically mock up all the objects without Members.MustSpecifyReturnValues).
All I want is to mock a singleton, and any calls I don't explicitly tell it to expect to throw an exception on. I thought it would be simple, but apparently not ! Sad
Has anyone any idea what I'm doing wrong?
Thanks
James
I think the simple solution will be to create a fake instance of the singleton class and use SwapNextInstace before the actual class constructor is called:
[TestMethod]
public void SetBhaciorOnSingleton()
{
var fake = Isolate.Fake.Instance<SingletonClass>();
Isolate.WhenCalled(() => fake.SomeFunction()).WillReturn(10);
// Set additional behavior on singleton class
Isolate.Swap.NextInstance<SingletonClass>().With(fake);
// This is where the class constructor is being called
var result = SingletonClass.GetInstace().SomeFunction();
Assert.AreEqual(10, result );
}
This solution should work with most scenarios unless the singleton class is created before the test.
If you need to set behavior after the class was created just use WhenCalled:
[TestMethod]
public void SetBhaciorOnSingleton()
{
var fake = Isolate.Fake.Instance<SingletonClass>();
Isolate.WhenCalled(() => fake.SomeFunction()).WillReturn(10);
Isolate.WhenCalled(() => SingletonClass.GetInstace()).WillReturn(fake);
var result = SingletonClass.GetInstace().SomeFunction();
Assert.AreEqual(10, result );
}
Disclaimer I work at Typemock.
You don't need to mock Singleton.GetInstance<>(). Using Isolate.Fake.AllInstances<>() instead of Isolate.Fake.Instance<>() you can mock the singleton. Then by setting the behavior on fake singleton behavior applied to all instances.
Take a look on the example:
public class Singleton
{
private Singleton() { }
static readonly Singleton instance = new Singleton();
public static Singleton Instance { get { return instance; } }
public int ReturnZero()
{
return 0;
}
}
[TestMethod]
public void FakeSingleton()
{
// Here we are setting the same behavior on all instances.
// The behavior we set on fake will apply to past instance as well
var fakeSingleton = Isolate.Fake.AllInstances<Singleton>();
Isolate.WhenCalled(() => fakeSingleton.ReturnZero()).WillReturn(10);
// Assert that the behavior works.
Assert.AreEqual(10, Singleton.Instance.ReturnZero());
}
Thanks.
I didn't try NextInstance before because it doesn't work on interfaces which I didn't really want to change.
But, I've tried it and it does work - although I was assuming the order of setting WhenCalled(s) doesn't really matter, but it definately does. If I do the WhenCalled after the Swap for instance, it doesn't work. It needs to go before the Swap. (Doesn't really make sense to me to be honest - it should be the same object).
However, the last example (one of the ways I had tried), doesn't work for me. I fake, set expecation on fake, and then set expectation on Singleton to return faked instance - but now it returns the concrete instance !
Could it have something to do with the way the constructors are called? I remember seeing something about that...
Alternatively I could use the Swap, but, I wanted to be able to setup all this stuff in a TestSetup, and make minor modifications to the expectations in the actual test, but that doesn't look possible. ?
The best solution is to not use singletons (or any other static mutable data). Just create one instance and use dependency injection to pass it to all objects who need it.
http://butunclebob.com/ArticleS.UncleBob.SingletonVsJustCreateOne
http://www.youtube.com/watch?v=-FRm3VPhseI