Say you have 3 functions, functionA, functionB, and functionC
functionC relies on functionA and functionB
functionA(a) {
return a
}
functionB(b) {
return b
}
functionC(a, b){
return functionA(a) + functionB(b);
}
Now this is obviously a super simplified example.. but what's the correct way to test functionC? If I'm already testing functionA and functionB and theyre passing wouldn't testing functionC wind up being more than a unit test since it would be relying on functionA and functionB returns..
For the first two functions you would focus on their public contract - you would write as many tests as required to ensure all results for different cases are as expected.
But for the third function it might be sufficient to understand that the function should be invoking the other two functions. So you probably need less tests. You don't need to test again all cases required for testing A and B. You only want to verify the expected "plumbing" that C is responsible for.
In my opinion your tests should not know that functionC is using functionA and functionB. Normally you create automatic tests, to support change (code maintenance). What if you change the implementation of C? All the tests of functionC become invalid as well, that is unnecessary and dangerous, because that means, the refactorer must understand all the tests as well. Even if he/she is convinced, that he/she is not changing the contract. If you have a great testcoverage, why should he/she do that? So the public contract of functionC is to be tested in full!
There is a further danger, if the tests know too much about the inner workings of the sut(functionC) they tend to reimplement the code inside. So the same (probably faulty) code that does the implementation, checks if the implementation is correct.
Just an example: How would you implement the (whitebox) test of functionC. Mock functionA and functionB and look if the sum of the mocked results is produced. That is just good for the test-coverage (kpi??) but can be also quite misleading.
But what about the high extra effort of testing the functionality of functionA and functionB twice. If that is so, then probably reuse of the testing code is easily possible, if the reuse is not possible, I think that confirms my earlier statements the more.
The GhostCat answer is simple, fine and focus on the essential.
I will detail about some other points to consider, particularly the refactoring question.
Unit tests focus on API
The classes API (public functions) have to be unit tested.
If these 3 functions are public, each one has to be tested.
Besides, unit tests don't focus on implementation but expected behavior.
Today the composite function add individual function results, tomorrow it could substract them or anything else.
Testing the C() composite function doesn't mean testing again all scenarios of A() and B(), it means testing the expected behavior for C().
In some cases, unit testing a composite function in integration with individual functions doesn't generate many duplication concerning individual functions.
In other cases, it does. I will present it in the next point.
Example where testing the C() composite function may cause a duplication concern in the tests.
Suppose that the A() function accepts two integers :
function A(int a, int b){ ...}
It has the following constraints about the input parameters :
they have to be >=0
they have be inferior to 100
their sum has be inferior to 100
If one of these is not respected, an exception is thrown.
In the A() unit test, we will test each one of these scenarios. Each one probably in a distinct test case :
#Test
void A_throws_exception_when_one_of_params_is_not_superior_or_equal_to_0(){
...
}
#Test(expected = InvalidParamException.class);
void A_throws_exception_when_one_of_params_is_not_inferior_to_100(){
...
}
#Test(expected = InvalidParamException.class);
void A_throws_exception_when_params_sum_is_not_inferior_to_100(){
...
}
Aside the error cases, we could also multiple nominal scenarios for the A() function according to the passed parameters.
Suppose that the B() function has also multiple nominal and error scenarios.
So what about the unit test of C() that aggregates them ?
You should of course not re-test each one of these cases. It is a lot of duplication and besides it will have more combination by crossing cases of the two functions.
The next point presents how to prevent duplication.
Possible refactoring to improve the design and reduce the duplication in the unit tests of the composite function
As you write composite functions, the first thing that you should wonder is whether the composite functions should not be located in a specific component.
composite component -> unitary component(s)
Decoupling them may improve the overall design and give more specific responsibilities to components.
In addition, it provides also a natural way to reduce duplication in the unit tests of the composite component.
Indeed, you can, if required, stub/mock unitary component behaviors and don't need to create detailed fixtures for them.
Composite component unit tests can so focus on the composite component behavior.
So in our previous example, instead of testing all cases of A() and B() as we unit-testing the C() function, we could stub or mock A() and B() in order that they behavior as expected for the C() scenarios.
For example for the C() test scenario with error cases related to A() and B(), we don't need to repeat each A() or B() scenario cases :
#Test(expected = InvalidParamException.class);
void C_throws_exception_when_a_param_is_invalid(){
when(A(any,any)).thenThrow(new InvalidParamException());
C();
}
#Test(expected = InvalidParamException.class);
void C_throws_exception_when_b_param_is_invalid(){
when(B(any,any)).thenThrow(new InvalidParamException());
C();
}
Related
I've been doing some unit testing and just getting into the topic as a whole.
I stumbled upon the following scenario, suppose I have a class like this:
class A{
public B mehtod_1(B b){
b = method_2(b);
b = method_3(b);
b += 1;
return b;
}
public B method_2(B b){
// do something to B without external dependency
return B;
}
public B method_3(B b){
// do something else to B without external dependency
return B;
}
}
I can write tests for method_2 and method_3 without a problem, do different tests by configuring B in different ways and asserting the expected transformation on B after the call, those methods are atomic.
So my question is:
If I was to test method_1 in an atomic way I would have to mock the calls to method_2 and method_3 since if I would actually call these methods I would not test method_1 in an atomic manor.
In the latter case is method_2 was broken then the tests for method_1 and method_2 would break, and that would be misleading. If I'd mock the method_2 call inside the method_1 test, only the method_2 test would fail, giving a clearer indication of where the error is (namely somewhere in the business logic of method_1 given all other invoked methods worked as expected).
Did I understand the concept here correctly?
On the other hand it is correct, if both tests fail, since in the real world, method_1 cannot work without method_2 working.
My gut would say atomicity of tests is what is desired, meaning the first solution where there is one test for method_1, for every possible outcome of method_2 and method_3 (statically mocked).
Is there a "correct"/common/best practice way?
Immediate answer: in case we are talking Java here; and partial mocking is really of interest to you, you can look into using Mockito's spy concept.
But beyond that: you are getting unit testing wrong. What you call atomicity; I call worrying about implementation details. But it shouldn't matter "what exactly" that "method under test" actually does. You want to test the what, not the how.
Meaning: if that method has to call some other method(s) (that work fine in your unit test environment; without mocking); then there is no need thinking about mocking them!
You see: you care about the contract of each of your methods. That contract is what you want to test: given these input parameters, I expect that result/side effect/exception ...
Nonetheless, the fact that you have multiple public methods; and that they somehow depend on each other might be an indication of a design problem (as in: does it make sense that they are all public; is there some abstraction hiding in your interface that you should better express in other ways?). But that can only be decided given real code; real context.
I'm looking to better understand I should test functions that have many substeps or subfunctions.
Let's say I have the functions
// Modify the state of class somehow
public void DoSomething(){
DoSomethingA();
DoSomethingB();
DoSomethingC();
}
Every function here is public. Each subfunction has 2 paths. So to test every path for DoSomething() I'd have 2*2*2 = 8 tests. By writing 8 tests for DoSomething() I will have indirectly tested the subfunctions too.
So should I be testing like this, or instead write unit tests for each of the subfunctions and then only write 1 test case that measures the final state of the class after DoSomething() and ignore all the possible paths? A total of 2+2+2+1 = 7 tests. But is it bad then that the DoSomething() test case will depend on the other unit test cases to have complete coverage?
There appears to be a very prevalent religious belief that testing should be unit testing. While I do not intend to underestimate the usefulness of unit testing, I would like to point out that it is just one possible flavor of testing, and its extensive (or even exclusive) use is indicative of people (or environments) that are somewhat insecure about what they are doing.
In my experience knowledge of the inner workings of a system is useful as a hint for testing, but not as an instrument for testing. Therefore, black box testing is far more useful in most cases, though that's admittedly in part because I do not happen to be insecure about what I am doing. (And that is in turn because I use assertions extensively, so essentially all of my code is constantly testing itself.)
Without knowing the specifics of your case, I would say that in general, the fact that DoSomething() works by invoking DoSomethingA() and then DoSomethingB() and then DoSomethingC() is an implementation detail that your black-box test should best be unaware of. So, I would definitely not test that DoSomething() invokes DoSomethingA(), DoSomethingB(), and DoSomethingC(), I would only test to make sure that it returns the right results, and using the knowledge that it does in fact invoke those three functions as a hint I would implement precisely those 7 tests that you were planning to use.
On the other hand, it should be noted that if DoSomethingA() and DoSomethingB() and DoSomethingC() are also public functions, then you should also test them individually, too.
Definitely test every subfunction seperately (because they're public).
It would help you find the problem if one pops up.
If DoSomething only uses other functions, I wouldn't bother writing additional tests for it. If it has some other logic, I would test it, but assume all functions inside work properly (if they're in a different class, mock them).
The point is finding what the function does that is not covered in other tests and testing that.
Indirect testing should be avoided. You should write unit tests for each function explicitly. After that You should mock submethods and test your main function. For example :
You have a method which inserts a user to DB and method is like this :
void InsertUser(User user){
var exists = SomeExternal.UserExists(user);
if(exists)
throw new Exception("bla bla bla");
//Insert codes here
}
If you want to test InsertUser function, you should mock external/sub/nested methods and test behaviour of InsertUser function.
This example creates two tests: 1 - "When user exists then Should throw Exception" 2 - "When user does not exist then Should insert user"
if I have two classes: ClassA and ClassB.
ClassA calls a public method on ClassB i.e. in ClassA:
public function foo()
{
$this->classB->bar();
}
I have tests covering foo - but should I always write tests covering also bar in class b?
Given that the code is tested in directly im not sure if this is worthwhile or not?
The other way to look at it is in terms of the responsibility of the class.
That responsibility is what should be Asserted with tests.
Class can be ClassA or ClassB etc
Write Tests to assert responsibilities of Class A
Write Tests to assert responsibilities of Class B
Combine that with the view that there will a dependencies and object maps
You l naturally cover tests for your dependencies and then higher up the object map tracing to the root
In short
Class B needs to have tests
Class A which depends on some capability of Class B needs tests of its own
Note: the responsibility/capability of each of them will be different
Yes, you should test also bar. All your classes should have their own test case, where you test their public methods. Therefore you should have a separate test case for classB, where you test method bar().
The reason for this is that when an error occures in method bar() (or in another method of classB which bar() calls), your test will fail at method foo(), which can be confising and cause more time to find the actual error in method bar(). It may not seen to be a problem in your example, but in the real world, you test more complex code.
Another reason is the test complexity. Your tests should test your methods in all possible cases. Lets say, function bar() can be tested in 5 different cases, and also foo() can be tested in its own 5 cases. In case you would be testing only foo(), you would have to write a test to cover 5*5 = 25 cases. And if foo() were more complex, and called another method, lets say classB->bar2() with another 5 different cases, you would have to write a test that covers 5*5*5 = 125 cases! It is a lot easier to write 3 tests, echach for 5 cases, than 1 test to cover 125 cases. And those 3 tests would also be easier to read and maintain.
Yes, write the test. Otherwise, classB::bar() is not reusable (or even usable.)
If bar were a private method of ClassA, you wouldn't have to.
Should I write unit test for all nested methods or if writing one test for caller is enough?
For instance:
void Main()
{
var x = new A().AFoo();
}
public class A
{
public int AFoo()
{
// some logic
var x = new B().BFoo();
// might have some logic
return x;
}
}
public class B
{
public int BFoo()
{
// some logic
return ???;
}
}
Is that sufficient to write unit test for Main() method or I need to write tests for Main, A.AFoo(), B.BFoo() methods? How deep should I go?
Thanks in advance.
A testing purist would say that you need to create unit tests for classes A and B.
Each class should have all methods tested. If a method can do more than one thing (if you have an if statement, for example), then you should have a test for each path. If the tests are getting too complicated, it's probably a good idea to refactor the code to make the tests simpler.
Note as it stands right now, its hard to test A in isolation because it depends on B. If B is simple, as it is right now, it's probably ok. You might want to name your tests for A integration tests because technically they test both A and B together. Another option would be to have the method AFoo accept as a parameter the instance of B on which it operates. That way you could mock an instance of B and have a true unit test.
Unit tests are supposed to work on units, in the case of OOP the units are classes and the methods of the classes. That means that you should write a separate test class for each class under consideration, and at least one testing method for each method provided in the class. What is more, it is important to isolate the classes as much as possible so that a bug in class B does not cause a failure on class A. This is why Inversion of Control (Dependency Injection) is so useful, because if you can inject the instance of class B into the instance of class A, you can change B to be just a Mock object.
One of the reasons we write unit tests is to explain, in code, exactly how the methods of each class are expected to behave under all conditions, including and especially edge cases. It is hard to detail the expected behaviour of class B by writing tests on the main method.
I would recommend reading some material online explaining test driven development and how to mock objects, and perhaps use some of the excellent mocking libraries that exist such as JMock. See this question for more links.
Unit tests should help you to reduce your debugging effort. So when you just write unit tests for AFoo and none for BFoo, and one of your test fails, you probably won't know if the problem is part of class A or class B. Writing tests for BFoo too will help you to isolate the error in smaller amount of time.
Consider the following class:
public class MyIntSet
{
private List<int> _list = new List<int>();
public void Add(int num)
{
if (!_list.Contains(num))
_list.Add(num);
}
public bool Contains(int num)
{
return _list.Contains(num);
}
}
Following the "only test one thing" principle, suppose I want to test the "Add" function.
Consider the following possibility for such a test:
[TestClass]
public class MyIntSetTests
{
[TestMethod]
public void Add_AddOneNumber_SetContainsAddedNumber()
{
MyIntSet set = new MyIntSet();
int num = 0;
set.Add(num);
Assert.IsTrue(set.Contains(num));
}
}
My problem with this solution is that it actually tests 2 methods: Add() and Contains().
Theoretically, there could be a bug in both, that only manifests in scenarios where they are not called one after the other. Of course, Contains() now servers as a thin wrapper for List's Contains() which shouldn't be tested in itself, but what if it changes to something more complex in the future? Perhaps a simple "thin wrap" method should always be kept for testing purposes ?
An alternative approach might suggest mocking out or exposing (possibly using InternalsVisibleTo or PrivateObject) the private _list member and have the test inspect it directly, but that could potentially create test maintainability problems if someday the internal list is replaced by some other collection (maybe C5).
Is there a better way to do this?
Are any of my arguments against the above implementations flawed?
Thanks in advance,
JC
Your test seems perfectly OK to me. You may have misunderstood a principle of unit testing.
A single test should (ideally) only test one thing, that is true, but that does not mean that it should test only one method; rather it should only test one behaviour (an invariant, adherence to a certain business rule, etc.) .
Your test tests the behaviour "if you add to a new set, it is no longer empty", which is a single behaviour :-).
To address your other points:
Theoretically, there could be a bug in both, that only manifests in scenarios where they are not called one after the other.
True, but that just means you need more tests :-). For example, add two numbers, then call Contains, or call Contains without Add.
An alternative approach might suggest mocking out or exposing (possibly using InternalsVisibleTo) the private _list member and have the test inspect it directly, but that could potentially create test maintainability problems[...]
Very true, so don't do this. A unit test should always be against the public interface of the unit under test. That's why it's called a unit test, and not a "messing around inside a unit"-test ;-).
There are two possibilities.
You've exposed a flaw in your design. You should carefully consider if the actions that your Add method is executing is clear to the consumer. If you don't want people adding duplicates to the list, why even have a Contains() method? The user is going to be confused when it's not added to the list and no error is thrown. Even worse, they might duplicate the functionality by writing the exact same code before they call .Add() on their list collection. Perhaps it should be removed, and replaced with an indexer? It's not clear from your list class that it's not meant to hold duplicates.
The design is fine, and your public methods should rely on each other. This is normal, and there is no reason you can't test both methods. The more test cases you have, theoretically the better.
As an example, say you have a functions that just calls down into other layers, which may already be unit tested. That doesn't mean you don't write unit tests for the function even if it's simply a wrapper.
In practice, your current test is fine. For something this simple it's very unlikely that bugs in add() and contains() would mutually conspire to hide each other. In cases where you are really concerned about testing add() and add() alone, one solution is to make your _list variable available to your unit test code.
[TestClass]
public void Add_AddOneNumber_SetContainsAddedNumber() {
MyIntSet set = new MyIntSet();
set.add(0);
Assert.IsTrue(set._list.Contains(0));
}
Doing this has two drawbacks. One: it requires access to the private _list variable, which is a little complex in C# (I recommend the reflection technique). Two: it makes your test code dependent on the actual implementation of your Set implementation, which means you'll have to modify the test if you ever change the implementation. I'd never do this for something as simple as a collections class, but in some cases it may be useful.