I am wondering if I should unit test functions that have no control flow. This functions take some input, call a sequence of 5/6 other functions, then return some output.
Testing them seems a waste of time, since I don't see what I would be testing exactly. The other functions called already have unit test for them.
The main drawback for me is that I don't know what the output should be a priori, I would need to call the same functions in the test scripit to see if the results coincide; and then what am I testing? That the test function and the actual function have the same lines in the same order?
Thanks for any insight
Note: Same as last question, if you think it's primarily opinion based reformulate as "According to the principles advocated in Art of Unit Testing, should i unit test functions with no control flow?"
Short answer: yes, of course you do!
Long answer: how a method does something is in the end "implementation" detail. In that sense: you should not care at all if a method is using a switch, some if/elses, a loop, or just calls other methods in sequence.
Instead, you should understand the contract that your method provides: which input it takes; and what comes out of it (depending on the inputs maybe).
That is what you focus on: creating a setup where your method can run; to then check if the method upholds that contract.
Example:
public void foo(Bar bar) {
FooBar fooBar = bar.wobbel();
fooBar.throttle();
fooBar.rattle(this.someField);
}
that code above doesn't contain any control flow statements. But still, there are various points in there where things could go wrong (for example NullPointerExceptions). Don't you think it would be better to catch those using unit tests?
Related
Suppose we have a function add() as below:
void add(int a, int b) {
int sum=a+b;
cout<<sum;
sendSumToStorage(sum);
}
This simple function adds to input values, prints the sum to the console and also sends it to some external storage (say, a file). This is how we ideally want it in the application (meaning, we don't want it to return anything).
For purposes of unit testing, is it valid (from a design perspective) if we modify the function signature so that it returns the sum? We could then have a test like:
bool checkAdd() {
int res=add(3, 4);
if(res==7) return true;
else return false;
}
Better yet, is this (returning a value) the only way we could unit test it? Is there some valid way in which we could unit test the add() function without changing the function signature?
A function like the one in your example is considered really bad practice.
Why do I say this?
Well, you have a method called add which adds two numbers AND calls something else. Basically your method doesn't do one thing, but two, which violates the Single Responsibility Principle.
This makes things much harder to test because you can't test just the add method in isolation.
So you would separate that into two methods with good names which reflect what they do and test them separately.
If you don't want to have issues with state between your methods, you will have to start returning results where it makes sense.
Ignoring the fact that this example has a bad desing.
For cases like this, when you want to check some internal behaviour instead of API you should rather try using some testing libraries like gtest and gmock.
It allows you to describe more sophisticated expectations than just function result.
For example you can set expectation that some method will be called during code execution using EXPECT_CALL macro.
More details here:
https://github.com/google/googletest/blob/master/googlemock/docs/ForDummies.md
Answering your question, it's always a bad practice to modify any part of tested code for the purpose of testing. In that case you are not longer testing production code. As it was suggested before it's better to split the functionality into smaller parts and test them isolated.
Changing the design of code to improve testability is very common and generally considered as a good practice. Obviously, not all such changes are necessarily real improvements - sometimes better solutions exist.
In your case, the code is difficult to test because it combines computations (the addition) with interactions with depended-on components (output and storing data). In your case (as Andrei has pointed out) the function also violates the SRP, but mixing computations and interactions generally makes testing more difficult, even in cases where the SRP is not violated.
I understand, you would change the function such that it will return the computed value in addition to printing it and writing it to the storage. This will allow you to test the computational part of the function. The function will then, however, only be partially tested. And, the purpose of the function will be further obfuscated.
If instead you split the function into one function doing the computation and one doing the interactions, you can thoroughly test the first with unit-testing, and use integration-testing for the other. Again, the usefulness of this approach is independent of whether the code violates the SRP or not.
I'm looking to better understand I should test functions that have many substeps or subfunctions.
Let's say I have the functions
// Modify the state of class somehow
public void DoSomething(){
DoSomethingA();
DoSomethingB();
DoSomethingC();
}
Every function here is public. Each subfunction has 2 paths. So to test every path for DoSomething() I'd have 2*2*2 = 8 tests. By writing 8 tests for DoSomething() I will have indirectly tested the subfunctions too.
So should I be testing like this, or instead write unit tests for each of the subfunctions and then only write 1 test case that measures the final state of the class after DoSomething() and ignore all the possible paths? A total of 2+2+2+1 = 7 tests. But is it bad then that the DoSomething() test case will depend on the other unit test cases to have complete coverage?
There appears to be a very prevalent religious belief that testing should be unit testing. While I do not intend to underestimate the usefulness of unit testing, I would like to point out that it is just one possible flavor of testing, and its extensive (or even exclusive) use is indicative of people (or environments) that are somewhat insecure about what they are doing.
In my experience knowledge of the inner workings of a system is useful as a hint for testing, but not as an instrument for testing. Therefore, black box testing is far more useful in most cases, though that's admittedly in part because I do not happen to be insecure about what I am doing. (And that is in turn because I use assertions extensively, so essentially all of my code is constantly testing itself.)
Without knowing the specifics of your case, I would say that in general, the fact that DoSomething() works by invoking DoSomethingA() and then DoSomethingB() and then DoSomethingC() is an implementation detail that your black-box test should best be unaware of. So, I would definitely not test that DoSomething() invokes DoSomethingA(), DoSomethingB(), and DoSomethingC(), I would only test to make sure that it returns the right results, and using the knowledge that it does in fact invoke those three functions as a hint I would implement precisely those 7 tests that you were planning to use.
On the other hand, it should be noted that if DoSomethingA() and DoSomethingB() and DoSomethingC() are also public functions, then you should also test them individually, too.
Definitely test every subfunction seperately (because they're public).
It would help you find the problem if one pops up.
If DoSomething only uses other functions, I wouldn't bother writing additional tests for it. If it has some other logic, I would test it, but assume all functions inside work properly (if they're in a different class, mock them).
The point is finding what the function does that is not covered in other tests and testing that.
Indirect testing should be avoided. You should write unit tests for each function explicitly. After that You should mock submethods and test your main function. For example :
You have a method which inserts a user to DB and method is like this :
void InsertUser(User user){
var exists = SomeExternal.UserExists(user);
if(exists)
throw new Exception("bla bla bla");
//Insert codes here
}
If you want to test InsertUser function, you should mock external/sub/nested methods and test behaviour of InsertUser function.
This example creates two tests: 1 - "When user exists then Should throw Exception" 2 - "When user does not exist then Should insert user"
I'm new to test driven development and first time I'm tring to use it in a simple project.
I have a class, and I need to test creation, insertion and deletion of objects of this class. If I write three seperate test functions I need to duplicate initialization codes in other function. On the hand if I put all tests in one test function then it is a contradiction with one test per function. What should I do?
Here the situation:
tst_create()
{
createHead(head);
createBody(body);
createFoot(foot);
}
tst_insert()
{
createHead(head);
createBody(body);
createFoot(foot);
obj_id=insert(obj); //Also I need to delete obj_id somehow in order to preserve old state
}
tst_delete()
{
createHead(head);
createBody(body);
createFoot(foot);
obj_id=insert(obj);
delete(obj_id);
}
vs
tstCreateInsertDelete()
{
createHead(head);
createBody(body);
createFoot(foot);
obj_id=insert(obj);
delete(obj_id);
}
Rather than "One test per function", try thinking about it as, "One aspect of behaviour per function".
What does inserting an object give you? How about deleting an object? Why are these valuable? How can you tell you've done them? Write an example of how the code might be used, and why that behaviour is valuable. That then becomes your test.
When you've worked out what the behaviour is that you're interested in, extract out the duplication only if it makes the test more readable. TDD isn't just about testing; it's also about providing documentation, and helping you think about the responsibility of each element of code and the design of that code. The tests will probably be read far more than they're written, so readability has to come first.
If necessary, put all the behaviour you're interested in in one method, and just make sure it's readable. You can add comments if required.
Factor out the duplication in your tests.
Depending on your test framework, there may be support for defining a setup method that's called before each test execution and a teardown method that's called after each test.
Regardless, you can extract the common stuff so that you only have to repeat a call to a single shared setup.
If you tell us what language and test framework you use, we might be able to give more specific advice.
I have a method that checks some assumptions and either follows the happy path, or terminates along the unhappy paths. I've either designed it poorly, or I'm missing the method for testing that the control of the flow.
if (this.officeInfo.OfficeClosed)
{
this.phoneCall.InformCallerThatOfficeIsClosedAndHangUp();
return;
}
if (!this.operators.GetAllOperators().Any())
{
this.phoneCall.InformCallerThatNoOneIsAvailableAndSendToVoicemail();
return;
}
Call call=null;
forach(var operator in this.operators.GetAllOperators())
{
call = operator.Call();
if(call!=null) {break;}
}
and so on. I've got my dependencies injected. I've got my mocks moq'd. I can make sure that this or that is called, but I don't know how to test that the "return" happens. If TDD means I don't write a line until I have a test that fails without it, I'm stuck.
How would you test it? Or is there a way to write it that makes it more testable?
Update: Several answers have been posted saying that I should test the resultant calls, not the flow control. The problem I have with this approach, is that every test is required to setup and test the state and results of the other tests. This seems really unwieldy and brittle. Shouldn't I be able to test the first if clause alone, and then test the second one alone? Do I really need to have a logarithmically expanding set of tests that start looking like Method_WithParameter_DoesntInvokeMethod8IfMethod7IsTrueandMethod6IsTrueAndMethod5IsTrueAndMethod4IsTrueAndMethod3IsFalseAndMethod2IsTrueAndMethod1isAaaaccck()?
I think you want to test the program's outputs: for example, that when this.officeInfo.OfficeClosed then the program does invoke this.phoneCall.InformCallerThatOfficeIsClosedAndHangUp() and does not invoke other methods such as this.operators.GetAllOperators().
I think that your test does this by asking its mock objects (phoneCall, etc.) which of their methods was invoked, or by getting them to throw an exception if any of their methods are invoked unexpectedly.
One way to do it is to make a log file of the program's inputs (e.g. 'OfficeClosed returns true') and outputs: then run the test, let the test generate the log file, and then assert that the contents of the generated log file match the expected log file contents for that test.
I'm not sure that's really the right approach. You care about whether or not the method produced the expected result, not necessarily how control "flowed" through the particular method. For example, if phoneCall.InformCallerThatOfficeIsClosedAndHangUp is called, then I assume some result is recorded somewhere. So in your unit test, you would be asserting that result was indeed recorded (either by checking a database record, file, etc.).
With that said, it's important to ensure that your unit tests indeed cover your code. For that, you can use a tool like NCover to ensure that all of your code is being excercised. It'll generate a coverage report which will show you exactly which lines were executed by your unit tests and more importantly, which ones weren't.
You could go ballistic and use a strategy pattern. Something along the lines of having an interface IHandleCall, with a single void method DoTheRightThing(), and 3 classes HandleOfficeIsClosed, HandleEveryoneIsBusy, HandleGiveFirstOperatorAvailable, which implement the interface. And then have code like:
IHandleCall handleCall;
if (this.officeInfo.OfficeClosed)
{
handleCall = new HandleOfficeIsClosed();
}
else if other condition
{
handleCall = new OtherImplementation();
}
handleCall.DoTheRightThing();
return;
That way you can get rid of the multiple return points in your method. Note that this is a very dirty outline, but essentially at that point you should extract the if/else into some factory, and then the only thing you have to test is that your class calls the factory, and that handleCall.DoTheRightThing() is called - (and of course that the factory returns the right strategy).
In any case, because you have already guarded against no operator available, you could simplify the end to:
var operator = this.operators.FindFirst();
call = operator.Call();
Don't test the flow control, just test the expected behavior. That is, unit testing does not care about the implementation details, only that the behavior of the method matches the specifications of the method. So if Add(int x, int y) should produce the result 4 on input x = 2, y = 2, then test that the output is 4 but don't worry about how Add produced the result.
To put it another way, unit testing should be invariant under implementation details and refactoring. But if you're testing implementation details in your unit testing, then you can't refactor without breaking the unit tests. For example, if you implement a method GetPrime(int k) to return the kth prime then check that GetPrime(10) returns 29 but don't test the flow control inside the method. If you implement GetPrime using the Sieve of Eratóstenes and have tested the flow control inside the method and later refactor to use the Sieve of Atkin, your unit tests will break. Again, all that matters is that GetPrime(10) returns 29, not how it does it.
If you are stuck using TDD it's a good thing: it means that TDD drives your design and you are looking into how to change it so you can test it.
You can either:
1) verify state: check SUT state after SUT execution or
2) verify behavior: check that mock object calls complied with test expectations
If you don't like how either of these approaches look in your test it's time to refactor the code.
The pattern described by Aaron Feng and K. Scott Allen would solve for my problem and it's testability concerns. The only issue I see is that it requires all the computation to be performed up front. The decision data object needs to be populated before all of the conditionals. That's great unless it requires successive round trips to the persistence storage.
For example this article introduces them.
What is the benefit?
Static analysis seems cool but at the same time it would prevent the ability to pass null as a parameter in unit test. (if you followed the example in the article that is)
While on the topic of unit testing - given how things are now surely there is no point for code contracts if you already practice automated testing?
Update
Having played with Code Contracts I'm a little disappointed. For example, based on the code in the accepted answer:
public double CalculateTotal(Order order)
{
Contract.Requires(order != null);
Contract.Ensures(Contract.Result<double>() >= 0);
return 2.0;
}
For unit testing, you still have to write tests to ensure that null cannot be passed, and the result is greater than or equal to zero if the contracts are business logic. In other words, if I was to remove the first contract, no tests would break, unless I had specifically had a test for this feature. This is based on not using the static analysis built into the better (ultimate etc...) editions of Visual Studio however.
Essentially they all boil down to an alternate way of writing traditional if statements. My experience actually using TDD, with Code Contracts shows why, and how I went about it.
I don't think unit testing and contracts interfere with each other that much, and if anything contracts should help unit testing since it removes the need to add tedious repetitive tests for invalid arguments. Contracts specify the minimum you can expect from the function, whereas unit tests attempt to validate the actual behaviour for a particular set of inputs. Consider this contrived example:
public class Order
{
public IEnumerable Items { get; }
}
public class OrderCalculator
{
public double CalculateTotal(Order order)
{
Contract.Requires(order != null);
Contract.Ensures(Contract.Result<double>() >= 0);
return 2.0;
}
}
Clearly the code satisfies the contract, but you'd still need unit testing to validate it actually behaves as you'd expect.
What is the benefit?
Let's say that you want to make sure that a method never returns null. Now with unit tests, you have to write a bunch of test cases where you call the method with varying inputs and verify that the output is not null. Trouble is, you can't test all possible inputs.
With code contracts, you just declare that the method never returns null. The static analyzer will then complain if it is not possible to prove that. If it doesn't complain, you know that your assertion is correct for all possible inputs.
Less work, perfect correctness guarantees. What's not to like?
Contracts allow you say what the actual purpose of the code is, as opposed to letting whatever the code does with whatever random arguments are handed it standing as the definition from the point of view of the compiler, or the next reader of the code. This allows significantly better static analysis and code optimization.
For instance, if I declare an integer parameter (using the contract notation) to be in the range of 1 to 10, and I have a local array in my function declared the same size, that is indexed by the parameter, the compiler can tell that there is no possibility of subscript error, thus producing better code.
You can state that null is valid value in a contract.
The purpose of unit testing is to verify dynamically that the code achieves whatever stated purpose it has. Just because you've written a contract for a function, doesn't mean the code does that, or that static analysis can verify the code does that. Unit testing won't go away.
Well it will not interfere with unit-testing in general. But as I saw you mentioned something about TDD.
If I think about it from that perspective I guess it could/may change the procedure from the standard one
create method (just signature)
create Unit test -> implement the test
run the test: let it fail
implement the method, hack it to the end just to make it working
run the test: see it pass
refactor your (possibly messy) method body
(re-run the test just to see you've not broken anything)
This would be the really hard-full-featured unit-testing procedure. In such a context I guess you could insert code contracts between the 1st and 2nd point like
create method (just signature)
insert code contracts for the methods input parameters
create Unit test -> implement the test
...
The advantage I see at the moment is that you can write easier unit tests in the sense that you wouldn't have to check every possible path since some is already taken into account by your defined contracts. It just gives you additional checking, but it wouldn't replace unit testing since there will always be more logic within the code, more path that have to be tested with unit tests as usual.
Edit
Another possibility I didn't consider before would be to add the code contracts in the refactoring part. Basically as additional way of assuring things. But that would somehow be redundant and since people don't like to do redundant stuff...