Suppose we have a function add() as below:
void add(int a, int b) {
int sum=a+b;
cout<<sum;
sendSumToStorage(sum);
}
This simple function adds to input values, prints the sum to the console and also sends it to some external storage (say, a file). This is how we ideally want it in the application (meaning, we don't want it to return anything).
For purposes of unit testing, is it valid (from a design perspective) if we modify the function signature so that it returns the sum? We could then have a test like:
bool checkAdd() {
int res=add(3, 4);
if(res==7) return true;
else return false;
}
Better yet, is this (returning a value) the only way we could unit test it? Is there some valid way in which we could unit test the add() function without changing the function signature?
A function like the one in your example is considered really bad practice.
Why do I say this?
Well, you have a method called add which adds two numbers AND calls something else. Basically your method doesn't do one thing, but two, which violates the Single Responsibility Principle.
This makes things much harder to test because you can't test just the add method in isolation.
So you would separate that into two methods with good names which reflect what they do and test them separately.
If you don't want to have issues with state between your methods, you will have to start returning results where it makes sense.
Ignoring the fact that this example has a bad desing.
For cases like this, when you want to check some internal behaviour instead of API you should rather try using some testing libraries like gtest and gmock.
It allows you to describe more sophisticated expectations than just function result.
For example you can set expectation that some method will be called during code execution using EXPECT_CALL macro.
More details here:
https://github.com/google/googletest/blob/master/googlemock/docs/ForDummies.md
Answering your question, it's always a bad practice to modify any part of tested code for the purpose of testing. In that case you are not longer testing production code. As it was suggested before it's better to split the functionality into smaller parts and test them isolated.
Changing the design of code to improve testability is very common and generally considered as a good practice. Obviously, not all such changes are necessarily real improvements - sometimes better solutions exist.
In your case, the code is difficult to test because it combines computations (the addition) with interactions with depended-on components (output and storing data). In your case (as Andrei has pointed out) the function also violates the SRP, but mixing computations and interactions generally makes testing more difficult, even in cases where the SRP is not violated.
I understand, you would change the function such that it will return the computed value in addition to printing it and writing it to the storage. This will allow you to test the computational part of the function. The function will then, however, only be partially tested. And, the purpose of the function will be further obfuscated.
If instead you split the function into one function doing the computation and one doing the interactions, you can thoroughly test the first with unit-testing, and use integration-testing for the other. Again, the usefulness of this approach is independent of whether the code violates the SRP or not.
Related
Let's say we have function foo. foo checks, before its actual code runs, a couple of initial conditions, which are laid out as consecutive if-conditions. If an error occurs, the user is informed with an alert/toast. The ifs are laid out in the following manner:
function foo() {
if (!condition_one) {
alert(text_one);
return;
}
if (!condition_two) {
alert(text_two);
return;
}
if (!condition_three) {
alert(text_three);
return;
}
if (!condition_four) {
alert(text_four);
return;
}
// ...
}
Before writing this function, we write our first unit test, in accordance with TDD principles. This first test checks the case that condition_one fails. The second test checks that condition_one succeeds, i.e., text_one is not alerted.
We now copy the first test case and transform it, such that it become the test that checking that condition_two fails. We have to extend the first test case for this, since we need to have the first condition succeed to even get to the second condition. We now write the succeeding test for condition_two and repeat the process until we have tested all initial conditions.
The issue is, that each time we go to the next condition, all setup code of all previous conditions is accumulated and the actual setup code for this condition gets lost in the clutter and it is hard to know what we are even testing.
What are possible solutions to this problem? I understand that this is exactly what Aspect-Oriented-Programming is trying to remedy but that is not an option for me.
PS: this issue also arises in other large if-else structures and is thus more widely applicable than this specific scenario.
Design is what we do to get more of what we want than we would get by just doing it.
The answer is that we need to think about the "what we want" for our tests, and then make that happen.
In this case, one of the "what you wants" is the ability to distinguish the important detail of the test from the background noise. That's usually achieved by moving the background noise out of the body of the test.
In the abstract, your third test has roughly this shape
assume condition_one
assume not condition_two
foo()
assert alert(text_two)
Implicit in this description is the fact that we don't care about anything other than condition_one and condition_two.
So in code, that could look like
InitialConditions.any()
InitialConditions.condition_one(true)
InitialConditions.condition_two(false)
foo()
assert alert(text_two)
As you got along, Given (whatever that is) is getting more complicated to allow you to express more precisely the assumptions of each test - but the prose that is each test is still basically linear with the complexity of the details, not the complexity of the whole.
A good introductory read would be Nat Pryce on Test Data Builders (2007).
As far as testing is concerned, every functions couldn't properly be tested if they don't follow Single Responsibility Principle (SRP). As can be seen in foo function, there are too many conditions and as a result of it, there are some reasons to change this method. In this sort of methods, writing test is too hard and sometime it would be impossible.
I urge that refactoring this method for sticking to SRP, then writing test would be a piece of cake.
I am wondering if I should unit test functions that have no control flow. This functions take some input, call a sequence of 5/6 other functions, then return some output.
Testing them seems a waste of time, since I don't see what I would be testing exactly. The other functions called already have unit test for them.
The main drawback for me is that I don't know what the output should be a priori, I would need to call the same functions in the test scripit to see if the results coincide; and then what am I testing? That the test function and the actual function have the same lines in the same order?
Thanks for any insight
Note: Same as last question, if you think it's primarily opinion based reformulate as "According to the principles advocated in Art of Unit Testing, should i unit test functions with no control flow?"
Short answer: yes, of course you do!
Long answer: how a method does something is in the end "implementation" detail. In that sense: you should not care at all if a method is using a switch, some if/elses, a loop, or just calls other methods in sequence.
Instead, you should understand the contract that your method provides: which input it takes; and what comes out of it (depending on the inputs maybe).
That is what you focus on: creating a setup where your method can run; to then check if the method upholds that contract.
Example:
public void foo(Bar bar) {
FooBar fooBar = bar.wobbel();
fooBar.throttle();
fooBar.rattle(this.someField);
}
that code above doesn't contain any control flow statements. But still, there are various points in there where things could go wrong (for example NullPointerExceptions). Don't you think it would be better to catch those using unit tests?
I'm new to test driven development and first time I'm tring to use it in a simple project.
I have a class, and I need to test creation, insertion and deletion of objects of this class. If I write three seperate test functions I need to duplicate initialization codes in other function. On the hand if I put all tests in one test function then it is a contradiction with one test per function. What should I do?
Here the situation:
tst_create()
{
createHead(head);
createBody(body);
createFoot(foot);
}
tst_insert()
{
createHead(head);
createBody(body);
createFoot(foot);
obj_id=insert(obj); //Also I need to delete obj_id somehow in order to preserve old state
}
tst_delete()
{
createHead(head);
createBody(body);
createFoot(foot);
obj_id=insert(obj);
delete(obj_id);
}
vs
tstCreateInsertDelete()
{
createHead(head);
createBody(body);
createFoot(foot);
obj_id=insert(obj);
delete(obj_id);
}
Rather than "One test per function", try thinking about it as, "One aspect of behaviour per function".
What does inserting an object give you? How about deleting an object? Why are these valuable? How can you tell you've done them? Write an example of how the code might be used, and why that behaviour is valuable. That then becomes your test.
When you've worked out what the behaviour is that you're interested in, extract out the duplication only if it makes the test more readable. TDD isn't just about testing; it's also about providing documentation, and helping you think about the responsibility of each element of code and the design of that code. The tests will probably be read far more than they're written, so readability has to come first.
If necessary, put all the behaviour you're interested in in one method, and just make sure it's readable. You can add comments if required.
Factor out the duplication in your tests.
Depending on your test framework, there may be support for defining a setup method that's called before each test execution and a teardown method that's called after each test.
Regardless, you can extract the common stuff so that you only have to repeat a call to a single shared setup.
If you tell us what language and test framework you use, we might be able to give more specific advice.
For example this article introduces them.
What is the benefit?
Static analysis seems cool but at the same time it would prevent the ability to pass null as a parameter in unit test. (if you followed the example in the article that is)
While on the topic of unit testing - given how things are now surely there is no point for code contracts if you already practice automated testing?
Update
Having played with Code Contracts I'm a little disappointed. For example, based on the code in the accepted answer:
public double CalculateTotal(Order order)
{
Contract.Requires(order != null);
Contract.Ensures(Contract.Result<double>() >= 0);
return 2.0;
}
For unit testing, you still have to write tests to ensure that null cannot be passed, and the result is greater than or equal to zero if the contracts are business logic. In other words, if I was to remove the first contract, no tests would break, unless I had specifically had a test for this feature. This is based on not using the static analysis built into the better (ultimate etc...) editions of Visual Studio however.
Essentially they all boil down to an alternate way of writing traditional if statements. My experience actually using TDD, with Code Contracts shows why, and how I went about it.
I don't think unit testing and contracts interfere with each other that much, and if anything contracts should help unit testing since it removes the need to add tedious repetitive tests for invalid arguments. Contracts specify the minimum you can expect from the function, whereas unit tests attempt to validate the actual behaviour for a particular set of inputs. Consider this contrived example:
public class Order
{
public IEnumerable Items { get; }
}
public class OrderCalculator
{
public double CalculateTotal(Order order)
{
Contract.Requires(order != null);
Contract.Ensures(Contract.Result<double>() >= 0);
return 2.0;
}
}
Clearly the code satisfies the contract, but you'd still need unit testing to validate it actually behaves as you'd expect.
What is the benefit?
Let's say that you want to make sure that a method never returns null. Now with unit tests, you have to write a bunch of test cases where you call the method with varying inputs and verify that the output is not null. Trouble is, you can't test all possible inputs.
With code contracts, you just declare that the method never returns null. The static analyzer will then complain if it is not possible to prove that. If it doesn't complain, you know that your assertion is correct for all possible inputs.
Less work, perfect correctness guarantees. What's not to like?
Contracts allow you say what the actual purpose of the code is, as opposed to letting whatever the code does with whatever random arguments are handed it standing as the definition from the point of view of the compiler, or the next reader of the code. This allows significantly better static analysis and code optimization.
For instance, if I declare an integer parameter (using the contract notation) to be in the range of 1 to 10, and I have a local array in my function declared the same size, that is indexed by the parameter, the compiler can tell that there is no possibility of subscript error, thus producing better code.
You can state that null is valid value in a contract.
The purpose of unit testing is to verify dynamically that the code achieves whatever stated purpose it has. Just because you've written a contract for a function, doesn't mean the code does that, or that static analysis can verify the code does that. Unit testing won't go away.
Well it will not interfere with unit-testing in general. But as I saw you mentioned something about TDD.
If I think about it from that perspective I guess it could/may change the procedure from the standard one
create method (just signature)
create Unit test -> implement the test
run the test: let it fail
implement the method, hack it to the end just to make it working
run the test: see it pass
refactor your (possibly messy) method body
(re-run the test just to see you've not broken anything)
This would be the really hard-full-featured unit-testing procedure. In such a context I guess you could insert code contracts between the 1st and 2nd point like
create method (just signature)
insert code contracts for the methods input parameters
create Unit test -> implement the test
...
The advantage I see at the moment is that you can write easier unit tests in the sense that you wouldn't have to check every possible path since some is already taken into account by your defined contracts. It just gives you additional checking, but it wouldn't replace unit testing since there will always be more logic within the code, more path that have to be tested with unit tests as usual.
Edit
Another possibility I didn't consider before would be to add the code contracts in the refactoring part. Basically as additional way of assuring things. But that would somehow be redundant and since people don't like to do redundant stuff...
Suppose you have a method:
public void Save(Entity data)
{
this.repositoryIocInstance.EntitySave(data);
}
Would you write a unit test at all?
public void TestSave()
{
// arrange
Mock<EntityRepository> repo = new Mock<EntityRepository>();
repo.Setup(m => m.EntitySave(It.IsAny<Entity>());
// act
MyClass c = new MyClass(repo.Object);
c.Save(new Entity());
// assert
repo.Verify(m => EntitySave(It.IsAny<Entity>()), Times.Once());
}
Because later on if you do change method's implementation to do more "complex" stuff like:
public void Save(Entity data)
{
if (this.repositoryIocInstance.Exists(data))
{
this.repositoryIocInstance.Update(data);
}
else
{
this.repositoryIocInstance.Create(data);
}
}
...your unit test would fail but it probably wouldn't break your application...
Question
Should I even bother creating unit tests on methods that don't have any return types* or **don't change anything outside of internal mock?
Don't forget that unit tests isn't just about testing code. It's about allowing you to determine when behaviour changes.
So you may have something that's trivial. However, your implementation changes and you may have a side effect. You want your regression test suite to tell you.
e.g. Often people say you shouldn't test setters/getters since they're trivial. I disagree, not because they're complicated methods, but someone may inadvertently change them through ignorance, fat-finger scenarios etc.
Given all that I've just said, I would definitely implement tests for the above (via mocking, and/or perhaps it's worth designing your classes with testability in mind and having them report status etc.)
It's true your test is depending on your implementation, which is something you should avoid (though it is not really that simple sometimes...) and is not necessarily bad. But these kind of tests are expected to break even if your change doesn't break the code.
You could have many approaches to this:
Create a test that really goes to the database and check if the state was changed as expected (it won't be a unit test anymore)
Create a test object that fakes a database and do operations in-memory (another implementation for your repositoryIocInstance), and verify the state was changed as expected. Changes to the repository interface would incurr in changes to this object as well. But your interfaces shouldn't be changing much, right?
See all of this as too expensive, and use your approach, which may incur on unnecessarily breaking tests later (but once the chance is low, it is ok to take the risk)
Ask yourself two questions. "What is the manual equivalent of this unit test?" and "is it worth automating?". In your case it would be something like:
What is manual equivalent?
- start debugger
- step into "Save" method
- step into next, make sure you're inside IRepository.EntitySave implementation
Is it worth automating? My answer is "no". It is 100% obvious from the code.
From hundreds of similar waste tests I didn't see a single which would turn out to be useful.
The general rule of thumb is, that you test all things, that could probably break. If you are sure, that the method is simple enough (and stays simple enough) to not be a problem, that let it out with testing.
The second thing is, you should test the contract of the method, not the implementation. If the test fails after a change, but not the application, then your test tests not the right thing. The test should cover cases that are important for your application. This should ensure, that every change to the method that doesn't break the application also don't fail the test.
A method that does not return any result still changes the state of your application. Your unit test, in this case, should be testing whether the new state is as intended.
"your unit test would fail but it probably wouldn't break your application"
This is -- actually -- really important to know. It may seem annoying and trivial, but when someone else starts maintaining your code, they may have made a really bad change to Save and (improbably) broken the application.
The trick is to prioritize.
Test the important stuff first. When things are slow, add tests for trivial stuff.
When there isn't an assertion in a method, you are essentially asserting that exceptions aren't thrown.
I'm also struggling with the question of how to test public void myMethod(). I guess if you do decide to add a return value for testability, the return value should represent all salient facts necessary to see what changed about the state of the application.
public void myMethod()
becomes
public ComplexObject myMethod() {
DoLotsOfSideEffects()
return new ComplexObject { rows changed, primary key, value of each column, etc };
}
and not
public bool myMethod()
DoLotsOfSideEffects()
return true;
The short answer to your question is: Yes, you should definitely test methods like that.
I assume that it is important that the Save method actually saves the data. If you don't write a unit test for this, then how do you know?
Someone else may come along and remove that line of code that invokes the EntitySave method, and none of the unit tests will fail. Later on, you are wondering why items are never persisted...
In your method, you could say that anyone deleting that line would only be doing so if they have malign intentions, but the thing is: Simple things don't necessarily stay simple, and you better write the unit tests before things get complicated.
It is not an implementation detail that the Save method invokes EntitySave on the Repository - it is part of the expected behavior, and a pretty crucial part, if I may say so. You want to make sure that data is actually being saved.
Just because a method does not return a value doesn't mean that it isn't worth testing. In general, if you observe good Command/Query Separation (CQS), any void method should be expected to change the state of something.
Sometimes that something is the class itself, but other times, it may be the state of something else. In this case, it changes the state of the Repository, and that is what you should be testing.
This is called testing Inderect Outputs, instead of the more normal Direct Outputs (return values).
The trick is to write unit tests so that they don't break too often. When using Mocks, it is easy to accidentally write Overspecified Tests, which is why most Dynamic Mocks (like Moq) defaults to Stub mode, where it doesn't really matter how many times you invoke a given method.
All this, and much more, is explained in the excellent xUnit Test Patterns.