Consider the following class:
public class MyIntSet
{
private List<int> _list = new List<int>();
public void Add(int num)
{
if (!_list.Contains(num))
_list.Add(num);
}
public bool Contains(int num)
{
return _list.Contains(num);
}
}
Following the "only test one thing" principle, suppose I want to test the "Add" function.
Consider the following possibility for such a test:
[TestClass]
public class MyIntSetTests
{
[TestMethod]
public void Add_AddOneNumber_SetContainsAddedNumber()
{
MyIntSet set = new MyIntSet();
int num = 0;
set.Add(num);
Assert.IsTrue(set.Contains(num));
}
}
My problem with this solution is that it actually tests 2 methods: Add() and Contains().
Theoretically, there could be a bug in both, that only manifests in scenarios where they are not called one after the other. Of course, Contains() now servers as a thin wrapper for List's Contains() which shouldn't be tested in itself, but what if it changes to something more complex in the future? Perhaps a simple "thin wrap" method should always be kept for testing purposes ?
An alternative approach might suggest mocking out or exposing (possibly using InternalsVisibleTo or PrivateObject) the private _list member and have the test inspect it directly, but that could potentially create test maintainability problems if someday the internal list is replaced by some other collection (maybe C5).
Is there a better way to do this?
Are any of my arguments against the above implementations flawed?
Thanks in advance,
JC
Your test seems perfectly OK to me. You may have misunderstood a principle of unit testing.
A single test should (ideally) only test one thing, that is true, but that does not mean that it should test only one method; rather it should only test one behaviour (an invariant, adherence to a certain business rule, etc.) .
Your test tests the behaviour "if you add to a new set, it is no longer empty", which is a single behaviour :-).
To address your other points:
Theoretically, there could be a bug in both, that only manifests in scenarios where they are not called one after the other.
True, but that just means you need more tests :-). For example, add two numbers, then call Contains, or call Contains without Add.
An alternative approach might suggest mocking out or exposing (possibly using InternalsVisibleTo) the private _list member and have the test inspect it directly, but that could potentially create test maintainability problems[...]
Very true, so don't do this. A unit test should always be against the public interface of the unit under test. That's why it's called a unit test, and not a "messing around inside a unit"-test ;-).
There are two possibilities.
You've exposed a flaw in your design. You should carefully consider if the actions that your Add method is executing is clear to the consumer. If you don't want people adding duplicates to the list, why even have a Contains() method? The user is going to be confused when it's not added to the list and no error is thrown. Even worse, they might duplicate the functionality by writing the exact same code before they call .Add() on their list collection. Perhaps it should be removed, and replaced with an indexer? It's not clear from your list class that it's not meant to hold duplicates.
The design is fine, and your public methods should rely on each other. This is normal, and there is no reason you can't test both methods. The more test cases you have, theoretically the better.
As an example, say you have a functions that just calls down into other layers, which may already be unit tested. That doesn't mean you don't write unit tests for the function even if it's simply a wrapper.
In practice, your current test is fine. For something this simple it's very unlikely that bugs in add() and contains() would mutually conspire to hide each other. In cases where you are really concerned about testing add() and add() alone, one solution is to make your _list variable available to your unit test code.
[TestClass]
public void Add_AddOneNumber_SetContainsAddedNumber() {
MyIntSet set = new MyIntSet();
set.add(0);
Assert.IsTrue(set._list.Contains(0));
}
Doing this has two drawbacks. One: it requires access to the private _list variable, which is a little complex in C# (I recommend the reflection technique). Two: it makes your test code dependent on the actual implementation of your Set implementation, which means you'll have to modify the test if you ever change the implementation. I'd never do this for something as simple as a collections class, but in some cases it may be useful.
Related
When I'm writing some typescript code, I wanted to unit test some private method of Foo Class.
class Foo {
private privateMethodWantToTest(argument) {
return 'Bar'
}
}
So, I separated function like this:
export function privateMethodWantToTest(this: Foo, argument) {
return 'Bar'
}
class Foo {
private privateMethodWantToTest(argument) {
return privateMethodWantToTest.call(this, argument)
}
}
This approach may work, but It's a bit redundant.
And exporting all function may be a bad idea.
What do you think about this approach ?
Is there better solution for unit test, or should't I test private method ?
It depends. First, testing is about finding (or preventing) bugs. Bugs are in the implementation details: Different implementations have different potential bugs. For example, think about the different ways to implement a Fibonacci function: as iterative/recursive function, closed form expression (Moivre/Binet), lookup table: The interface is always the same, the possible bugs differ significantly. Another way to understand this is that, when looking at coverage, you always look at the implementation, not the interface.
There are, however, other (secondary) goals for unit-testing, namely to have a robust test suite that does not break unnecessarily when implementation details change. For example, if the name of a private function changed, it would be unfortunate if the unit-testing suite broke. Therefore, often it is beneficial to test implementation details / private functions indirectly (that is, testing them via calls to the public interface).
But, even if tests are implemented against the public API and thus do not break when internals change, some changes to the internals can make the existing and still working tests useless: Tests for the Moivre/Binet implementation of Fibonacci will continue to work if you switch to a lookup table. A lookup table will most likely, however, require a different testing approach. The fact that the tests still work in this case does not mean the test suite does not have to be maintained.
As said, testing private methods / functions in tests has disadvantages, because it can make your tests fragile. There are situations, however, when it is not practical to test these functions from the public API. Then, the best tradeoff can still be to test the private functions separately (remember: primary goal is finding bugs, non-fragile test-suite is secondary goal). You will, however, have to find ways to make the private methods accessible to your tests. One possibility is, as you have shown, to make them public, and you can test them as public functions.
Note that this does not make your testsuite less fragile: Just making some implementation detail publicly accessible does not change its nature of being an implementation detail. There is no magic like "replace private by public and implementation details become more stable". Strictly speaking, you make an implementation detail accessible for the purpose of testing - not for the access from others. This would be an architectural constraint and should be communicated, like, dont't use these classes / methods / functions. One possibility is, to name them accordingly.
I want to use Google test to test my class.
Lets assume I have a state machine implementation and the current state is private
so I have a method SetNextState that looks like that:
void setNextState
{
switch(m_currentState) //m_currentState is a private member
{
case INIT_STATE:
{
if some conditions occurred m_currentState=GO_STATE
}
......
}
}
so I have several cases and each define the behavior to move from certain state to another.
My question:
How do I perform tests on that method assuming the state is relevant only to this class so there is no output
How do I set its value to be, for example "GO_STATE" to test the GO_STATE case
and how do i check the m_currentState at the end of the test
Im trying to avoid putting friends etc. in my UUT code since I want it to be as original as possible
You don't. You do the same thing that your actual program will do, which is provide an input, then examine the result; you say there's no output, but there must be some effect, otherwise the class is pointless!
Failing that, you could make the test a "friend" of the class so that it can inspect its internals, or add an immutable getter for the current state (and who really cares if your class's users get to see that?) but neither option is really in the spirit of the thing.
In my experience, you'll occasionally realise that you're not really unit testing any more but instead functional testing, and Google Test may not be the right tool for that job. If your class is as big as it sounds, that could be the case here. Conversely, you could help yourself by splitting the class into smaller chunks, then unit testing those. Depends what you're going for, really.
Lightness Races in Orbit is correct. However, if sometimes you feel like it's useful to test the private member functions of your class, it often means that your class could be split in multiple smaller pieces.
If you don't think those smaller components are useful to the clients of your library, you can simply hide them in a detail:: namespace and then create unit tests as usual. This will allow you to test the internal behavior of your classes without polluting your public API.
After much considerations I decided to wrap my UUT with a helper which provides set and get to the relevant private members.and use it in the test procedure before calling the tested API
Original code
===============
class UUT //That's the actual class I want to test
{
protected:
int m_protectedMember;
public:
void methodToTest()
{
//Do something with m_protectedMember use its value as input
//and set it as output
}
};
In the tester
==============
class UUTHelper: public UUT
{
public:
int getProtectedMember() { return m_protectedMember; }
void setProtectedMember(int value) { m_protectedMember = value; }
};
The pros:
My test code is very simple and I easily create complicated scenarios .
I test the real code without any "friends" or any other manipulations.
The cons:
As written in the discussion, not the best "good practice", touching private members
Thank you all :)
Consider you have the following method:
public Foo ParseMe(string filepath)
{
// break up filename
// validate filename & extension
// retrieve info from file if it's a certain type
// some other general things you could do, etc
var myInfo = GetFooInfo(filename);
// create new object based on this data returned AND data in this method
}
Currently I have unit tests for GetFooInfo, but I think I also need to build unit tests for ParseMe. In a situation like this where you have a two methods that return two different properties - and a change in either of them could break something - should unit tests be created for both to determine the output is as expected?
I like to err on the side of caution and be more wary about things breaking and ensuring that maintenance later on down the road is easier, but I feel very skeptical about adding very similar tests in the test project. Would this be bad practice or is there any way to do this more efficiently?
I'm marking this as language agnostic, but just in case it matters I am using C# and NUnit - Also, I saw a post similar to this in title only, but the question is different. Sorry if this has already been asked.
ParseMe looks sufficiently non-trivial to require a unit test. To answer your precise question, if "you have a two methods that return two different properties - and a change in either of them could break something" you should absolutely unit test them.
Even if the bulk of the work is in GetFooInfo, at minimum you should test that it's actually called. I know nothing about NUnit, but I know in other frameworks (like RSpec) you can write tests like GetFooInfo.should be_called(:once).
It is not a bad practice to test a method that is calling another method. In fact, it is a good practice. If you have a method calling another method, it is probably performing additional functionality, which should be tested.
If you find yourself unit testing a method that calls a method that is also being unit tested, then you are probably experiencing code reuse, which is a good thing.
I agree with #tsm - absolutely test both methods (assuming both are public).
This may be a smell that the method or class is doing too much - violating the Single Responsibility Principle. Consider doing an Extract Class refactoring and decoupling the two classes (possibly with Dependency Injection). That way you could test both pieces of functionality independently. (That said, I'd only do that if the functionality was sufficiently complex to warrant it. It's a judgment call.)
Here's an example in C#:
public interface IFooFileInfoProvider
{
FooInfo GetFooInfo(string filename);
}
public class Parser
{
private readonly IFooFileInfoProvider _fooFileInfoProvider;
public Parser(IFooFileInfoProvider fooFileInfoProvider)
{
// Add a null check
_fooFileInfoProvider = fooFileInfoProvider;
}
public Foo ParseMe(string filepath)
{
string filename = Path.GetFileName(filepath);
var myInfo = _fooFileInfoProvider.GetFooInfo(filename);
return new Foo(myInfo);
}
}
public class FooFileInfoProvider : IFooFileInfoProvider
{
public FooInfo GetFooInfo(string filename)
{
// Do I/O
return new FooInfo(); // parameters...
}
}
Many developers, me included, take a programming by contract approach. That requires you to consider each method as a black box. If the method delegates to another method to accomplish its task does not matter, when you are testing the method. But you should also test all large or complicated parts of your program as units. So whether you need to unit test the GetFooInfo depends on how complicated that method is.
Suppose I have several unit tests in a test class ([TestClass] in VSUnit in my case). I'm trying to test just one thing in each test (doesn't mean just one Assert though). Imagine there's one test (e.g. Test_MethodA() ) that tests a method used in other tests as well. I do not want to put an assert on this method in other tests that use it to avoid duplicity/maintainability issues so I have the assert in only this one test. Now when this test fails, all tests that depend on correct execution of that tested method fail as well. I want to be able to locate the problem faster so I want to be somehow pointed to Test_MethodA. It would e.g. help if I could make some of the tests in the test class execute in a particular order and when they fail I'd start looking for cause of the failure in the first failing test. Do you have any idea how to do this?
Edit: By suggesting that a solution would be to execute the tests in a particular order I have probably went too far and in the wrong direction. I don't care about the order of the tests. It's just that some of the tests will always fail if a prequisite isn't valid. E.g. I have a test class that tests a DAO class (ok, probably not a UNIT test, but there's logic in the database stored procedures that needs to be tested but that's not the point here I think). I need to insert some records into a table in order to test that a method responsible for retrieving the records (let's call it GetAll()) gets them all in the correct order e.g. I do the insert by using a method on the DAO class. Let's call it Insert(). I have tests in place that verify that the Insert() method works as expected. Now I want to test the GetAll() method. In order to get the database in a desired state I use the Insert() method. If Insert() doesn't work, most tests for GetAll() will fail. I'd prefer to mark the tests that can't pass because Insert() doesn't work as inconclusive rather than failed. It would ease finding the cause of the problem if I know which method/test to look into first.
You can't (and shouldn't) execute unit tests in a specific order. The underlying reason for this is to prevent Interacting Tests - I realize that your motivation for requesting such a feature is different, but that's the reason why unit test frameworks don't allow you to order tests. In fact, last time I checked, xUnit.net even randomizes the order.
One could argue that the fact that some of your tests depend on a different method call on the same class is a symptom of tight coupling, but that's not always the case (state machines come to mind).
However, if possible, consider using a Back Door instead of the other method in question.
If you can't do either that or decouple the interdependency (e.g. by making the first method virtual and using the Extract and Override technique), you will have to live with it.
Here's an example:
public class MyClass
{
public virtual void FirstMethod() { // do something... }
public void SecondMethod() {}
}
Since FirstMethod is virtual, you can derive from MyClass and override its behavior. You can also use a dynamic mock to do that for you. With Moq, it would look like this:
var sutStub = new Mock<MyClass>();
// by default, Moq overrides all virtual methods without calling base
// Now invoke both methods in sequence:
sutStub.Object.FirstMethod(); // overriden by Moq, so it does nothing
sutSutb.Object.SecondMethod();
I think I would indeed have the assertion on the method_A() result in every tests relying on its result, even if this introduces some duplication. Then I would use the assertion message to point to the method_A() failure.
assert("method_A() returned true", true, rc);
Perhaps will I end extracting the method_A() call and the assertion into an helper function to remove the duplication.
Now let's imagine method_A() queries an object and returns it, or NULL when no object is found. Then this assertion is a guard ; and it is necessary with languages suchas C, C++ that do not have NullPointerException.
I'm afraid you can't do this. The only solution is to redesign your code and break it up into smaller methods so that unit tests can call these one by one. Of course this isn't always desirable.
With Visual Studio you can order your tests: see here. But I'd like to advise you to stay away from this technique as much as possible: unit tests are meant to be run anywhere, anytime and in every order.
EDIT: why is this a problem for you? All failing tests point to the same method anyway...
I have the following scenario:
public class CarManager
{
..
public long AddCar(Car car)
{
try
{
string username = _authorizationManager.GetUsername();
...
long id = _carAccessor.AddCar(username, car.Id, car.Name, ....);
if(id == 0)
{
throw new Exception("Car was not added");
}
return id;
} catch (Exception ex) {
throw new AddCarException(ex);
}
}
public List AddCars(List cars)
{
List ids = new List();
foreach(Car car in cars)
{
ids.Add(AddCar(car));
}
return ids;
}
}
I am mocking out _reportAccessor, _authorizationManager etc.
Now I want to unittest the CarManager class.
Should I have multiple tests for AddCar() such as
AddCarTest()
AddCarTestAuthorizationManagerException()
AddCarTestCarAccessorNoId()
AddCarTestCarAccessorException()
For AddCars() should I repeat all previous tests as AddCars() calls AddCar() - it seems like repeating oneself? Should I perhaps not be calling AddCar() from AddCars()? < p/>
Please help.
There are two issues here:
Unit tests should do more than test methods one at a time. They should be designed to prove that your class can do the job it was designed for when integrated with the rest of the system. So you should mock out the dependencies and then write a test for each way in which you class will actually be used. For each (non-trivial) class you write there will be scenarios that involve the client code calling methods in a particular pattern.
There is nothing wrong with AddCars calling AddCar. You should repeat tests for error handling but only when it serves a purpose. One of the unofficial rules of unit testing is 'test to the point of boredom' or (as I like to think of it) 'test till the fear goes away'. Otherwise you would be writing tests forever. So if you are confident a test will add no value them don't write it. You may be wrong of course, in which case you can come back later and add it in. You don't have to produce a perfect test first time round, just a firm basis on which you can build as you better understand what your class needs to do.
Unit Test should focus only to its corresponding class under testing. All attributes of class that are not of same type should be mocked.
Suppose you have a class (CarRegistry) that uses some kind of data access object (for example CarPlatesDAO) which loads/stores car plate numbers from Relational database.
When you are testing the CarRegistry you should not care about if CarPlateDAO performs correctly; Since our DAO has it's own unit test.
You just create mock that behaves like DAO and returns correct or wrong values according to expected behavior. You plug this mock DAO to your CarRegistry and test only the target class without caring if all aggregated classes are "green".
Mocking allows separation of testable classes and better focus on specific functionality.
When unittesting the AddCar class, create tests that will exercise every codepath. If _authorizationManager.GetUsername() can throw an exception, create a test where your mock for this object will throw. BTW: don't throw or catch instances of Exception, but derive a meaningful Exception class.
For the AddCars method, you definitely should call AddCar. But you might consider making AddCar virtual and override it just to test that it's called with all cars in the list.
Sometimes you'll have to change the class design for testability.
Should I have multiple tests for
AddCar() such as
AddCarTest()
AddCarTestAuthorizationManagerException()
AddCarTestCarAccessorNoId()
AddCarTestCarAccessorException()
Absolutely! This tells you valuable information
For AddCars() should I repeat all previous tests as AddCars() calls AddCar() - it seems
like repeating oneself? Should I perhaps not be calling AddCar() from AddCars()?
Calling AddCar from AddCars is a great idea, it avoids violating the DRY principle. Similarly, you should be repeating tests. Think of it this way - you already wrote tests for AddCar, so when testing AddCards you can assume AddCar does what it says on the tin.
Let's put it this way - imagine AddCar was in a different class. You would have no knowledge of an authorisation manager. Test AddCars without the knowledge of what AddCar has to do.
For AddCars, you need to test all normal boundary conditions (does an empty list work, etc.) You probably don't need to test the situation where AddCar throws an exception, as you're not attempting to catch it in AddCars.
Writing tests that explore every possible scenario within a method is good practice. That's how I unit test in my projects. Tests like AddCarTestAuthorizationManagerException(), AddCarTestCarAccessorNoId(), or AddCarTestCarAccessorException() get you thinking about all the different ways your code can fail which has led to me find new kinds of failures for a method I might have otherwise missed as well as improve the overall design of the class.
In a situation like AddCars() calling AddCar() I would mock the AddCar() method and count the number of times it's called by AddCars(). The mocking library I use allows me to create a mock of CarManager and mock only the AddCar() method but not AddCars(). Then your unit test can set how many times it expects AddCar() to be called which you would know from the size of the list of cars passed in.