When writing unit tests for a stateful object with many optional methods, do I define my units as the individual methods of the class or as particular paths that can be taken through the class? Or is my unit the entire class?
To illustrate, imagine a class that sends a single e-mail and has several methods like: send, attachBinary, attachText, clearAttachments, addSender, addRecipient, addCC, setBody, setSubject. Only a single call to addRecipient is strictly necessary for an email to be valid, but I can imagine any number of paths to take: adding attachments, removing previously attached files and adding new ones, changing the recipient, etc.
How should I go about writing tests for a class like this? If I'm testing paths, am I still writing unit tests despite this feeling more like integration testing?
"Unit" or "Not Unit"
If all the "parts" really belong to the class, then your class is the unit, and as long as you're testing methods of this unit, that's, ahem, unit testing.
If you feel like your object has methods that need to be tested completely differently, maybe you class can be split in smaller "units", that you can test to.
Testing paths or methods
Some of those methods might be private, and can only be exercised by calling a few, "public" methods ; that's fine. Don't think one method needs to have a single corresponding tests.
This convention is dangerous, as it can imply you need one test per method:
#Test
private void testSendMail() {
// Code for a test for mail with attachement
// Code for another test for mail without attachement
// Code for hundreds of other tests...
}
This one is probably healthier :
#Test
private void testMailsCanBeSentWithAttachement() {
}
#Test
private void testMailsCanBeSentWithMultipartAttachement() {
}
// etc...
Writing many tests to test a single method, with many different behavior, is usually fine, too.
Although, if it adds overhead, testing the individual "smaller" methods is probably ok.
SPOILER : Private methods
You'll get to a point where the simplest things to do will be to call a "private" method in a test. As this point, changing the method to be public or protected become a philosophy debate. My (personal) opinion is to go on with your life, extract or expose some protected methods, and test them.
"What's in a name ?" vs "Does size matter" ?
Finally, the difference between "unit" and "integration" test is a fine subject for debates in a bar ; the more I write tests, the more I like separating tests by "size".
Related
When I'm writing some typescript code, I wanted to unit test some private method of Foo Class.
class Foo {
private privateMethodWantToTest(argument) {
return 'Bar'
}
}
So, I separated function like this:
export function privateMethodWantToTest(this: Foo, argument) {
return 'Bar'
}
class Foo {
private privateMethodWantToTest(argument) {
return privateMethodWantToTest.call(this, argument)
}
}
This approach may work, but It's a bit redundant.
And exporting all function may be a bad idea.
What do you think about this approach ?
Is there better solution for unit test, or should't I test private method ?
It depends. First, testing is about finding (or preventing) bugs. Bugs are in the implementation details: Different implementations have different potential bugs. For example, think about the different ways to implement a Fibonacci function: as iterative/recursive function, closed form expression (Moivre/Binet), lookup table: The interface is always the same, the possible bugs differ significantly. Another way to understand this is that, when looking at coverage, you always look at the implementation, not the interface.
There are, however, other (secondary) goals for unit-testing, namely to have a robust test suite that does not break unnecessarily when implementation details change. For example, if the name of a private function changed, it would be unfortunate if the unit-testing suite broke. Therefore, often it is beneficial to test implementation details / private functions indirectly (that is, testing them via calls to the public interface).
But, even if tests are implemented against the public API and thus do not break when internals change, some changes to the internals can make the existing and still working tests useless: Tests for the Moivre/Binet implementation of Fibonacci will continue to work if you switch to a lookup table. A lookup table will most likely, however, require a different testing approach. The fact that the tests still work in this case does not mean the test suite does not have to be maintained.
As said, testing private methods / functions in tests has disadvantages, because it can make your tests fragile. There are situations, however, when it is not practical to test these functions from the public API. Then, the best tradeoff can still be to test the private functions separately (remember: primary goal is finding bugs, non-fragile test-suite is secondary goal). You will, however, have to find ways to make the private methods accessible to your tests. One possibility is, as you have shown, to make them public, and you can test them as public functions.
Note that this does not make your testsuite less fragile: Just making some implementation detail publicly accessible does not change its nature of being an implementation detail. There is no magic like "replace private by public and implementation details become more stable". Strictly speaking, you make an implementation detail accessible for the purpose of testing - not for the access from others. This would be an architectural constraint and should be communicated, like, dont't use these classes / methods / functions. One possibility is, to name them accordingly.
I'm looking to better understand I should test functions that have many substeps or subfunctions.
Let's say I have the functions
// Modify the state of class somehow
public void DoSomething(){
DoSomethingA();
DoSomethingB();
DoSomethingC();
}
Every function here is public. Each subfunction has 2 paths. So to test every path for DoSomething() I'd have 2*2*2 = 8 tests. By writing 8 tests for DoSomething() I will have indirectly tested the subfunctions too.
So should I be testing like this, or instead write unit tests for each of the subfunctions and then only write 1 test case that measures the final state of the class after DoSomething() and ignore all the possible paths? A total of 2+2+2+1 = 7 tests. But is it bad then that the DoSomething() test case will depend on the other unit test cases to have complete coverage?
There appears to be a very prevalent religious belief that testing should be unit testing. While I do not intend to underestimate the usefulness of unit testing, I would like to point out that it is just one possible flavor of testing, and its extensive (or even exclusive) use is indicative of people (or environments) that are somewhat insecure about what they are doing.
In my experience knowledge of the inner workings of a system is useful as a hint for testing, but not as an instrument for testing. Therefore, black box testing is far more useful in most cases, though that's admittedly in part because I do not happen to be insecure about what I am doing. (And that is in turn because I use assertions extensively, so essentially all of my code is constantly testing itself.)
Without knowing the specifics of your case, I would say that in general, the fact that DoSomething() works by invoking DoSomethingA() and then DoSomethingB() and then DoSomethingC() is an implementation detail that your black-box test should best be unaware of. So, I would definitely not test that DoSomething() invokes DoSomethingA(), DoSomethingB(), and DoSomethingC(), I would only test to make sure that it returns the right results, and using the knowledge that it does in fact invoke those three functions as a hint I would implement precisely those 7 tests that you were planning to use.
On the other hand, it should be noted that if DoSomethingA() and DoSomethingB() and DoSomethingC() are also public functions, then you should also test them individually, too.
Definitely test every subfunction seperately (because they're public).
It would help you find the problem if one pops up.
If DoSomething only uses other functions, I wouldn't bother writing additional tests for it. If it has some other logic, I would test it, but assume all functions inside work properly (if they're in a different class, mock them).
The point is finding what the function does that is not covered in other tests and testing that.
Indirect testing should be avoided. You should write unit tests for each function explicitly. After that You should mock submethods and test your main function. For example :
You have a method which inserts a user to DB and method is like this :
void InsertUser(User user){
var exists = SomeExternal.UserExists(user);
if(exists)
throw new Exception("bla bla bla");
//Insert codes here
}
If you want to test InsertUser function, you should mock external/sub/nested methods and test behaviour of InsertUser function.
This example creates two tests: 1 - "When user exists then Should throw Exception" 2 - "When user does not exist then Should insert user"
Consider you have the following method:
public Foo ParseMe(string filepath)
{
// break up filename
// validate filename & extension
// retrieve info from file if it's a certain type
// some other general things you could do, etc
var myInfo = GetFooInfo(filename);
// create new object based on this data returned AND data in this method
}
Currently I have unit tests for GetFooInfo, but I think I also need to build unit tests for ParseMe. In a situation like this where you have a two methods that return two different properties - and a change in either of them could break something - should unit tests be created for both to determine the output is as expected?
I like to err on the side of caution and be more wary about things breaking and ensuring that maintenance later on down the road is easier, but I feel very skeptical about adding very similar tests in the test project. Would this be bad practice or is there any way to do this more efficiently?
I'm marking this as language agnostic, but just in case it matters I am using C# and NUnit - Also, I saw a post similar to this in title only, but the question is different. Sorry if this has already been asked.
ParseMe looks sufficiently non-trivial to require a unit test. To answer your precise question, if "you have a two methods that return two different properties - and a change in either of them could break something" you should absolutely unit test them.
Even if the bulk of the work is in GetFooInfo, at minimum you should test that it's actually called. I know nothing about NUnit, but I know in other frameworks (like RSpec) you can write tests like GetFooInfo.should be_called(:once).
It is not a bad practice to test a method that is calling another method. In fact, it is a good practice. If you have a method calling another method, it is probably performing additional functionality, which should be tested.
If you find yourself unit testing a method that calls a method that is also being unit tested, then you are probably experiencing code reuse, which is a good thing.
I agree with #tsm - absolutely test both methods (assuming both are public).
This may be a smell that the method or class is doing too much - violating the Single Responsibility Principle. Consider doing an Extract Class refactoring and decoupling the two classes (possibly with Dependency Injection). That way you could test both pieces of functionality independently. (That said, I'd only do that if the functionality was sufficiently complex to warrant it. It's a judgment call.)
Here's an example in C#:
public interface IFooFileInfoProvider
{
FooInfo GetFooInfo(string filename);
}
public class Parser
{
private readonly IFooFileInfoProvider _fooFileInfoProvider;
public Parser(IFooFileInfoProvider fooFileInfoProvider)
{
// Add a null check
_fooFileInfoProvider = fooFileInfoProvider;
}
public Foo ParseMe(string filepath)
{
string filename = Path.GetFileName(filepath);
var myInfo = _fooFileInfoProvider.GetFooInfo(filename);
return new Foo(myInfo);
}
}
public class FooFileInfoProvider : IFooFileInfoProvider
{
public FooInfo GetFooInfo(string filename)
{
// Do I/O
return new FooInfo(); // parameters...
}
}
Many developers, me included, take a programming by contract approach. That requires you to consider each method as a black box. If the method delegates to another method to accomplish its task does not matter, when you are testing the method. But you should also test all large or complicated parts of your program as units. So whether you need to unit test the GetFooInfo depends on how complicated that method is.
I have a PersonDao that I'm writing unit tests against.
There are about 18-20 methods in PersonDao of the form -
getAllPersons()
getAllPersonsByCategory()
getAllPersonsUnder21() etc
My Approach to testing this was to create a PersonDaoTest with about 18 test methods testing each of the method in PersonDao
Then I created a PersonDaoPaginationTest that tested these 18 methods by applying pagination parameters.
Is this in anyway against the TDD best practices? I was told that this creates confusion and is against the best practices since this is non-standard. What was suggested is merging the two classes into PersonDaoTest instead.
As I understand is, the more broken down into many classes your code is, the better, please comment.
The fact that you have a set of 18 tests that you are going to have to duplicate to test a new feature is a smell that suggests that your PersonDao class is taking on multiple responsibilities. Specifically, it appears to be responsible both for querying/filter and for pagination. You may want to take a look at whether you can do a bit of design work to extract the pagination functionality into a separate class which could then be tested independently.
But in answer to your question, if you find that you have a class that you want to remain complex, then it's perfectly fine to use multiple test classes as a way of organizing a large number of tests. #Gishu's answer of grouping tests by their setup is a good approach. #Ryan's answer of grouping by "facets" or features is another good approach.
Can't give you a sweeping answer without looking at the code... except use whatever seems coherent to you and your team.
I've found that grouping tests based on their setup works out nicely in most cases. i.e if 5 tests require the same setup, they usually fit nicely into a test-fixture. if the 6th test requires a different setup (more or less) break it out into a separate test fixture.
This also leads to test-fixtures that are feature-cohesive (i.e. tests grouped on feature), give it a try. I'm not aware of any best practice that says you need to have one test class per production class... in practice I find I have n test classes per production classes, the best practice would be to use good names and keep related tests close (in a named folder).
My 2 cents: when you have a large class like that that has different "facets" to it, like pagination, I find it can often make for more understandable tests to not pack them all into one class. I can't claim to be a TDD guru, but I practice test-first development religiously, so to speak. I don't do it often, but it's not exactly rare, either, that I'll write more than a single test class for a particular class. Many people seem to forget good coding practices like separation of concerns when writing tests, though. I'm not sure why.
I think one test class per class is fine - if your implementation has many methods, then your test class will have many methods - big deal.
You may consider a couple of things however:
Your methods seem a bit "overly specific" and could use some abstraction or generalisation, for example instead of getAllPersonsUnder21() consider getAllPersonsUnder(int age)
If there are some more general aspects of your class, consider testing them using some common test code using call backs. For a trivial example to illustrate testing that both getAllPersons() returns multiple hits, do this:
#Test
public void testGetAllPersons() {
assertMultipleHits(new Callable<List<?>> () {
public List<?> call() throws Exception {
return myClass.getAllPersons(); // Your call back is here
}
});
}
public static void assertMultipleHits(Callable<List<?>> methodWrapper) throws Exception {
assertTrue("failure to get multiple items", methodWrapper.call().size() > 0);
}
This static method can be used by any class to test if "some method" returns multiple hits. You could extends this to do lots of tests over the same callback, for example running it with and without a DB connection up, testing that it behaves correctly in each case.
I'm working on test automation of a web app using selenium. It is not unit testing but you might find that some principles apply. Tests are very complex and we figured out that the only way to implement tests in a way that meets all our requirements was having 1 test per class. So we consider that each class is an individual test, then, we were able to use methods as the different steps of the test. For example:
public SignUpTest()
{
public SignUpTest(Map<String,Object> data){}
public void step_openSignUpPage(){}
public void step_fillForm(){}
public void step_submitForm(){}
public void step_verifySignUpWasSuccessfull(){}
}
All the steps are dependent, they follow the order specified and if someone fail the others will not be executed.
Of course, each step is a test by itself, but they all together form the sing up test.
The requirements were something like:
Tests must be data driven, this is, execute the same test in parallel with different inputs.
Tests must run in different browsers in parallel as well. So each
test will run "input_size x browsers_count" times in parallel.
Tests will focus in a web workflow, for example, "sign up with valid data" and they will be split into smaller tests units for each step of the workflow. It will make things easier to
maintain, and debug (when you have a failure, it will say:
SignUpTest.step_fillForm() and you'll know immediately what's wrong).
Tests steps share the same test input and state (for example, the id of the user created). Imagine if you put in the same class
steps of different tests, for example:
public SignUpTest()
{
public void signUpTest_step_openSignUpPage(){}
public void signUpTest_step_step_fillForm(){}
public void signUpTest_step_step_submitForm(){}
public void signUpTest_step_verifySignUpWasSuccessfull(){}
public void signUpNegativeTest_step_openSignUpPage(){}
public void signUpNegativeTest_step_step_fillFormWithInvalidData(){}
public void signUpNegativeTest_step_step_submitForm(){}
public void signUpNegativeTest_step_verifySignUpWasNotSuccessfull(){}
}
Then, having in the same class state belonging to the 2 tests will be
a mess.
I hope I was clear and you may find this useful. At the end, choosing what will represent your test: if a class or a method is just a decision that I think will depend int: what is the target of a test (in my case, a workflow around a feature), what's easier to implement and maintain, if a test fail how you make the failure more accurate and how you make it easier to debug, what will lead you to more readable code, etc.
I follow the technique specified in Roy Osherove's The Art Of Unit Testing book while naming test methods - MethodName_Scenario_Expectation.
It suits perfectly well for my 'unit' tests. But,for tests that I write in 'controller' or 'coordinator' class, there isn't necessarily a method which I want to test.
For these tests, I generate multiple conditions which make up one scenario and then I verify the expectation. For example, I may set some properties on different instances, generate an event and then verify that my expectation from controller/coordinator is being met. Now, my controller handles events using a private event handler. Here my scenario is that, I set some properties, say 3
condition1,condition2 and condition3
Also, my scenario includes
an event is raised
I don't have a method name as my event handler is private. How do I name such a test method?
I would use several tests and a different naming convention in this case:
Name the test class ClassName_Scenario (so you would have more than one test class for the same class)
Name the test methods Expectation1, Expectation2, Expectation3...
Also, instead of initializing the context in each test method, it would be initialized in a [SetUp] method
So you would have something like this:
[TestFixture]
public class ClassName_WhenScenarioX
{
[SetUp]
void InitContextForScenarioX()
{
}
[Test]
void ShouldDoY()
{
Assert.That(...)
}
[Test]
void ShouldRaiseEventZ()
{
Assert.That(...)
}
}
Note that this will only work if the order of execution of your asserts is not important (each test is independent, even if they all depend on the same initial context)
I tend to use almost full sentences to describe what the tested class should actually be providing. This way unit tests are almost a documentation of the class. I tend to avoid adding technical details to the name of the test though, because (1) these are in the content of the test anyway (2) someone might change the content of the test but not the name.
Examples:
[TestFixture]
public class ClassNameTests
{
[SetUp]
void BeforeEveryTest()
{
}
[Test]
void ParsesCsvStringsToLists()
{
Assert.That(...)
}
[Test]
void ThrowsMyExceptionWhenInputStringIsMalformed()
{
Assert.That(...)
}
}
I usually name my test methods after the scenario (and, if needed, its subcases), but I rarely include the method name or the expectations into the test name.
To me the scenario is most important, and this is what is not always obvious from the code, as it is on a higher level of abstraction than the code. So I need a nice descriptive test name to communicate it well. However, the called method is seen in the code, similarly the asserts / annotations document the expectations. And I am a big fan of the KISS principle...
I must add that I am working with legacy code, and our scenarios and unit tests typically are more bulky then a textbook unit test would be. Also, the interfaces we test are fairly simple, often having a single "execute" style of method per class.
If condition1,condition2 and condition3 is a business rule then name it after the rule.