Related
I like unit testing, it is proving its worth immensely for the last year and a half or so ive used it. However I always have a problem or rather a niggle with private methods (and protected).
I don't want to make them public or use internals visible to attribute. I want a clean and crisp solution - that is testable and i'd be proud to let someone else look at.
I am coming to the conclusion that if a private method really needs testing independantly then maybe it should be moved out onto another interface and use association to expose the functionality to the calling method. I believe this in essence to be the Facade pattern.
Is this really the best way to go about this?
Or more objectively ... are there any other approaches I have totally overlooked?
Edit: Are we talking about a specific language?
I am working in C#. I had kept code out of the question as i was looking for something abstract. Coming back to it today i realise that is perhaps folly due to languages really being that different.
So some code:
public class CopmlexClass
{
public void SomeMethod()
{ }
private void workerMethod()
{ }
}
would get re factored into
public class CopmlexClass
{
public void SomeMethod()
{ }
public IComplexClassWorker Worker { get; set; }
}
public interface IComplexClassWorker
{
void WorkerMethod();
}
In fact id probably prefer to use constructor injection and not even expose the property
My question is: is that the best way? what are the alternatives bar reflection / internals visible to attribute?
A private method which needs to be tested independently can be the result of the following:
your class is doing too much - its public methods implement functionality which is too complex to be unit tested as a whole, and/or
the effects of calling the specific private method can't be directly sensed from outside the class.
Both cases are usually a clear call to extract another class containing some private method(s) of the original class, turned into public, thus made directly testable. (The sometimes challenging part is to find logically cohesive chunks of functionality which can form useful classes on their own right. You may not always get a perfect result at first - in refactoring, sometimes one needs to make a compromise, a small step into some not-yet-clearly-defined direction. In the long term, such a step may open up new possibilities, call attention to other similar code parts, or the new class may start to attract code bits from other places, eventually forming a coherent class. Or turning into something completely new, but better than what you originally envisioned.)
Exposing the private method via another interface / facade is IMO not going to solve the problem in the long term, only muddles the waters. Classes should have a well defined, complete and minimal interface. Exposing private methods in any way may open up ways to compromise the internal state of the object, which is a Bad Thing.
When we started writing unit tests in our team a couple of years ago we started with the rules you set out above - i.e. we test the public interface of an assembly.
We expected one advantage to be in detecting unreachable code. If the code coverage tools detect code blocks which not being tested, then either tests are missing or the code is unreachable and should be removed.
But in practice we haven't stuck to them. We have a very modular design - more than 30 projects in our main solution (most having a matching unit tests project). We now usually give the test project access to the internals of the project under test.
I think one problem is that we are not automatically using code coverage to detect missing tests or unreachable code. Because this is a manual process, it doesn't get done.
I have a PersonDao that I'm writing unit tests against.
There are about 18-20 methods in PersonDao of the form -
getAllPersons()
getAllPersonsByCategory()
getAllPersonsUnder21() etc
My Approach to testing this was to create a PersonDaoTest with about 18 test methods testing each of the method in PersonDao
Then I created a PersonDaoPaginationTest that tested these 18 methods by applying pagination parameters.
Is this in anyway against the TDD best practices? I was told that this creates confusion and is against the best practices since this is non-standard. What was suggested is merging the two classes into PersonDaoTest instead.
As I understand is, the more broken down into many classes your code is, the better, please comment.
The fact that you have a set of 18 tests that you are going to have to duplicate to test a new feature is a smell that suggests that your PersonDao class is taking on multiple responsibilities. Specifically, it appears to be responsible both for querying/filter and for pagination. You may want to take a look at whether you can do a bit of design work to extract the pagination functionality into a separate class which could then be tested independently.
But in answer to your question, if you find that you have a class that you want to remain complex, then it's perfectly fine to use multiple test classes as a way of organizing a large number of tests. #Gishu's answer of grouping tests by their setup is a good approach. #Ryan's answer of grouping by "facets" or features is another good approach.
Can't give you a sweeping answer without looking at the code... except use whatever seems coherent to you and your team.
I've found that grouping tests based on their setup works out nicely in most cases. i.e if 5 tests require the same setup, they usually fit nicely into a test-fixture. if the 6th test requires a different setup (more or less) break it out into a separate test fixture.
This also leads to test-fixtures that are feature-cohesive (i.e. tests grouped on feature), give it a try. I'm not aware of any best practice that says you need to have one test class per production class... in practice I find I have n test classes per production classes, the best practice would be to use good names and keep related tests close (in a named folder).
My 2 cents: when you have a large class like that that has different "facets" to it, like pagination, I find it can often make for more understandable tests to not pack them all into one class. I can't claim to be a TDD guru, but I practice test-first development religiously, so to speak. I don't do it often, but it's not exactly rare, either, that I'll write more than a single test class for a particular class. Many people seem to forget good coding practices like separation of concerns when writing tests, though. I'm not sure why.
I think one test class per class is fine - if your implementation has many methods, then your test class will have many methods - big deal.
You may consider a couple of things however:
Your methods seem a bit "overly specific" and could use some abstraction or generalisation, for example instead of getAllPersonsUnder21() consider getAllPersonsUnder(int age)
If there are some more general aspects of your class, consider testing them using some common test code using call backs. For a trivial example to illustrate testing that both getAllPersons() returns multiple hits, do this:
#Test
public void testGetAllPersons() {
assertMultipleHits(new Callable<List<?>> () {
public List<?> call() throws Exception {
return myClass.getAllPersons(); // Your call back is here
}
});
}
public static void assertMultipleHits(Callable<List<?>> methodWrapper) throws Exception {
assertTrue("failure to get multiple items", methodWrapper.call().size() > 0);
}
This static method can be used by any class to test if "some method" returns multiple hits. You could extends this to do lots of tests over the same callback, for example running it with and without a DB connection up, testing that it behaves correctly in each case.
I'm working on test automation of a web app using selenium. It is not unit testing but you might find that some principles apply. Tests are very complex and we figured out that the only way to implement tests in a way that meets all our requirements was having 1 test per class. So we consider that each class is an individual test, then, we were able to use methods as the different steps of the test. For example:
public SignUpTest()
{
public SignUpTest(Map<String,Object> data){}
public void step_openSignUpPage(){}
public void step_fillForm(){}
public void step_submitForm(){}
public void step_verifySignUpWasSuccessfull(){}
}
All the steps are dependent, they follow the order specified and if someone fail the others will not be executed.
Of course, each step is a test by itself, but they all together form the sing up test.
The requirements were something like:
Tests must be data driven, this is, execute the same test in parallel with different inputs.
Tests must run in different browsers in parallel as well. So each
test will run "input_size x browsers_count" times in parallel.
Tests will focus in a web workflow, for example, "sign up with valid data" and they will be split into smaller tests units for each step of the workflow. It will make things easier to
maintain, and debug (when you have a failure, it will say:
SignUpTest.step_fillForm() and you'll know immediately what's wrong).
Tests steps share the same test input and state (for example, the id of the user created). Imagine if you put in the same class
steps of different tests, for example:
public SignUpTest()
{
public void signUpTest_step_openSignUpPage(){}
public void signUpTest_step_step_fillForm(){}
public void signUpTest_step_step_submitForm(){}
public void signUpTest_step_verifySignUpWasSuccessfull(){}
public void signUpNegativeTest_step_openSignUpPage(){}
public void signUpNegativeTest_step_step_fillFormWithInvalidData(){}
public void signUpNegativeTest_step_step_submitForm(){}
public void signUpNegativeTest_step_verifySignUpWasNotSuccessfull(){}
}
Then, having in the same class state belonging to the 2 tests will be
a mess.
I hope I was clear and you may find this useful. At the end, choosing what will represent your test: if a class or a method is just a decision that I think will depend int: what is the target of a test (in my case, a workflow around a feature), what's easier to implement and maintain, if a test fail how you make the failure more accurate and how you make it easier to debug, what will lead you to more readable code, etc.
I've used the self-shunt unit testing pattern a few times over the years. As I was explaining it to someone recently they argued that it violated the SRP. The argument is that the test class can now be changed for one of two reasons: when the test changes, or when the method signature on the interface that the test is implementing changes. After thinking about it for a while, it seems like this is a correct assessment, but I wanted to get other peoples' opinions. Thoughts?
Reference:
http://www.objectmentor.com/resources/articles/SelfShunPtrn.pdf
My take on this is that the test class technically violates the SRP, but it doesn't violate the spirit of SRP. The alternative to self-shunting is to have a mock class separate from the test class.
With the separate mock class you might think that it's all self contained and satisfies the SRP, however the semantic coupling to the mock class's attributes is still there. So, really, we didn't achieve any meaningful separation.
Taking the example out of the PDF:
public class ScannerTest extends TestCase implements Display
{
public ScannerTest (String name) {
super (name);
}
public void testScan () {
// pass self as a display
Scanner scanner = new Scanner (this);
// scan calls displayItem on its display
scanner.scan ();
assertEquals (new Item (“Cornflakes”), lastItem);
}
// impl. of Display.displayItem ()
void displayItem (Item item) {
lastItem = item;
}
private Item lastItem;
}
Now we make a Mock:
public class DisplayMock implements Display
{
// impl. of Display.displayItem ()
void displayItem (Item item) {
lastItem = item;
}
public Item getItem() {
return lastItem;
}
private Item lastItem;
}
public class ScannerTest extends TestCase
{
public ScannerTest (String name) {
super (name);
}
public void testScan () {
// pass self as a display
DisplayMock dispMock = new DisplayMock();
Scanner scanner = new Scanner (dispMock );
// scan calls displayItem on its display
scanner.scan ();
assertEquals (new Item (“Cornflakes”), dispMock.GetItem());
}
}
In practical terms (IMHO) the higher coupling of TestClass to DisplayMock is a greater evil than the violation of the SRP for TestClass. Besides, with the use mocking frameworks, this problem goes away completely.
EDIT I have just encountered a brief mention of self-shunt pattern in Robert C. Martin's excellent book Agile Principles, Patterns, and Practices in C#. Here is the snippet out of the book:
We can accomplish this by using an abstract interface for the database. One implementation of this abstract interface uses the real database. Another implementation is test code written to simulate the behavior of the database and to check that the database calls are being made correctly. Figure 29-5 shows the structure. The PayrollTest module tests the PayrollModule by making calls to it and also implements the Database interface so that it can trap the calls that Payroll makes to the database. This allows PayrollTest to ensure that Payroll is behaving properly. It also allows PayrollTest to simulate many kinds of database failures and problems that are otherwise difficult to create. This is a testing pattern known as SELF-SHUNT, also sometimes known as mocking or spoofing.
So, the guy who coined the SRP (which is talked about in great detail in the same book) has no qualms using self-shunt pattern. In light of that, I'd say you are pretty safe from the OOP (Objected Orientated Police) when using this pattern.
In my opinion it is a violation, but a very minor one.
You test class is now a test class and a dependency for whatever you are testing.
However, is that a bad thing? For a couple of simple tests, probably not. As your number of test cases grows, you'll probably want to refactor and use a mock class to separate some of the concerns. (As the link you pasted says, self-shunt is a stepping stone to mocking). But if the number of test cases remains static and low, what's the problem?
I think a little pragmatism is required. Does it violate the SRP? Yes, but I'm guessing probably not as much as some of the code in the system you're testing. Do you need to do anything about it? No, not as long as the code is clear and maintainable, which for me is always the bottom line. SRP is a guideline, not a rule.
If the interface being implemented or shunted is changed, it's relatively likely that the test suite will have to change as well. So I don't really see it as a violation of SRP.
I prefer to have a little more control over the mocks/stubs that I'm creating. When I tried using the self shunt pattern I ended up making my test class more complex. By creating the mocks as locals within a test method I ended up having cleaner code.
FWIW unless you're using something powerful like C# (or python or an equivalent) your test code will change when you change an interface.
If unit-test names can become outdated over time and if you consider that the test itself is the most important thing, then is it important to choose wise test names?
ie
[Test]
public void ShouldValidateUserNameIsLessThan100Characters() {}
verse
[Test]
public void UserNameTestValidation1() {}
The name of any method should make it clear what it does.
IMO, your first suggestion is a bit long and the second one isn't informative enough. Also it's probably a bad idea to put "100" in the name, as that's very likely to change. What about:
public void validateUserNameLength()
If the test changes, the name should be updated accordingly.
Yes, the names are totally important, specially when you are running the tests in console or continuous integration servers. Jay Fields wrote a post about it.
Moreover, put good test names with one assertion per test and your suite will give you great reports when a test fails.
Very. Equally important as choosing good method and variable names.
Much more if your test suite is going to referred to by new devs in the future.
As for your original question, definitely Answer1. Typing in a few more characters is a small price to pay for
the readability. For you and others. It'll eliminate the 'what was I thinking here?' as well as 'WTF is this guy getting at in this test?'
Quick zoom in when you're in to fix something someone else wrote
instant update for any test-suite visitor. If done correctly, just going over the names of the test cases will inform the reader of the specs for the unit.
Yes.
[Test]
public void UsernameValidator_LessThanLengthLimit_ShouldValidate() {}
Put the test subject first, the test statement next, and the expected result last.
That way, you get a clear indication of what it is doing, and you can easily sort by name :)
In Clean Code, page 124, Robert C. Martin writes:
The moral of the story is simple: Test code is just as important as production code. It is not a second-class citizen. It requires thought, design, and care. It must be kept as clean as production code.
I think if one can not find a good concise name for a test method it's a sign that design of this test is incorrect. Also good method name helps you to find out what happened in less time.
Yes, the whole point of the test name is that it tells you what doesn't work when the test fails.
i wouldn't put conditions that test needs to meet in the name, because conditions may change in time. in your example, i'd recommend naming like
UserNameLengthValidate()
or
UserNameLengthTest()
or something similar to explain what the test does, but not presuming the testing/validation parameters.
Yes, the names of the code under test (methods, properties, whatever) can change, but I contend your existing tests should fail if the expectations change. That is the true value of having well-constructed tests, not perusing a list of test names. That being said, well named test methods are great tools for getting new developers on board, helping them locate "executable documentation" with which they can kick the tires of existing code -- so I would keep the names of test methods up to date just as I would keep the assertions made by the test methods up to date.
I name my test using the following pattern. Each test fixture attempts to focus on one class and is usually name {ClassUnderTest}Test. I name each test method {MemberUnderTest}_{Assertion}.
[TestFixture]
public class IndexableFileTest
{
[Test]
public void Connect_InitializesReadOnlyProperties()
{
// ...
}
[Test,ExpectedException(typeof(NotInitializedException))]
public void IsIndexable_ErrorWhenNotConnected()
{
// ...
}
[Test]
public void IsIndexable_True()
{
// ...
}
[Test]
public void IsIndexable_False()
{
// ...
}
}
Having a very descriptive name helps to instantly see what is not working correctly, so that you don't actually need to look at the unit test code.
Also, a list of all the unit tests describes the intended behavior of the unit, and can be used (more or less) as documentation to the behavior of the unit under test.
Note, this only works when unit tests are very specific and do not validate too much within one unit test.
So for example:
[Test]
void TestThatExceptionIsRaisedWhenStringLengthLargerThen100()
[Test]
void TestThatStringLengthOf99IsAccepted()
The name needs to matter within reason. I don't want an email from the build saying that test 389fb2b5-28ad3 failed, but just knowing that it was a UserName test as opposed to something else would help ensure the right person gets to do the diagnosis.
[RowTest]
[Row("GoodName")]
[Row("GoodName2")]
public void Should_validate_username()
{
}
[RowTest]
[Row("BadUserName")]
[Row("Bad%!Name")]
public void Should_invalidate_username()
{
}
This might make more sense for more complex types of validation really.
Yes, they are. I'd personally recommend looking at SSW's rules to better unit tests. It contains some very helpful naming guidelines.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
General
Follow the same standards for all tests.
Be clear about what each test state is.
Be specific about the expected behavior.
Examples
1) MethodName_StateUnderTest_ExpectedBehavior
Public void Sum_NegativeNumberAs1stParam_ExceptionThrown()
Public void Sum_NegativeNumberAs2ndParam_ExceptionThrown ()
Public void Sum_simpleValues_Calculated ()
Source: Naming standards for Unit Tests
2) Separating Each Word By Underscore
Public void Sum_Negative_Number_As_1st_Param_Exception_Thrown()
Public void Sum_Negative_Number_As_2nd_Param_Exception_Thrown ()
Public void Sum_Simple_Values_Calculated ()
Other
End method names with Test
Start method names with class name
I am pretty much with you on this one man. The naming conventions you have used are:
Clear about what each test state is.
Specific about the expected behaviour.
What more do you need from a test name?
Contrary to Ray's answer I don't think the Test prefix is necessary. It's test code, we know that. If you need to do this to identify the code, then you have bigger problems, your test code should not be mixed up with your production code.
As for length and use of underscore, its test code, who the hell cares? Only you and your team will see it, so long as it is readable, and clear about what the test is doing, carry on! :)
That said, I am still quite new to testing and blogging my adventures with it :)
This is also worth a read: Structuring Unit Tests
The structure has a test class per class being tested. That’s not so unusual. But what was unusual to me was that he had a nested class for each method being tested.
e.g.
using Xunit;
public class TitleizerFacts
{
public class TheTitleizerMethod
{
[Fact]
public void NullName_ReturnsDefaultTitle()
{
// Test code
}
[Fact]
public void Name_AppendsTitle()
{
// Test code
}
}
public class TheKnightifyMethod
{
[Fact]
public void NullName_ReturnsDefaultTitle()
{
// Test code
}
[Fact]
public void MaleNames_AppendsSir()
{
// Test code
}
[Fact]
public void FemaleNames_AppendsDame()
{
// Test code
}
}
}
And here is why:
Well for one thing, it’s a nice way to keep tests organized. All the
tests (or facts) for a method are grouped together. For example, if
you use the CTRL+M, CTRL+O shortcut to collapse method bodies, you can
easily scan your tests and read them like a spec for your code.
I also like this approach:
MethodName_StateUnderTest_ExpectedBehavior
So perhaps adjust to:
StateUnderTest_ExpectedBehavior
Because each test will already be in a nested class
I tend to use the convention of MethodName_DoesWhat_WhenTheseConditions so for example:
Sum_ThrowsException_WhenNegativeNumberAs1stParam
However, what I do see a lot is to make the test name follow the unit testing structure of
Arrange
Act
Assert
Which also follows the BDD / Gherkin syntax of:
Given
When
Then
which would be to name the test in the manner of: UnderTheseTestConditions_WhenIDoThis_ThenIGetThis
so to your example:
WhenNegativeNumberAs1stParam_Sum_ThrowsAnException
However I do much prefer putting the method name being tested first, because then the tests can be arranged alphabetically, or appear alphabetically sorted in the member dropdown box in VisStudio, and all the tests for 1 method are grouped together.
In any case, I like separating the major sections of the test name with underscores, as opposed to every word, because I think it makes it easier to read and get the point of the test across.
In other words, I like: Sum_ThrowsException_WhenNegativeNumberAs1stParam better than Sum_Throws_Exception_When_Negative_Number_As_1st_Param.
I do name my test methods like other methods using "PascalCasing" without any underscores or separators. I leave the postfix Test for the method out, cause it adds no value. That the method is a test method is indicated by the attribute TestMethod.
[TestMethod]
public void CanCountAllItems() {
// Test the total count of items in collection.
}
Due to the fact that each Test class should only test one other class i leave the name of the class out of the method name. The name of the class that contains the test methods is named like the class under test with the postfix "Tests".
[TestClass]
public class SuperCollectionTests(){
// Any test methods that test the class SuperCollection
}
For methods that test for exceptions or actions that are not possible, i prefix the test method with the word Cannot.
[TestMethod]
[ExpectedException(typeOf(ArgumentException))]
public void CannotAddSameObjectAgain() {
// Cannot add the same object again to the collection.
}
My naming convension are base on the article "TDD Tips: Test Naming Conventions & Guidelines" of Bryan Cook. I found this article very helpful.
The first set of names is more readable to me, since the CamelCasing separates words and the underbars separate parts of the naming scheme.
I also tend to include "Test" somewhere, either in the function name or the enclosing namespace or class.
As long as you follow a single practice, it doesn't really matter. Generally, I write a single unit test for a method that covers all the variations for a method (I have simple methods;) and then write more complex sets of tests for methods that require it. My naming structure is thus usually test (a holdover from JUnit 3).
I use a 'T' prefix for test namespaces, classes and methods.
I try to be neat and create folders that replicate the namespaces, then create a tests folder or separate project for the tests and replicate the production structure for the basic tests:
AProj
Objects
AnObj
AProp
Misc
Functions
AFunc
Tests
TObjects
TAnObj
TAnObjsAreEqualUnderCondition
TMisc
TFunctions
TFuncBehavesUnderCondition
I can easily see that something is a test, I know exactly what original code it pertains to, (if you can't work that out, then the test is too convoluted anyway).
It looks just like the interfaces naming convention, (I mean, you don't get confused with things starting with 'I', nor will you with 'T').
It's easy to just compile with or without the tests.
It's good in theory anyway, and works pretty well for small projects.