Related
I am trying to learn TDD and unit testing concepts and I have seen the mantra: "red, green, refactor." I am curious about why should you refactor your code after the tests pass?
This makes no sense to me, because if the tests pass, then why are you messing with the code? I also see TDD mantras like "only write enough code to make the test pass."
The only reason I could come up with, is if to make the test pass with green, you just sloppily write any old code. You just hack together a solution to get a passing test. Then obviously the code is a mess, so you can clean it up.
EDIT:
I found this link on another stackoverflow post which I think confirms the only reason I came up with, that the original code to 'pass' the test can be very simple, even hardcoded: http://blog.extracheese.org/2009/11/how_i_started_tdd.html
Usually the first working version of the code - even if not a mess - still can be improved. So you improve it, making it cleaner, more readable, removing duplication, finding better variable/method names etc. This is refactoring. And since you have the tests, you can refactor safely, because the tests will show if you have inadvertently broken something.
Note that usually you are not writing code from scratch, but modifying/extending existing code to add/change functionality. And the existing code may not be ready to accommodate the new functionality seamlessly. So the first implementation of the new functionality may look awkward or inconvenient, or you may see that it is difficult to extend further. So you improve the design to incorporate all existing functionality in the simplest, cleanest possible way while still passing all the tests.
Your question is a rehash of the age old "if it works, don't fix it". However, as Martin Fowler explains in Refactoring, code can be broken in many different ways. Even if it passes all the tests, it can be hard to understand, thus hard to extend and maintain. Moreover, if it looks sloppy, future programmers will take even less care to keep it tidy, so it will deteriorate ever quicker, and eventually degrades into a complete unmaintainable mess. To prevent this, we refactor to always keep the code clean and tidy as much as possible. If we (or our predecessors) have already let it become messy, refactoring is a huge effort with no obvious immediate benefit for management and stakeholders; thus they can hardly be convinced to support a large scale refactoring in practice. Therefore we refactor in small, even trivial steps, after every code change.
I have seen the mantra: "red, green, refactor."
it's not a 'mantra', it's a routine.
I also see TDD mantras like "only write enough code to make the test pass."
That's a guideline.
now your question:
The only reason I could come up with, is if to make the test pass with green, you just sloppily write any old code. You just hack together a solution to get a passing test. Then obviously the code is a mess, so you can clean it up.
You're almost there. The key is in the 'Design' part of TDD. You're not only coding, you're still designing your solution. That means that the exact API might not be set in stone still, and your tests might not reflect the final design (because it's not done yet). While coding "only enough to pass the test", you will hit some issues that might change your mind and guide the design. Only after you have some working code you're able to improve it.
Also, the refactor step involves the whole code, not only what you've just written to pass the last test. As the coding advances, you have more and more complex interactions between all parts of your code, the best time to refactor it is as soon as it's working.
Precisely because of this very early refactoring step, you shouldn't worry about the quality of the first iteration. it's just a proof of concept that helps in the design.
It's hard to see how the OP's skepticism isn't justified. TDD's workflow is rooted in the avoidance of premature design decisions by imposing a significant cost, if not precluding, 'seat of the pants' coding that could quickly devolve into an ill-advised YAGNI safari.[1]
The mechanism for this deferral of premature design is the 'smallest possible test'/'smallest possible code' workflow that is designed to stave off the temptation to 'fix' a perceived shortcoming or requirement before it would ordinarily need to be addressed or even encountered, i.e, presumably the shortcoming would (ought?) to be addressed in some future test case mapped directly to an acceptance criteria that in turn captures a particular business objective.
Furthermore, tests in TDD are supposed to a) help clarify design requirements, b) surface problems with a design[2], and c) serve as project assets that capture and document the effort applied to a particular story, so substituting a self-directed refactoring effort for a properly composed test not only precludes any insight the test might provide but also denies management and project planners information on the true cost of implementing a particular feature.[3]
Accordingly, I would suggest that a new test case, purpose built for introducing an additional requirement into the design, is the proper way to address any perceived shortcoming beyond a stylistic change to the current code under test, and the 'Refactor' phase, however well-intentioned, flies in the face of this philosophy, and is in fact an invitation to do the very sort of premature, YAGNI design safaris that TDD is supposed to prevent. I believe that Robert Martin's version of the 3 rules is consistent with this interpretation. [4 - A blatant appeal to authority]
[1] The previously cited http://blog.extracheese.org/2009/11/how_i_started_tdd.html elegantly demonstrates the value of deferring design decisions until the last possible moment. (Although perhaps the Fibonacci sequence is a somewhat artificial example).
[2] See https://blog.thecodewhisperer.com/permalink/how-a-smell-in-the-tests-points-to-a-risk-in-the-design
[3] Adding a "tech" or "spike" story (smell or not) to the backlog would be the appropriate method for ensuring that formal processes are followed and development effort is documented and justified... and if you can't convince the Product Owner to add it, then you shouldn't be throwing time at it.
[4] http://www.butunclebob.com/ArticleS.UncleBob.TheThreeRulesOfTdd
Because you should never refactor non-working code. If you do, then you won't know whether the errors were originally in there or due to your refactoring. If they all pass before refactoring, then fail, then you know the change you did broke something.
They don't mean to write any sloppy old code to pass a test. There is a difference between minimal and sloppy. A zen garden is minimal, but not sloppy.
However, the minimal changes you made here and there, might, in retrospect, be better combined into some other procedure that is called by both of them. After getting both tests working separately is the time to refactor. It's easier to refactor than to try and guess an architecture that's going to minimally cover all the test cases.
You make the code behave correctly first, then factor it well. If you do it the other way around you run the risk of making a mess/duplication/code smells while fixing it.
It's usually easier to restructure working code into well factored code than it is to try and design well factored code upfront.
The reason for refactoring working code is for maintenance. You want to remove duplication for reasons such as only having to fix something in one place, and also knowing that when you fix something somewhere you haven't missed the same bug in the similar code elsewhere. You want to rename vars, methods, classes if their meaning has changed from what you originally intended.
Overall, writing working code is non-trivial, and writing well factored code is non-trivial. If you are trying to do both at once you may do neither to your full potential, so giving full attention to one first and then the other is useful.
Iterative, Evolutionary Refactoring is a good approach, but first...
Somethings that should not go unsaid...
To build on top of some high-level notes above, you should understand some important concepts from Complex Systems Theory. The key concepts to note circumvolve a system's environmental structure, how a systems grows, how it behaves, and how its components interact.
Sensitive Dependence Upon Initial Conditions (Chaos Theory):
A system's behavior will be amplified toward its most influential tendency -- meaning, if you've many Broken Windows which influence how a developer will write the next module or interact with an existing one, then this developer is more likely to break another window. Its even tempting to break a window just because its the only one not broken.
Entropy:
There are many, many definitions of entropy out there; one that I find becoming to Software Engineering is: The amount of energy in a system which cannot be used for additional work. This is why reusability is crucial. Entropy is found mostly in terms of duplicate logic and comprehensibility. Furthermore, this ties closely back to the Butterfly Effect (Sensitive Dependence Upon Initial Conditions) and Broken Windows -- the more duplicate logic, the more CopyPaste for additional implementations and it is more than 1X per implementation to maintain it all.
Variable Amplification and Dampening (Emergence Theory and Network Theory):
Breaking a bad design is a good implementation, though it seems all hell breaks loose when it happens the first few times. This is why it is sensible to have an Architecture which can support many adaptations. As your system heads toward entropy, you need a way for modules to interact with each other correctly -- this is where Interfaces come in. If each of your modules cannot interact unless they've agreed to a consistent contract. Without this, you'll see your system immediately start adapting to poor implementations -- and whichever wheel is the squeakiest will get the oil; the other modules will become a headache. So, not only do bad implementations cause more bad implementations, they also create undesirable behavior at the System's Scale -- causing your system, at large, to adapt to varying implementations and amplifying entropy at the highest scale. When this happens, all you can do is keep patching and hope that one change will not conflict with these adaptations -- causing emergent, unpredictable bugs.
The key to all this is to envelop your modules into their own, discrete subsystems, and provide a Defined Architecture which can allow them to communicate -- such as a Mediator. This brings a collection of (Decoupled) behavior into a Bottom-Up System which can then focus its complexity into a component designed exactly for it.
With this type of architectural approach, you shouldn't have significant pain on the 3rd term of "Red, Green, Refactor". The question is, how can your scrum master measure this in terms of benefit to the user & stakeholders?
You should not take the "only write enough code to make the test pass." mantra too literal.
Remember your application isn't ready just because all your tests passes. You clearly would like to refactor your code after tests passes to make sure the code is readable and well architechted. The tests are there to help you refactor so refactoring is a big part of TDD.
First, thanks for taking a look into Test Driven Development. It is an awesome technique that can be applied to many coding situations that can help you develop some great code while also giving you confidence in what the code can and can't do.
If you look at subtitle on the cover of Martin Fowler's book "Refactoring" it also answers your question - "Improving the Design Of Existing Code"
Refactorings are transformations to your code that should not alter the program's behavior.
By refactoring, you can make the program easier to maintain now, and 6 months from now, and it can also make the code easier for the next developer to understand.
I am very new to test-driven development (TDD), not yet started using it.
But I know that we have to write tests first and then the actual code to pass the test and refactor it till the design is good.
My concern over TDD is where it fits in our systems development life cycle (SDLC).
Suppose I get a requirement of making an order processing system.
Now, without having any model or design for this system, how can I start writing tests?
Shouldn't we require to define the entities and their attributes to proceed?
If not, is it possible to develop a big system without any design?
There is two levels of TDD, ATDD or acceptance test driven development, and normal TDD which is driven by unit tests.
I guess the relationship between TDD and design is influenced by the somewhat "agile" concept that source code IS the design of a software product. A lot of people reinforce this by translating TDD as Test Driven Design rather than development. This makes a lot of sense as TDD should be seen as having a lot more to do with driving the design than testing. Having acceptance and unit tests at the end of it is a nice side effect.
I cannot really say too much about where it fits into your SDLC without knowing more about it, but one nice workflow is:
For every user story:
Write acceptance tests using a tool like FitNesse or Cucumber, this would specify what the desired outputs are for the given inputs, from a perspective that the user understands. This level automates the specifications, or can even replace specification documentation in ideal situations.
Now you will probably have a vague idea of the sort of software design you might need as far as classes / behaviour etc goes.
For each behaviour:
Write a failing test that shows how calling code you would like to use the class.
Implement the behaviour that makes the test pass
Refactor both the test and actual code to reflect good design.
Go onto the next behaviour.
Go onto the next user story.
Of course the whole time you will be thinking of the evolving high level design of the system. Ideally TDD will lead to a flexible design at the lower levels that permits the appropriate high design to evolve as you go rather than trying to guess it at the beginning.
It should be called Test Driven Design, because that is what it is.
There is no practical reason to separate the design into a specific phase of the project. Design happens all the time. From the initial discussion with the stakeholder, through user story creation, estimation, and then of course during your TDD sessions.
If you want to formalize the design using UML or whatever, that is fine, just keep in mind that the code is the design. Everything else is just an approximation.
And remember that You Aren't Gonna Need It (YAGNI) applies to everything, including design documents.
Writing test first forces you to think first about the problem domain, and acts as a kind of specification. Then in a 2nd step you move to solution domain and implement the functionality.
TDD works well iteratively:
Define your initial problem domain (can be small, evolutionary prototype)
Implement it
Grow the problem domain (add features, grow the prototype)
Refactor and implement it
Repeat step 3.
Of course you need to have a vague architectural vision upfront (technologies, layers, non-functional requirement, etc.). But the features that bring added-value to your your application can be introduced nicely with TDD.
See related question TDD: good for a starter?
With TDD, you don't care much about design. The idea is that you must first learn what you need before you can start with a useful design. The tests make sure that you can easily and reliably change your application when the time comes that you need to decide on your design.
Without TDD, this happens: You make a design (which is probably too complex in some areas plus you forgot to take some important facts into account since you didn't knew about them). Then you start implementing the design. With time, you realize all the shortcomings of your design, so you change it. But changing the design doesn't change your program. Now, you try to change your code to fit the new design. Since the code wasn't written to be changed easily, this will eventually fail, leaving you with two designs (one broken and the other in an unknown state) and code which doesn't fit either.
To start with TDD, turn your requirements into test. To do this, ask "How would I know that this requirement is fulfilled?" When you can answer this question, write a test that implements the answer to this question. This gives you the API which your (to be written) code must adhere to. It's a very simple design but one that a) always works and b) which is flexible (because you can't test unflexible code).
Also starting with the test will turn you into your own customer. Since you try hard to make the test as simple as possible, you will create a simple API that makes the test work.
And over time, you'll learn enough about your problem domain to be able to make a real design. Since you have plenty of tests, you can then change your code to fit the design. Without terminally breaking anything on the way.
That's the theory :-) In practice, you will encounter a couple of problems but it works pretty well. Or rather, it works better than anything else I've encountered so far.
Well of course you need a solid functional analysis first, including a domain model, without knowing what you'll have to create in the first place it's impossible to write your unit tests.
I use a test-driven development to program and I can say from experience it helps create more robust, focussed and simpler code. My recipe for TDD goes something likes this:
Using a unit-test framework (I've written my own) write code as you wish to use it and tests to ensure return values etc. are correct. This ensures you only write the code you're actually going to use. I also add a few more tests to check for edge cases.
Compile - you will get compiler errors!!!
For each error add declarations until you get no compiler errors. This ensures you have the minimum declarations for your code.
Link - you will get linker errors!!!
Write enough implementation code to remove the linker errors.
Run - you unit tests will fail. Write enough code to make the test succeed.
You've finished at this point. You have written the minimum code you need to implement your feature, and you know it is robust because of your tests. You will also be able to detect if you break things in the future. If you find any bugs, add a unit test to test for that bug (you may not have thought of an edge case for example). And you know that if you add more features to your code you won't make it incompatible to existing code that uses your feature.
I love this method. Makes me feel warm and fuzzy inside.
TDD implies that there is some existing design (external interface) to start with. You have to have some kind of design in mind in order to start writing a test. Some people will say that TDD itself requires less detailed design, since the act of writing tests provides feedback to the design process, but these concepts are generally orthogonal.
You need some form of specification, rather than a form of design -- design is about how you go about implementing something, specification is about what you're going to implement.
Most common form of specs I've seen used with TDD (and other agile processes) are user stories -- an informal kind of "use case" which tends to be expressed in somewhat stereotyped English sentences like "As a , I can " (the form of user stories is more or less rigid depending on the exact style/process in use).
For example, "As a customer, I can start a new order", "As a customer, I can add an entry to an existing order of mine", and so forth, might be typical if that's what your "order entry" system is about (the user stories would be pretty different if the system wasn't "self-service" for users but rather intended to be used by sales reps entering orders on behalf of users, of course -- without knowing what kind of order-entry system is meant, it's impossible to proceed sensibly, which is why I say you do need some kind of specification about what the system's going to do, though typically not yet a complete idea about how it's going to do it).
Let me share my view:
If you want to build an application, along the way you need to test it e.g check the values of variables you create by code inspection, of quickly drop a button that you can click on and will execute a part of code and pop up a dialog to show the result of the operation etc. on the other hand TDD changes your mindset.
Commonly, you just rely on the development environment like visual studio to detect errors as you code and compile and somewhere in your head, you know the requirement and just coding and testing via button and pop ups or code inspection. this is a Syntax debugging driven development . but when you are doing TDD, is a "semantic debugging driven development " because you write down your thoughts/ goals of your application first by using tests (which and a more dynamic and repeatable version of a white board) which tests the logic (or "semantic") of your application and fails whenever you have a semantic error even if you application passes syntax error (upon compilation).
In practice you may not know or have all the information required to build the application , since TDD kind of forces you to write tests first, you are compelled to ask more questions about the functioning of the application at a very early stage of development rather than building a lot only to find out that a lot of what you have written is not required (or at lets not at the moment). you can really avoid wasting your precious time with TDD (even though it may not feel like that initially)
A new project we began introduced a lot of new technologies we weren't so familiar with, and an architecture that we don't have a lot of practice in. In other words, the interfaces and interactions between service classes etc of what we're building are fairly volatile, even more so due to internal and customer feedback. Though I've always been frustrated by the ever-moving specification, I see this to some degree a necessary part of building something we've never built before - if we just stuck to the original design and scope, the end product would probably be a whole lot less innovative and useful than it's becoming.
I also introduced test-driven development (TDD), as the benefits are well-documented and conceptually I loved the idea. Two more new things to learn - NUnit and mocking - but seeing all those green circles made it all worthwhile.
Over time, however, those constant changes in design seemed to mean I was spending a whole lot more time changing my tests than I was on writing the code itself. For this reason alone, I've gone back to the old ways of testing - that is, not automated.
While I have no doubt that the application would be far more robust with hundreds of excellent unit tests, I've found the trade-off of time to launch the product to be mostly unacceptable. My question is, then - have any of you also found that TDD can be a hassle if you're prototyping something / building a beta version? Does TDD go much more naturally hand-in-hand with something where the specifications are more fixed, or where the developers have more experience in the language and technologies? Or have I done something fundamentally wrong?
Note that I'm not trying to criticise TDD here - just I'm not sure it's always the best fit for all situations.
The short answer is that TDD is very valuable for beta versions, but may be less so for prototyping.
I think it is very important to distinguish between beta versions and prototyping.
A beta version is essentially a production version that is just still in development, so you should definitely use TDD in that scenario.
A prototype/proof of concept is something you build with the express intent of throwing it away once you've gotten the answers out of it that you wanted.
It's true that project managers will tend to push for the prototype to be used as a basis for production code, but it is very important to resist that. If you know that's not possible, treat the prototype code as you would your production code, because you know it is going to become your production code in the future - and that means you should use TDD with it as well.
When you are learning a new technology, most code samples etc. are not written with unit tests in mind, so it can be difficult to translate the new technology to the unit testing mindset. It most definitely feels like a lot of overhead.
In my experience, however, unit testing often really forces you to push the boundaries of the new technology that you are learning. Very often, you need to research and learn all the different hooks the new technology provides, because you need to be able to isolate the technology via DI or the like.
Instead of only following the beaten path, unit testing frequently forces you to learn the technology in much more depth, so what may feel like overhead is actually just a more in-depth prototype - one that is often more valuable, because it covers more ground.
Personally, I think unit testing a new technology is a great learning tool.
The symptoms you seem to experience regarding test maintainability is a bit orthogonal, I think. Your tests may be Overspecified, which is something that can happen just as well when working with known technologies (but I think it is probably easier to fall into this trap when you are also learning a new technology at the same time).
The book xUnit Test Patterns describes the Overspecified Test antipattern and provides a lot of guidance and patterns that can help you write more maintainable tests.
I've found that thoroughly testing early results in lots of code thrown away and an empty feeling in the pit of your stomach.
Test what needs to be tested and not a line of code more. When you figure out how much that is, let me know.
When prototyping, I would say it depends on the type of prototyping. In evolutionary prototyping, where the prototype evolves into the final application, I would utilize unit testing as early as possible. If you are using throw-away prototyping, I wouldn't bother with unit testing - the final application is going to be nothing like the prototype.
I'm not sure what you mean by "beta", since a beta is almost a finished product. As soon as you start working on code that is going to be shipped, or has a potential to be shipped, make sure everything is well tested.
Now, pure test-driven development might be extreme, but it is important to make sure that all shippable code is as tested as possible, at the unit, integration, and system level.
have any of you also found that TDD can be a hassle if you're prototyping something / building a beta version?
I have.. Tons of times :)
Does TDD go much more naturally hand-in-hand with something where the specifications are more fixed, or where the developers have more experience in the language and technologies?
Not really. TDD works quite nice with changing requirements, but TDD is really for ensuring a stable and contract-driven design: Two things which prototypes doesn't really need that badly..
Or have I done something fundamentally wrong?
Doesn't look like it :) You've just seen that TDD consists of other things than golden trees..
"Over time, however, those constant changes in design seemed to mean I was spending a whole lot more time changing my tests than I was on writing the code itself"
Good. You should spend a lot of time on testing. It's important, and it's how you demonstrate that your code is right. "Less code than test" is a good benchmark.
That means that you were writing effective tests that demonstrated your expectations for the underlying technology.
You may want to consider doing this.
Some tests are "essential" or "core" or "enduring" features of the application independent of any technology choices. Focus on these. The should never change.
Some tests confirm the technology or implementation choices. These change all the time. Perhaps you should segregate these so that the technology changes lead to focused changes here.
Being given a roadmap with a moving X is frustrating.
TDD or no TDD.. 'having to spend majority of the time changing the tests instead of the code' indicates either that the specs were changed radically or you just over-mocked yourself a.k.a "fragile tests". I'd need more input from you to comment further.
Spike/Prototyping means trying to build something as a proof of concept or validation a design. With this definition, I'd say that you don't need to TDD your proto because your primary goal is learning / reducing the unknowns. Once you've done that you should throw away your proto and build your production version with automated tests (use TDD now). You ship these to the customer not 'tested prototypes'
However if manual testing has been working well for you, persist with it. I like to prove to myself and others that my code works at the push of a button - avoid human boredom of repetitive tasks and get thorough testing.
Shipping Prototypes will bite you sooner and harder than you ever imagine. Take it from me.
Prototyping is meant to be the used for "Would this kind of thing work"-exploration.
So there is no need for Testing. BUT! Always throw your Prototype away and code from ground zero!
In agile development there's the concept of a "spike" - a deep but narrow investigation of a solution or a technology. Once you're comfortable with how things should work you start over with a new implementation with a higher quailty level.
There's a pitfall with software labeled as "beta" - all of the sudden you end up with something not intended for production as a vital part of your application and you haven't got the time to redo it. This will most likely come back and bite you later on. A protoype should really be just a prototype - no more, no less.
I don't have a prescription for you, but I do have a diagnosis:
if you ever end up in a debugger, you shoulda had more tests. Even in the very earliest prototypes, you want the code you write to work, right? If I don't do TDD, and I run in to a bug, it's hard to retrofit unit tests to find the bug. It's tempting to go to the debugger. So, I aim my TDD efforts to produce a good enough set of tests so that I don't need the debugger. Doing this well requires lots of practice.
If you want to refactor but don't because of risk, you shoulda had more tests. If I'm going to be working with some code for more than a couple hours, I want to refactor to keep my velocity high. If I hesitate to refactor, it's gonna hurt my productivity, so I do everything I can to make refactoring safe and easy. Again, a good selection of tests is exactly what I need.
For pure prototyping, as people have said, not so useful. But often a prototype has elements of the application that will follow.
As you build it, what are the solid concepts that you expect to continue? What are the key domain models? It makes sense to build tests around these. Then, they provide a solid base on which to explore ideas. By having part of the code base solid and tested, it allows you to go further with the prototype than you could with no tests.
It's always a balance of picking the right things to test when. It sounds like from your description you are adopting quite a few new things at once-- sometimes this can be too much, and you need to retreat a little bit to optimize your progress.
What I typically do for this prototyping code is to write it without tests (or not many), but do the work under my src/test/java or wherever I'm putting my test code. That way, I won't inadvertently put it into production. If there's something I've prototyped that I like, then I'll create something in src/main/java (my prod code) and TDD it, pulling over code from the prototype one piece at a time as I add tests.
Over time, however, those constant changes in design seemed to mean I was spending a whole lot more time changing my tests than I was on writing the code itself.
I write (and run) automated tests of the system's I/O. The system's I/O depends on the functional requirements, and doesn't change when the system's implementation changes, so I can refactor the implementation without changing the automated tests: Should one test internal implementation, or only test public behaviour?
When I get in contact with a new technology, I usually write a couple of tests which I use as the basis of the prototype. The main advantage here is that it allows me to get used to the technology in small, digestible parts and that I document my progress in the only valid form of documentation: code.
That way, I can test assumptions (what happens when I persist an empty string in a database? DB2 will return an empty string, Oracle will return NULL) and make sure they don't change while I work on the prototype.
Think of this work as many, tiny prototypes all rolled into a couple of unit tests.
Now the design of my product will change over time but these spike tests are still valid: The underlying framework doesn't change just because I now pursue a different solution. In fact, these tests will allow me to migrate to the next version of the technology since they will ring an alarm when one of the assumptions I've been using has changed. Otherwise, I'd have to stay with the current version or be very careful with any upgrade since I couldn't be sure what it would break.
I would be cautious about dismissing TDD when prototyping. Primarily because prototypes tend to (but not always) evolve into the final product. If you can be sure that the prototypes are thrown out once they have established what ever it was they were started for, then you can be a bit more lax. If however it there is a chance that either the prototype will evolve into the final product, or parts of the code will be ported into the final product, then would endeavor to follow TDD principles.
Not only does this make your code more testable, but more importantly (in my mind) it encourages you to follow good software design principles.
If it's actually a prototype and you are going to throw it away when you're done, then take whatever route gets you through that phase. But, ask yourself, how often do your prototypes actually get thrown away vs. how often do they creep into you're final product?
There was a good presentation at JavaOne on TDD. One of the really nice outcomes was that you tended to understand your actual requirements much, much better when you had to author tests to meet them. This is in addition to all of the refactoring and code quality benefits that came along with the approach.
Let me first come out of closet. I'm a TDD believer. I'm trying to practice Test Driven Development as much as I can.
Some developers at my work refuse to even try it. I myself started TDD by trying to prove to one of my peers that Test Driven Development is a bad idea. The arguments are:
Why? I was pretty successful developer so far.
It's going to slow me down.
What's the best pro TDD argument did hear or used?
See also: What is the best reason for unit testing?
Perhaps they know better.
Unit testing by developers is an extremely useful practice and I cannot overemphasize its benefits, not only during initial development but also during refactoring when unit tests can catch early not only ordinary code defects but also the break of assumptions made by developers that were never captured in formal documentation and thus are likely lost by the time refactoring occurs.
That being said, TDD is no magic pixie dust:
the 'just write enough code to pass the test' approach gives false positives. There are often known fallacies and problems that the 'just enough' approach fails to address. Quick examples that come to mind are distributed systems fallacies or NUMA performance problems. Just capturing those requirements into simply expressing those test cases for TDD would turn into a full time job in itself.
the explosion of moqs goes out of control for any serious size project. mocks are code like any other code, they need to be maintained and just don't write themselves out of the blue.
TDD is often used as an excuse to eliminate QA testing. 'our developer have already written tested id, lets ship it' neglects completely the end-to-end feature oriented testing QA should cover
I don't trust the fox guarding the hen house. A wrong algorithm can still pass TDD with flying colors if the same mistakes are made in both the test and in the implementation.
All methodologies in the end try to use process to substitute talent.
My main quarrel with TDD is that is presented as a magic solution to most development problems but its cost is kept under the table by its advocates. Doubling or tripling your code base with moqs does not come for free. I much rather see a few comprehensive unit tests written during development. The test-first TDD approach, I'm yet to see its benefits in a real size project.
I understand I'll be egg-ed to death now for posting this, but what the heck, who cares...
No amount of argument will convince anyone to use TDD.
You have to SHOW them, and demonstrate the benefits. It's easier to make someones 'light go on' by showing rather than telling.
TDD is a "pay me now or pay me later" tradeoff. If you only count the time from starting coding to checking in your code then TDD often does take longer, especially when first learning TDD. The payoff comes later during the testing phase, and also in future rounds of coding.
For the testing phase, I found that with TDD:
I had substantially fewer bugs. My last TDD code I had bugs only due to requirements misunderstandings (or changes) or in the areas where I wasn't able to bring the code under test (PHP code in that case).
The bugs I had were generally easier to reproduce under test, because I had already gotten the system under test.
Fixing the bugs was faster, and with the tests I could have a greater belief that I didn't introduce new bugs.
The code itself had the following properties:
As I started out thinking like a client of the code, the code tended to be easier to use. (This is one of the benefits of writing tests first).
The code is easier to test.
Writing unit tests is easier (and in many cases more fun) just before rather than after, so more tests are written.
The code is easier to refactor and clean up. This was particularly true with Python, where automatic refactoring tools have a harder time.
Because of that, when it came time to revisit the code, it was easier to understand and easier to change, plus we had at least some regression tests already in place.
What this means is that the payback for TDD time may be months later. Furthermore, starting TDD with legacy code is particularly hard. Then there is time needed to both learn how to write good tests (a bad test set can either be insufficient or worse be brittle making it harder, not easier, to do refactorings) and how to get a complex system under test.
I have to admit I haven't been really able to get too many other people to switch to TDD. I think I switched largely because I wanted an easier way of testing and also I had the opportunity to learn how with a small code base and personal project.
Different people will be convinced (or not) in different ways, so the only honest answer is "it depends".
One way I've seen work several times is to sit with someone after they've been struggling with a chunk of code, and recreate it using TDD. The resulting code is usually smaller and clearer.
I don't practice TDD. Although I see how it is good if you have complex projects in which you have many different test cases to test, I don't see a great benefit in using it in, say, a simple web application.
One way someone could convince me to use TDD would be if we took the same project and did them side by side, see who comes up with better results and who completes the task faster.
Pair with them. You don't have to call it "pair programming" - that's scary to someone who's reluctant to even consider "radical" techniques like TDD - but if the two of you sit at a desk and work together on the same problem, it's easy to demonstrate the value of TDD. That can't be the end of the conversation, but it's one hell of a start. Gives you credibility for the rest of the conversation, and gives you something real as a basis for further discussion.
The "aha" moment for me was reading chapter 2 of "Test-Driven Development in Microsoft.Net" by James Newkirk. (Not that the rest of the book wasn't important...he dedicates several chapters to building a multi-tiered application in TDD).
He builds a simple stack, but you get to see the code "evolve" its complexity instead of starting out complex.
Even then, you will still have trouble convincing nay-sayers because it appears that TDD requires a lot more work than traditional programming. Most anti-TDD developers, however, forget to factor in the development time for unit tests at the end, at least in my experience.
The arguments you listed are not rational, logical arguments. They have no reasoning behind them (unless you've actually just summarized much longer real arguments.)
As such, I don't think that you will be able to convince anyone who makes those claims with rational arguments of your own. The best way will be to appeal to the source of their arguments; experience. Either get them to use TDD for a while on a provisional basis to see what they think of it, or else do TDD work yourself that is clearly very good work, and present it as an example to them.
(I'm not a TDD believer. This is a practical way you could convince me that it was a good idea.)
As a professional developer for 10+ years, the best argument I can put forward is that even I found my bugs before I got to a point of actually being able to "run" the application.
I also found that the design of my code was more robust and easier to change, and it gave me greater confidence to refactor.
"Pretty successful" doesn't equal "Really successful".
The other great advantage is that I don't have to write test harnesses anymore as the Unit Test runners effectively become my test harness.
Show them this presentation. It sold me.
http://www.slideshare.net/sebastian_bergmann/testing-with-phpunit-and-selenium-156187
Any programmer who's ever been faced with a really complex task with a lot of edge conditions should be able to see the value of TDD. When it comes to something like making sure a search engine will match certain strings, TDD is the only way you'll be able to stay sane during maintenance -- the only way to be sure you've fixed one case without breaking a few others is with automated testing.
Thorough unit tests reduce bugs occurrences, but they also reduce recidivation or the scope of damage caused by recidivation.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
What do I lose by adopting test driven design?
List only negatives; do not list benefits written in a negative form.
If you want to do "real" TDD (read: test first with the red, green, refactor steps) then you also have to start using mocks/stubs, when you want to test integration points.
When you start using mocks, after a while, you will want to start using Dependency Injection (DI) and a Inversion of Control (IoC) container. To do that you need to use interfaces for everything (which have a lot of pitfalls themselves).
At the end of the day, you have to write a lot more code, than if you just do it the "plain old way". Instead of just a customer class, you also need to write an interface, a mock class, some IoC configuration and a few tests.
And remember that the test code should also be maintained and cared for. Tests should be as readable as everything else and it takes time to write good code.
Many developers don't quite understand how to do all these "the right way". But because everybody tells them that TDD is the only true way to develop software, they just try the best they can.
It is much harder than one might think. Often projects done with TDD end up with a lot of code that nobody really understands. The unit tests often test the wrong thing, the wrong way. And nobody agrees how a good test should look like, not even the so called gurus.
All those tests make it a lot harder to "change" (opposite to refactoring) the behavior of your system and simple changes just becomes too hard and time consuming.
If you read the TDD literature, there are always some very good examples, but often in real life applications, you must have a user interface and a database. This is where TDD gets really hard, and most sources don't offer good answers. And if they do, it always involves more abstractions: mock objects, programming to an interface, MVC/MVP patterns etc., which again require a lot of knowledge, and... you have to write even more code.
So be careful... if you don't have an enthusiastic team and at least one experienced developer who knows how to write good tests and also knows a few things about good architecture, you really have to think twice before going down the TDD road.
Several downsides (and I'm not claiming there are no benefits - especially when writing the foundation of a project - it'd save a lot of time at the end):
Big time investment. For the simple case you lose about 20% of the actual implementation, but for complicated cases you lose much more.
Additional Complexity. For complex cases your test cases are harder to calculate, I'd suggest in cases like that to try and use automatic reference code that will run in parallel in the debug version / test run, instead of the unit test of simplest cases.
Design Impacts. Sometimes the design is not clear at the start and evolves as you go along - this will force you to redo your test which will generate a big time lose. I would suggest postponing unit tests in this case until you have some grasp of the design in mind.
Continuous Tweaking. For data structures and black box algorithms unit tests would be perfect, but for algorithms that tend to be changed, tweaked or fine tuned, this can cause a big time investment that one might claim is not justified. So use it when you think it actually fits the system and don't force the design to fit to TDD.
When you get to the point where you have a large number of tests, changing the system might require re-writing some or all of your tests, depending on which ones got invalidated by the changes. This could turn a relatively quick modification into a very time-consuming one.
Also, you might start making design decisions based more on TDD than on actually good design prinicipals. Whereas you may have had a very simple, easy solution that is impossible to test the way TDD demands, you now have a much more complex system that is actually more prone to mistakes.
I think the biggest problem for me is the HUGE loss of time it takes "getting in to it". I am still very much at the beginning of my journey with TDD (See my blog for updates my testing adventures if you are interested) and I have literally spent hours getting started.
It takes a long time to get your brain into "testing mode" and writing "testable code" is a skill in itself.
TBH, I respectfully disagree with Jason Cohen's comments on making private methods public, that's not what it is about. I have made no more public methods in my new way of working than before. It does, however involve architectural changes and allowing for you to "hot plug" modules of code to make everything else easier to test. You should not be making the internals of your code more accessible to do this. Otherwise we are back to square one with everything being public, where is the encapsulation in that?
So, (IMO) in a nutshell:
The amount of time taken to think (i.e. actually grok'ing testing).
The new knowledge required of knowing how to write testable code.
Understanding the architectural changes required to make code testable.
Increasing your skill of "TDD-Coder" while trying to improve all the other skills required for our glorious programming craft :)
Organising your code base to include test code without screwing your production code.
PS: If you would like links to positives, I have asked and answered several questions on it, check out my profile.
In the few years that I've been practicing Test Driven Development, I'd have to say the biggest downsides are:
Selling it to management
TDD is best done in pairs. For one, it's tough to resist the urge to just write the implementation when you KNOW how to write an if/else statement. But a pair will keep you on task because you keep him on task. Sadly, many companies/managers don't think that this is a good use of resources. Why pay for two people to write one feature, when I have two features that need to be done at the same time?
Selling it to other developers
Some people just don't have the patience for writing unit tests. Some are very proud of their work. Or, some just like seeing convoluted methods/functions bleed off the end of the screen. TDD isn't for everyone, but I really wish it were. It would make maintaining stuff so much easier for those poor souls who inherit code.
Maintaining the test code along with your production code
Ideally, your tests will only break when you make a bad code decision. That is, you thought the system worked one way, and it turns out it didn't. By breaking a test, or a (small) set of tests, this is actually good news. You know exactly how your new code will affect the system. However, if your tests are poorly written, tightly coupled or, worse yet, generated (cough VS Test), then maintaining your tests can become a choir quickly. And, after enough tests start to cause more work that the perceived value they are creating, then the tests will be the first thing to be deleted when schedules become compressed (eg. it gets to crunch time)
Writing tests so that you cover everything (100% code coverage)
Ideally, again, if you adhere to the methodology, your code will be 100% tested by default. Typically, thought, I end up with code coverage upwards of 90%. This usually happens when I have some template style architecture, and the base is tested, and I try to cut corners and not test the template customizations. Also, I have found that when I encounter a new barrier I hadn't previously encountered, I have a learning curve in testing it. I will admit to writing some lines of code the old skool way, but I really like to have that 100%. (I guess I was an over achiever in school, er skool).
However, with that I'd say that the benefits of TDD far outweigh the negatives for the simple idea that if you can achieve a good set of tests that cover your application but aren't so fragile that one change breaks them all, you will be able to keep adding new features on day 300 of your project as you did on day 1. This doesn't happen with all those who try TDD thinking it's a magic bullet to all their bug-ridden code, and so they think it can't work, period.
Personally I have found that with TDD, I write simpler code, I spend less time debating if a particular code solution will work or not, and that I have no fear to change any line of code that doesn't meet the criteria set forth by the team.
TDD is a tough discipline to master, and I've been at it for a few years, and I still learn new testing techniques all the time. It is a huge time investment up front, but, over the long term, your sustainability will be much greater than if you had no automated unit tests. Now, if only my bosses could figure this out.
On your first TDD project there are two big losses, time and personal freedom
You lose time because:
Creating a comprehensive, refactored, maintainable suite of unit and acceptance tests adds major time to the first iteration of the project. This may be time saved in the long run but equally it can be time you don't have to spare.
You need to choose and become expert in a core set of tools. A unit testing tool needs to be supplemented by some kind of mocking framework and both need to become part of your automated build system. You also want to pick and generate appropriate metrics.
You lose personal freedom because:
TDD is a very disciplined way of writing code that tends to rub raw against those at the top and bottom of the skills scale. Always writing production code in a certain way and subjecting your work to continual peer review may freak out your worst and best developers and even lead to loss of headcount.
Most Agile methods that embed TDD require that you talk to the client continually about what you propose to accomplish (in this story/day/whatever) and what the trade offs are. Once again this isn't everyone's cup of tea, both on the developers side of the fence and the clients.
Hope this helps
TDD requires you to plan out how your classes will operate before you write code to pass those tests. This is both a plus and a minus.
I find it hard to write tests in a "vacuum" --before any code has been written. In my experience I tend to trip over my tests whenever I inevitably think of something while writing my classes that I forgot while writing my initial tests. Then it's time to not only refactor my classes, but ALSO my tests. Repeat this three or four times and it can get frustrating.
I prefer to write a draft of my classes first then write (and maintain) a battery of unit tests. After I have a draft, TDD works fine for me. For example, if a bug is reported, I will write a test to exploit that bug and then fix the code so the test passes.
Prototyping can be very difficult with TDD - when you're not sure what road you're going to take to a solution, writing the tests up-front can be difficult (other than very broad ones). This can be a pain.
Honestly I don't think that for "core development" for the vast majority of projects there's any real downside, though; it's talked down a lot more than it should be, usually by people who believe their code is good enough that they don't need tests (it never is) and people who just plain can't be bothered to write them.
Well, and this stretching, you need to debug your tests. Also, there is a certain cost in time for writing the tests, though most people agree that it's an up-front investment that pays off over the lifetime of the application in both time saved debugging and in stability.
The biggest problem I've personally had with it, though, is getting up the discipline to actually write the tests. In a team, especially an established team, it can be hard to convince them that the time spent is worthwhile.
The downside to TDD is that it is usually tightly associated with 'Agile' methodology, which places no importance on documentation of a system, rather the understanding behind why a test 'should' return one specific value rather than any other resides only in the developer's head.
As soon as the developer leaves or forgets the reason that the test returns one specific value and not some other, you're screwed. TDD is fine IF it is adequately documented and surrounded by human-readable (ie. pointy-haired manager) documentation that can be referred to in 5 years when the world changes and your app needs to as well.
When I speak of documentation, this isn't a blurb in code, this is official writing that exists external to the application, such as use cases and background information that can be referred to by managers, lawyers and the poor sap who has to update your code in 2011.
I've encountered several situations where TDD makes me crazy. To name some:
Test case maintainability:
If you're in a big enterprise, many chances are that you don't have to write the test cases yourself or at least most of them are written by someone else when you enter the company. An application's features changes from time to time and if you don't have a system in place, such as HP Quality Center, to track them, you'll turn crazy in no time.
This also means that it'll take new team members a fair amount of time to grab what's going on with the test cases. In turn, this can be translated into more money needed.
Test automation complexity:
If you automate some or all of the test cases into machine-runnable test scripts, you will have to make sure these test scripts are in sync with their corresponding manual test cases and in line with the application changes.
Also, you'll spend time to debug the codes that help you catch bugs. In my opinion, most of these bugs come from the testing team's failure to reflect the application changes in the automation test script. Changes in business logic, GUI and other internal stuff can make your scripts stop running or running unreliably. Sometimes the changes are very subtle and difficult to detect. Once all of my scripts report failure because they based their calculation on information from table 1 while table 1 was now table 2 (because someone swapped the name of the table objects in the application code).
If your tests are not very thorough you might fall into a false sense of "everything works" just because you tests pass. Theoretically if your tests pass, the code is working; but if we could write code perfectly the first time we wouldn't need tests. The moral here is to make sure to do a sanity check on your own before calling something complete, don't just rely on the tests.
On that note, if your sanity check finds something that is not tested, make sure to go back and write a test for it.
The biggest problem are the people who don't know how to write proper unit tests. They write tests that depend on each other (and they work great running with Ant, but then all of sudden fail when I run them from Eclipse, just because they run in different order). They write tests that don't test anything in particular - they just debug the code, check the result, and change it into test, calling it "test1". They widen the scope of classes and methods, just because it will be easier to write unit tests for them. The code of unit tests is terrible, with all the classical programming problems (heavy coupling, methods that are 500 lines long, hard-coded values, code duplication) and is a hell to maintain. For some strange reason people treat unit tests as something inferior to the "real" code, and they don't care about their quality at all. :-(
You lose the ability to say you are "done" before testing all your code.
You lose the capability to write hundreds or thousands of lines of code before running it.
You lose the opportunity to learn through debugging.
You lose the flexibility to ship code that you aren't sure of.
You lose the freedom to tightly couple your modules.
You lose option to skip writing low level design documentation.
You lose the stability that comes with code that everyone is afraid to change.
You lose a lot of time spent writing tests. Of course, this might be saved by the end of the project by catching bugs faster.
Refocusing on difficult, unforeseen requirements is the constant bane of the programmer. Test-driven development forces you to focus on the already-known, mundane requirements, and limits your development to what has already been imagined.
Think about it, you are likely to end up designing to specific test cases, so you won't get creative and start thinking "it would be cool if the user could do X, Y, and Z". Therefore, when that user starts getting all excited about potential cool requirements X, Y, and Z, your design may be too rigidly focused on already specified test cases, and it will be difficult to adjust.
This, of course, is a double edged sword. If you spend all your time designing for every conceivable, imaginable, X, Y, and Z that a user could ever want, you will inevitably never complete anything. If you do complete something, it will be impossible for anyone (including yourself) to have any idea what you're doing in your code/design.
You will lose large classes with multiple responsibilities.
You will also likely lose large methods with multiple responsibilities.
You may lose some ability to refactor, but you will also lose some of the need to refactor.
Jason Cohen said something like:
TDD requires a certain organization for your code. This might be architecturally wrong; for example, since private methods cannot be called outside a class, you have to make methods non-private to make them testable.
I say this indicates a missed abstraction -- if the private code really needs to be tested, it should probably be in a separate class.
Dave Mann
The biggest downside is that if you really want to do TDD properly you will have to fail a lot before you succeed. Given how many software companies work (dollar per KLOC) you will eventually get fired. Even if your code is faster, cleaner, easier to maintain, and has less bugs.
If you are working in a company that pays you by the KLOCs (or requirements implemented -- even if not tested) stay away from TDD (or code reviews, or pair programming, or Continuous Integration, etc. etc. etc.).
I second the answer about initial development time. You also lose the ability to confortably work without the safety of tests. I've also been described as a TDD nutbar, so you could lose a few friends ;)
It's percieved as slower. Long term that's not true in terms of the grief it will save you down the road, but you'll end up writing more code so arguably you're spending time on "testing not coding". It's a flawed argument, but you did ask!
It can be hard and time consuming writing tests for "random" data like XML-feeds and databases (not that hard). I've spent some time lately working with weather data feeds. It's quite confusing writing tests for that, at least as i don't have too much experience with TDD.
You have to write applications in a different way: one which makes them testable. You'd be surprised how difficult this is at first.
Some people find the concept of thinking about what they're going to write before they write it too hard. Concepts such as mocking can be difficult for some too. TDD in legacy apps can be very difficult if they weren't designed for testing. TDD around frameworks that are not TDD friendly can also be a struggle.
TDD is a skill so junior devs may struggle at first (mainly because they haven't been taught to work this way).
Overall though the cons become solved as people become skilled and you end up abstracting away the 'smelly' code and have a more stable system.
unit test are more code to write, thus a higher upfront cost of development
it is more code to maintain
additional learning required
Good answers all. I would add a few ways to avoid the dark side of TDD:
I've written apps to do their own randomized self-test. The problem with writing specific tests is even if you write lots of them they only cover the cases you think of. Random-test generators find problems you didn't think of.
The whole concept of lots of unit tests implies that you have components that can get into invalid states, like complex data structures. If you stay away from complex data structures there's a lot less to test.
To the extent your application allows it, be shy of design that relies on the proper ordering of notifications, events and side-effects. Those can easily get dropped or scrambled so they need a lot of testing.
Let me add that if you apply BDD principles to a TDD project, you can alleviate a few of the major drawbacks listed here (confusion, misunderstandings, etc.). If you're not familiar with BDD, you should read Dan North's introduction. He came up the concept in answer to some of the issues that arose from applying TDD at the workplace. Dan's intro to BDD can be found here.
I only make this suggestion because BDD addresses some of these negatives and acts as a gap-stop. You'll want to consider this when collecting your feedback.
It takes some time to get into it and some time to start doing it in a project but... I always regret not doing a Test Driven approach when I find silly bugs that an automated test could have found very fast. In addition, TDD improves code quality.
You have to make sure your tests are always up to date, the moment you start ignoring red lights is the moment the tests become meaningless.
You also have to make sure the tests are comprehensive, or the moment a big bug appears, the stuffy management type you finally convinced to let you spend time writing more code will complain.
The person who taught my team agile development didn't believe in planning, you only wrote as much for the tiniest requirement.
His motto was refactor, refactor, refactor. I came to understand that refactor meant 'not planning ahead'.
Development time increases : Every method needs testing, and if you have a large application with dependencies you need to prepare and clean your data for tests.
TDD requires a certain organization for your code. This might be inefficient or difficult to read. Or even architecturally wrong; for example, since private methods cannot be called outside a class, you have to make methods non-private to make them testable, which is just wrong.
When code changes, you have to change the tests as well. With refactoring this can be a
lot of extra work.