I must admit that I often struggle with practising Test-Driven Development. In spite of using Ruby on Rails which makes TDD super easy because it's baked-in, I find writing tests to be so boring! It's like dental flossing; I know that I should do it but struggle to muster much enthusiasm.
What techniques do you use to make writing tests interesting? For example, one tip I saw was to invent a little story around the test fixture data rather than just using meaningless, unrelated data.
If you write the tests first, they are your specifications for coding.
All the thinking has to be done when writing tests. "What should it do?" "How will I know it's done it?" "What interfaces does it have that will need to be mocked?"
Further, if you structure your tests using a simple naming convention (using "shoulds") you can more easily determine what's supposed to be happening.
See http://weblogs.asp.net/rosherove/archive/2005/04/03/TestNamingStandards.aspx for some thoughts on this.
If you write the tests last, they are boring, since you know the code works.
Writing negative tests is usually more interesting than the "sunny day" ones. Think through all the inventive ways you could break your class (passing in null, values too big/small etc).
Not only will it give your brain a different angle to chew on it'll also make your class more robust since people will call it with null, big numbers etc etc.
I'm worried that this sounds like a code smell.
Are the tests boring because they're very repetitive?
Are the tests covering the same things multiple times? (i.e. the test cases don't just test one thing at a time so there's lots of repeated testing of the same things...?)
You might be bored because the tests are written at the wrong level of abstraction or they force you to do lots of busywork that isn't necessary.
It sounds like something needs to be refactored or at least abstracted so that each test expresses just what's new or different from the rest of the code.
If lots of tests seem obvious or tedious, there's something missing from the abstractions that you're using.
I'd start looking for patterns in the kinds of tests that you feel are boring or tedious and see if something can't be done - like creating a tiny test framework to help make those tests easier to write.
On examination, you might just need to delete some redundant tests and clean up the naming - so that it's clear exactly what you need to test and what you can rely on being tested elsewhere in the test suite.
This is all about trade-offs. I think it's worth re-assessing the kinds of tests you're writing and having a look at what alternatives there might be.
If you're bored when writing tests, then you're testing the wrong things. I'm writing tests when something failed, when I didn't understand something or when something new comes up. This way, my tests are never futile or to comply to an "100% code coverage" policy and I'm never bored.
There's a case to be made that boringness isn't altogether bad. I'd say it's stronger with respect to your regular code than to your test code, but it probably applies to tests, too.
Excitement comes when you don't know what your code is doing, when you don't trust it, when every time you run it - or release it - there's that little guy sitting on your shoulder shrieking "No!". When you spend a lot of time in the debugger; when your code is too complex, tangled and gnarly (not in a good way) and scary.
Boredom may be the opposite of excitement, and seen in that light, boring is good. One foot after the other, predictable step after predictable step, we write nice simple reliable working code. Red-green-refactor.
Nice simple reliable working code is something I can get enthusiastic about.
If you are using TDD correctly, then you should write the test before you write the code. It should be a good test to ensure that the code you are writing works, and should be a small increment.
As such, it is really part of development. What is different from writing one unit test vs. writing one function that you need to implement your code?
Saying that you find writing tests boring, is like saying "I find writing I/O boring .. is there anything I can do to make it more interesting?" or "I find writing UI boring .."
Well, actually writing any kind of code can be boring, or interesting ... but that's more a function of the developer than of the code :) My friend is being forced to write code for a company, although he's not really a programmer, and his comment is "I don't see how you can do this all day!!!"
Since you are a developer, my feeling is that you do like writing code, so the real problem is that you are not correctly following TDD and making tests a real part of your development. Even though a framework may attempt to make this necessary, it is really up to you to correctly follow the process (i.e. write the test first) and to really integrate it with your development.
Then, it is really an insignificant part of the overall development, like checking in code, commenting, formatting - all of which some people might find "boring" but are necessary. It doesn't bother us because it is just part of development and we find development interesting.
First I want to write production code, so I strive to write test first : nor writing any line of code without a failing test. This is not always possible but at least it forces me to write tests.
Then I try to break the code I have written using boundary tests, negative cases, wrong API usage (e.g. missing or several initialization calls) ...
Also I run the test often ; the "all tests passed" message at the end makes me feel comfortable about what has been written so far ... and I'm also happy when I found (and fix) a bug.
Sometimes, I'm having fun with the names and the numbers I'm using for my tests (birth date, favorite player number, phone numbers ...).
Related
IN SDLC, the Testing procedure should be right after implementation. However, Test-driven development encourages us to do testing while doing implementation. And in my lecture course, Prof said test cases should be part of the design.
I am a junior developer, to implement a new feature, when should I design and document my test cases?
I found that it is not so practical to test all the cases after finishing the implementation. It is because once a cases is failed, I have to change the codes and retest all cases again. Is there another way to overcome and avoid this? I know automated testes is one of the solution, but somehow, automated testes cannot stimulate all of the test cases, especially integration test cases which involves different parties.
Also, in my test cases, should I test all parts of the code? OR just test the functionality of that features request? OR it actually depends on how much time you got?
Many thanks.
Your question isn't so easy to answer, because, as you say, "it actually depends on how much time you got." Here are some opinions though:
Test after implementation: No
As a programmer, you're an expensive and scarce resource with multiple deadlines stacked up on top of each other. So effectively, this means "never test". After you've implemented one chunk of code, you will move on to the next chunk of code, and mean to come back to write tests "when you have time" (you never have time).
There are also the problems you mention. If you do all your testing after the code is written, and your tests discover something fundamentally wrong, you have to go back and fix all your code as well as all your tests.
Test while implementing: Yes
This method is actually really helpful once you get a rhythm for it. You write a bit of a class, then write a bit of a unit test, and continually revise your tests and your code until you're finished. I believe it is actually faster than writing code without tests.
It is also particularly helpful when working on a large project. Running a unit test to see if a little module is working is instantaneous. Building and loading your entire application to see if a little module is working may take several minutes. It may also interrupt your concentration (which costs at least 10 minutes).
What to test: As much as possible
100% test coverage is probably never practical. But absolutely test the critical pieces of your program, things that perform mathematical computation or lots of business logic. Test everything that's leftover as much as possible. There's no reason to test a "toString()" function, unless that happens to be critical to business logic or something.
Also, keep your tests as simple as possible, just inputs and outputs. Most of my test functions are two or three lines. If your function is hard to test because there are too many combinations, it's a sign that your function might need to be broken up a little bit. Make sure to test edge cases and "impossible" scenarios.
My experience:
Document your tests with if the code is not self-explanatory or if the case being tested is a 'tricky' corner case which is not obvious at first sight (although the code may be).
Don't create separate documents for your tests. Put everything in comments and Javadocs if you are using Java. In other words, keep this information close to the code. That's where it is needed.
About designing and implementation: just iterate. Write some implementation, then a bit of testing code for it, then more implementation code, etc... until you are done with both implementation and test code. It goes faster than writing all implementation, then testing, then rewriting failing implementation code. You can't anticipate all tests to implement at design time, it is impossible. So, no worries if you don't get it all.
If you cover more than 80% of the code you are already good, more is better. Sometimes, code can't be tested. I recommend using test coverage tools, such as Emma for Java.
OR it actually depends on how much time you got?
The time you save by not testing is never ever ever paying for the time you have to spent to solve bugs later in the project. A proper test set always pays a lot down the road, always.
Looking at posts like this and others, it seems that the correct way to do TDD is to write a test for a feature, get just that feature to pass, and then add another test and refactor as necessary until it passes, then repeat.
My question is: why is this approach used? I completely understand the write tests first idea, because it helps your design. But why wouldn't I create all tests for a specific function, and then implement that function all at once until all tests pass?
The approach comes from the Extreme Programming principal of You Aren't Going to Need It. If you actually write a single test and then the code that makes it pass then repeating that process you usually find that you write just enough to get things working. You don't invent new features that are not needed. You don't handle corner cases that don't exist.
Try an experiment. Write out the list of tests you think you need. Set it aside. Then go with the one test at a time approach. See if the lists differ and why. When I do that I almost always end up with fewer tests. I almost always find that I invented a case that I didn't need if I do it the all the tests first way.
For me, it is about "thought burden." If I have all of the possible behaviors to worry about at once, my brain is strained. If I approach them one at a time, I can give full attention to solving the immediate problem.
I believe this derives from the principle of "YAGNI" ("You're Ain't Gonna Need It")(*), which states that classes should be as simple as necessary, with no extra features. Hence when you need a feature, you write a test for it, then you write the feature, then you stop. If you wrote a number of tests first, clearly you would be merely speculating on what your API would need to be at some point in the future.
(*) I generally translate that as "You are too stupid to know what will be needed in the future", but that's another topic......
imho it reduces the chance of over engineering the piece of code you are writing.
Its just easier to add unnecessary code when you are looking at different usage scenarios.
Dan North has suggested that there is no such thing as test-driven design because the design is not really driven out by testing -- that these unit tests only become tests once functionality is implemented, but during the design phase you are really designing by example.
This makes sense -- your tests are setting up a range of sample data and conditions with which the system under test is going to operate, and you drive out design based on these example scenarios.
Some of the other answers suggest that this is based on YAGNI. This is partly true.
Beyond that, though, there is the issue of complexity. As is often stated, programming is about managing complexity -- breaking things down into comprehensible units.
If you write 10 tests to cover cases where param1 is null, param2 is null, string1 is empty, int1 is negative, and the current day of the week is a weekend, and then go to implement that, you are having to juggle a lot of complexity at once. This opens up space to introduce bugs, and it becomes very difficult to sort out why tests are failing.
On the other hand, if you write the first test to cover an empty string1, you barely have to think about the implementation. Once the test is passing, you move on to a case where the current day is a weekend. You look at the existing code and it becomes obvious where the logic should go. You run tests and if the first test is now failing, you know that you broke it while implementing the day-of-the-week thing. I'd even recommend that you commit source between tests so that if you break something you can always revert to a passing state and try again.
Doing just a little at a time and then verifying that it works dramatically reduces the space for the introduction of defects, and when your tests fail after implementation you have changed so little code that it is very easy to identify the defect and correct it, because you know that the existing code was already working properly.
This is a great question. You need to find a balance between writing all tests in the universe of possible tests, and the most likely user scenarios. One test is, IMHO, not enough, and I typically like to write 3 or 4 tests which represent the most common uses of the feature. I also like to write a best case test and a worst case test as well.
Writing many tests helps you to anticipate and understand the potential use of your feature.
I believe TDD advocates writing one test at a time because it forces you to think in terms of the principle of doing the simplest thing that could possibly work at each step of development.
I think the article you sent is exactly the answer. If you write all the tests first and all of the scenarios first, you will probably write your code to handle all of those scenarios at once and most of the time you probably end up with code that is fairly complex to handle all of these.
On the other hand, if you go one at a time, you will end up refactoring your existing code each time to end up with code probably as simple as it can be for all the scenarios.
Like in the case of the link you gave in your question, had they written all the tests first, I am pretty sure they would have not ended up with a simple if/else statement, but probably a fairly complex recursive piece of code.
The reason behind the principle is simple. How practical it is to stick to is a separate question.
The reason is that if you are writing more code that what is needed to pass the current test you are writing code that is, by definition, untested. (It's nothing to do with YAGNI.)
If you write the next test to "catch up" with the production code then you've just written a test that you haven't seen fail. The test may be called "TestNextFeature" but it may as well return true for all the evidence you have on it.
TDD is all about making sure that all code - production and tests - is tested and that all those pesky "but I'm sure I wrote it right" bugs don't get into the code.
I would do as you suggest. Write several tests for a specific function, implement the function, and ensure that all of the tests for this function pass. This ensures that you understand the purpose and usage of the function separately from your implementation of it.
If you need to do a lot more implementation wise than what is tested by your unit tests, then your unit tests are likely not comprehensive enough.
I think part of that idea is to keep simplicity, keep to designed/planned features, and make sure that your tests are sufficient.
Lots of good answers above - YAGNI is the first answer that jumps to mind.
The other important thing about the 'just get the test passing' guideline though, is that TDD is actually a three stage process:
Red > Green > Refactor
Frequently revisiting the final part, the refactoring, is where a lot of the value of TDD is delivered in terms of cleaner code, better API design, and more confidence in the software. You need to refactor in really small short blocks though lest the task become too big.
It is hard to get into this habit, but stick with it, as it's an oddly satisfying way to work once you get into the cycle.
This question already has answers here:
Closed 13 years ago.
Duplicate:
Why should I practice Test Driven Development and how should I start?
For a developer that doesn't know about Test-Driven Development, what problem(s) will be solved by adopting TDD?
[EDIT] Let's assume that the developer already (ab)uses a unit testing framework.
Here are three reasons that TDD might help a developer/team:
Better understanding of what you're going to write
Enforces the policy of writing tests a little better
Speeds up development
One reason to write the tests first is to have a better understanding of the actual code before you write it. To me, this is the main benefit of test driven development. When you write the test cases first, you think more critically about the corner cases. It's then easier to address them when you write the code and ensure that they're accurate.
Another reason is to actually enforce writing the tests. Often when people do unit-testing without the TDD, they have a testing framework set up, write some new code, and then quit. They think that the code already works just fine, so why write tests? It's simple enough that it won't break, right? But now you've lost the advantages of doing unit-tests in the first place (completely different discussion). Write them first, and they're already there.
Writing these tests first could mean that you don't need to launch the program in a debugging environment (slow — especially for larger projects) to test if a few small things work. Of course there's no excuse for not doing so before committing changes.
Convincing yourself or other people to write the tests first may be difficult. You may have better luck getting them to write both at the same time which may be just as beneficial.
Presumably you test code that you've written before you commit it to a repository.
If that's not true you have other issues to deal with.
If it is true, you can look at writing tests using a framework as a way to automate those main routines or drivers that you currently write so you can run all of them automatically at the push of a button. You don't have to pore over output to decide if the test passed or failed; you embed the success or failure of the test in the code and get a thumbs up or down decision right away. Running all the tests at once reduces the chances of a "whack a mole" situation where you fix something in one class and break something else. All the tests have to pass.
Sounds good so far, yes?
The TDD folks just take it one step further by demanding that you write the test FIRST before you write the class. It fails, of course, because you haven't written the class. It's their way of guaranteeing that you write test classes.
If you're already using a test framework, getting good value out of the tests you write, and have meaningful code coverage up around 70%, then I think you're doing well. I'm not sure that TDD will give you much more value. It's up to you to decide whether or not you go that extra mile. Personally, I don't do it. I write tests after the class and refactor if I feel the need. Some people might find it helpful to write the test first knowing it'll fail, but I don't.
(This is more of a comment agreeing with duffymo's answer than an answer of its own.)
duffymo answers:
The TDD folks just take it one step further by demanding that you write the test FIRST before you write the class. It fails, of course, because you haven't written the class. It's their way of guaranteeing that you write test classes.
I think it's actually to force coders to think about what their code is doing. Having to think about a test makes one consider what the code is supposed to do: what the pre-conditions and post-conditions are, which functions are primitive and which are composed of primitive functions, what the minimal necessary public interface is, and what's an implementation detail.
These are all things I routinely think about, so like you, "test first" doesn't add a whole lot, for me. And frankly (I know this is heresy in some circles) I like to "anchor" the core ideas of a class by sketching out the public interface first; that way I can look at it, mentally use it, and see if it's as clean as I thought it was. (A class or a library should be easy and intuitive for client programmers to use.)
In other words, I do what TDD tries to ensure happens by writing tests first, but like duffymo, I get there a different way.
And the real point of "test first" is to get a coder to pause and think like a designer. It's silly to make a fetish of how the programmer enters that state; for those who don't do it naturally, "test first" serves as a ritual to get them there. For those who do, "test first" doesn't add much -- and can get in the way of the programmer's habitual way of getting into that state.
Again, we want to look at results, not rituals. If a junior guy needs a ritual, a "stations of the cross" or a rosary* to "get in the groove", "test first" serves that purpose. If someone has their own way to get there, that's great too.
Note that I'm not saying that code shouldn't be tested. It should. It gives us a safety net, which in turn allows us to concentrate our attention on writing good code, even audacious code, because we know the net is there to catch errors.
All I am saying is that fetishistic insistence on "test first" confuses the method (one of many) with the goal, making the programmer think about what he's coding.
* To be ecumenical, I'll note that both Catholics and Muslims use rosaries. And again, it's a mechanical, muscle-memory way to put oneself into a certain frame of mind. It's a fetish (in the original sense of a magic object, not the "sexual fetish" meaning) or good-luck charm. So is saying "Om mani padme hum", or sitting zazen, or stroking a "lucky" rabbit's foot, (Not so lucky for the rabbit.) The philosopher Jerry Fodor, when thinking about hard problems, has a similar ritual: he repeats to himself, "C'mon, Jerry, you can do it!" (I tried that too, but since my name is not Jerry, it didn't work for me. ;) )
Ideally:
You won't waste time writing features you don't need. You'll have a comprehensive unit test suite to serve as a safety net for refactoring. You'll have executable examples of how your code is intended to be used. Your development flow will be smoother and faster; you'll spend less time in the debugger.
But most of all, your design will be better. Your code will be better factored - loosely coupled, highly cohesive - and better formed - smaller, better-named methods & classes.
For my current project (which runs on a relatively heavyweight process), I have adopted a peculiar form of TDD that consists of writing skeleton test cases based on requirements documents and GUI mockups. I write dozens, sometimes hundreds of those before starting to implement anything (this runs totally against "pure" TDD which says you should write a few tests, then immediately start on a skeleton implementation).
I have found this to be an excellent way to review the requirements documents. I have to think about the behaviour described in them much more intensively than if I just were to read them . In consequence, I find many more inconsistencies and gaps in them which I would otherwise only have found during implementation. This way, I can ask for clarification earlier and have better requirements when I start implementing.
Then, during implementation, the tests are a way to measure how far I've yet to go. And they prevent me from forgetting anything (don't laugh, that's a real problem when you work on larger use cases).
And the moral is: even when your dev process doesn't really support TDD, it can still be done in a way, and improve quality and productivity.
I personally do not use TDD, but one of the biggest pro's I can see with the methology is that customer satisfaction ensurance. Basically, the idea is that the steps of your development process are these:
1) Talk to customer about what the application is supposed to do, and how it is supposed to react to different situations.
2) Translate the outcome of 1) into Unit Tests, which each test one feature or scenario.
3) Write simple, "sloppy" code that (barely) passes the tests. When this is done, you have met your customer's expectations.
4) Refactor the code you wrote in 3) until you think you've done it in the most effective way possible.
When this is done you have hopefully produced high-quality code, that meets your customer's needs. If the customer now wants a new feature, you start the cycle over - discuss the feature, write a test that makes sure it works, write code that passes the test, refactor.
And as others have said, each time you run your tests you ensure that the old code still works, and that you can add new functionality without breaking old one.
Most of the people I have talked to don't use a complete TDD model. They usually find the best testing model that works for them. Find yours play with TDD and find where you are the most productive.
TDD (Test Driven Development/ Design) provides the following advantages
ensures you know the story card's acceptance criteria before you start
ensures that you know when to stop coding (i.e., when the acceptance criteria has been meet thus prevents gold platting)
As a result you end up with code that is
testable
clean design
able to be refactored with confidence
the minimal code necessary to satisfy the story card
a living specification of how the code works
able to support a sustainable pace of new features
I made a big effort to learn TDD for Ruby on Rails development. It took several days before I really got into it and it. I was very skeptical but I made the effort because programmers I respect support it.
At this point I feel it was definitely worth the effort. There are several benefits which I'm sure others will be happy to list for you. To me the most important advantage is that it helps avoid that nightmare situation late in a project where something suddenly breaks for no apparent reason and then you're spending a day and a half with the debugger. It helps prevent your code base from deteriorating as you add more and more logic to it.
It is common knowledge that writing tests and having a large number of automated tests are a Good Thing.
However, without TDD, it often just becomes tedious. People write tests, and then leave it, and the tests do not get updated as they should, nor do new features get tested as often as they should either.
A big part of this is because the code has become a pain to test - TDD will influence your design so that it is much easier to test. Because you've used TDD, you have a good number of tests, which makes it much easier to find regressions whenever your code or requirements change, simplifying debugging drammatically, causing an appreciation of good TDD and encouraging more tests to be written when changes are needed - and we're back to the start of the cycle.
There are many advantages:
Higher code quality
Fewer bugs
Less wasted time
Any of those alone would be sufficient justification to implement TDD.
I know that TDD style is writing the test first, see it fails then go and make it green, which is good stuff. Sometimes it really works for me.
However especially when I was experimenting with some stuff (i.e. not sure about the design, not sure if it's going to work) or frantically writing code, I don't want to write unit tests, it breaks my flow.
I tend to write unit tests later on and especially just before stuff getting too complicated. Also there is another problem writing them later is generally more boring.
I'm not quite sure if this is a good approach (definitely not the best).
What do you think? Do you code write your unit tests later? Or how do you deal this flow problem or experimental design / code stage.
What I've learned is that there is no experimental code, at least not working in production environments and/or tight deadlines. Experiments are generally carried out until something "works" at which point that becomes the production code.
The other side of this is that TDD from the start will result in better design of your code. You'll be thinking more about it, reworking it, refactoring it more frequently than if you write the tests after the fact.
I've written tests after the fact. Better late then never. They are always worth having.
However, I have to say, the first time I wrote them before writing the tested code, it was extremely satisfying. No more fiddling around with manual testing. I was surprised just how good it felt.
Also, I tend to write unit tests before refactoring legacy code - which, almost by definition, means that I'm writing tests to test code that's already written. Provides a security blanket that makes me more comfortable with getting into big blocks of code written by others.
"I'm not quite sure if this is a good approach (definitely not the best)."
Not good? Why not?
Are you designing for testability? In that case, your design is test-driven. What more can anyone ask for?
Whether the tests come first, in the middle or last doesn't matter as much as designing for testability. In the end, changes to the design will make tests fail, and you can fix things. Changes to the tests in anticipation of design changes will make the tests fail, also. Both are fine.
If you get to the end of your design work, and there's something hard to test in the middle, then you failed to do TDD. You'll have to refactor your design to make it testable.
I often take the same approach you're talking about. What seems to work well is to treat the exerimental code exactly as such, and then start a proper design based on what you've learned. From here you can write your tests first. Otherwise, you're left with lots of code that was written as temporary or experimental, and probably won't get around to writing tests for all of it.
I would say that for normal development, TDD works extremely well. There are cases where you may not need to write the tests first (or even at all), but those are infrequent. Sometimes, however, you need to do some exploration to see what will work. I would consider this to be a "spike", and I don't necessarily think that TDD is absolutely necessary in this case. I would probably not use the actual "spike" code in my project. After all, it was just an exploration and now that I have a better idea of how it ought to work, I can probably write better code (and tests) than my "spike" code. If I did decide to use my "spike" code, I'd probably go back and write tests for it.
Now, if you find that you've violated TDD and written some production code before your tests - it happens - then, too, I'd go back and write the tests. In fact, on the occasions where this has happened to me I've often found things that I've neglected once I start writing the tests because more tests come to mind that aren't handled by the code. Eventually, you get back in the TDD rythym (and vow never to do that again).
Consider the psychological tendencies associated with sunk cost. That is, when you get to the second part of the equation, that laziness gene we all have makes us want to protect the work we have already done. The consequences?
If you write the tests first...
You tend to write the code to fit the tests. This encourages the "simplest thing that solves the problem" type development and keeps you focused on solving the problem not working on meta-problems.
If you write the code first...
You will be tempted to write the tests to fit the code. In effect this is the equivalent of writing the problem to fit your answer, which is kind of backwards and will quite often lead to tests that are of lesser value.
Although I'd be surprised if 1 programmer out of 50 ALWAYS writes tests first, I'd still argue that it is something to strive for if you want to write good software.
I usually write my tests first but sometime while experimenting I write the code after. Once I get an idea of what my code is supposed to do, I stop the code and start the tests.
Writing the code first is natural when you're trying to figure out how your code is going to work. Writing the test first helps you determine what your code show do (not how it should do it). If you're writing the code first, you're trying to solve the problem without completely defining the problem. This isn't necessarily "bad", but you are using unit tests as a regression tool rather than a development tool (again, not "bad" - just not TDD).
VS 2008 has a nice feature that will generate test classes for an object, the tests needs to me tweaked but it dose a lot of the grunt work for you. Its really nice for crating tests for your less then diligent co-workers.
Another good point for this is it help to prevent you from missing something, expectantly when your working on code that isn't yours.
if your using a different testing framework then MSUnitTest, it's fairly simple to convert then tests from MSUnit to Nunit, etc. just do some copy and past.
I would like to say that I always write Unit tests first but of course I don't (for numerous reasons well known to any real programmer :-)). What I (ok, also not always...) do is to convert every bug which takes me more than five minutes to find into a unit test. Even before I fix it. This has the following advantages:
It documents the bug and alerts me if it shows up again at a later point of time.
It helps in finding the bug, since I have a well-defined place to put debugging code into (setting up my data structures, call the right methods, set breakpoints on etc.) Before I discovered unit testing I modified the main() function for this testing code resulting in strange results when I forgot to remove it afterwards ...
Usually it gives me good ideas what else could go wrong, so it quite often evolves in a whole bunch of unit tests and resulting in more than one bug getting discovered resp. fixed.
I know that one of the defining principles of Test driven development is that you write your Unit tests first and then write code to pass those unit tests, but is it necessary to do it this way?
I've found that I often don't know what I am testing until I've written it, mainly because the past couple of projects I've worked on have more evolved from a proof of concept rather than been designed.
I've tried to write my unit tests before and it can be useful, but it doesn't seem natural to me.
Some good comments here, but I think that one thing is getting ignored.
writing tests first drives your design. This is an important step. If you write the tests "at the same time" or "soon after" you might be missing some design benefits of doing TDD in micro steps.
It feels really cheesy at first, but it's amazing to watch things unfold before your eyes into a design that you didn't think of originally. I've seen it happen.
TDD is hard, and it's not for everybody. But if you already embrace unit testing, then try it out for a month and see what it does to your design and productivity.
You spend less time in the debugger and more time thinking about outside-in design. Those are two gigantic pluses in my book.
There have been studies that show that unit tests written after the code has been written are better tests. The caveat though is that people don't tend to write them after the event. So TDD is a good compromise as at least the tests get written.
So if you write tests after you have written code, good for you, I'd suggest you stick at it.
I tend to find that I do a mixture. The more I understand the requirements, the more tests I can write up front. When the requirements - or my understanding of the problem - are weak, I tend to write tests afterwards.
TDD is not about the tests, but how the tests drive your code.
So basically you are writing tests to let an architecture evolve naturally (and don't forget to refactor !!! otherwise you won't get much benefit out of it).
That you have an arsenal of regression tests and executable documentation afterwards is a nice sideeffect, but not the main reason behind TDD.
So my vote is:
Test first
PS: And no, that doesn't mean that you don't have to plan your architecture before, but that you might rethink it if the tests tell you to do so !!!!
I've lead development teams for the past 6-7 years. What I can tell for sure is that as a developer and the developers I have worked with, it makes a phenomenal difference in the quality of the code if we know where our code fits into the big picture.
Test Driven Development (TDD) helps us answer "What?" before we answer "How?" and it makes a big difference.
I understand why there may be apprehensions about not following it in PoC type of development/architect work. And you are right it may not make a complete sense to follow this process. At the same time, I would like to emphasize that TDD is a process that falls in the Development Phase (I know it sounds obsolete, but you get the point :) when the low level specification are clear.
I think writing the test first helps define what the code should actually do. Too many times people don't have a good definition of what the code is supposed to do or how it should work. They simply start writing and make it up as they go along. Creating the test first makes you focus on what the code will do.
Not always, but I find that it really does help when I do.
I tend to write them as I write my code. At most I will write the tests for if the class/module exists before I write it.
I don't plan far enough ahead in that much detail to write a test earlier than the code it is going to test.
I don't know if this is a flaw in my thinking or method's or just TIMTOWTDI.
I start with how I would like to call my "unit" and make it compile.
like:
picker = Pick.new
item=picker.pick('a')
assert item
then I create
class Pick
def pick(something)
return nil
end
end
then I keep on using the Pick in my "test" case so I could see how I would like it to be called and how I would treat different kinds of behavior. Whenever I realize I could have trouble on some boundaries or some kind of error/exception I try to get it to fire and get an new test case.
So, in short. Yes.
The ratio doing test before is a lot higher than not doing it.
Directives are suggestion on how you could do things to improve the overall quality or productivity or even both of the end product. They are in no ways laws to be obeyed less you get smitten in a flash by the god of proper coding practice.
Here's my compromise on the take and I found it quite useful and productive.
Usually the hardest part to get right are the requirements and right behind it the usability of your class, API, package... Then is the actual implementation.
Write your interfaces (they will change, but will go a long way in knowing WHAT has to be done)
Write a simple program to use the interfaces (them stupid main). This goes a long way in determining the HOW it is going to be used (go back to 1 as often as needed)
Write tests on the interface (The bit I integrated from TDD, again go back to 1 as often as needed)
write the actual code behind the interfaces
write tests on the classes and the actual implementation, use a coverage tool to make sure you do not forget weid execution paths
So, yes I write tests before coding but never before I figured out what needs to be done with a certain level of details. These are usually high level tests and only treat the whole as a black box. Usually will remain as integration tests and will not change much once the interfaces have stabilized.
Then I write a bunch of tests (unit tests) on the implementation behind it, these will be much more detailed and will change often as the implementation evolves, as it get's optimized and expanded.
Is this strictly speaking TDD ? Extreme ? Agile...? whatever... ? I don't know, and frankly I don't care. It works for me. I adjust it as needs go and as my understanding of software development practice evolve.
my 2 cent
I've been programming for 20 years, and I've virtually never written a line of code that I didn't run some kind of unit test on--Honestly I know people do it all the time, but how someone can ship a line of code that hasn't had some kind of test run on it is beyond me.
Often if there is no test framework in place I just write a main() into each class I write. It adds a little cruft to your app, but someone can always delete it (or comment it out) if they want I guess. I really wish there was just a test() method in your class that would automatically compile out for release builds--I love my test method being in the same file as my code...
So I've done both Test Driven Development and Tested development. I can tell you that TDD can really help when you are a starting programmer. It helps you learn to view your code "From outside" which is one of the most important lessons a programmer can learn.
TDD also helps you get going when you are stuck. You can just write some very small piece that you know your code has to do, then run it and fix it--it gets addictive.
On the other hand, when you are adding to existing code and know pretty much exactly what you want, it's a toss-up. Your "Other code" often tests your new code in place. You still need to be sure you test each path, but you get a good coverage just by running the tests from the front-end (except for dynamic languages--for those you really should have unit tests for everything no matter what).
By the way, when I was on a fairly large Ruby/Rails project we had a very high % of test coverage. We refactored a major, central model class into two classes. It would have taken us two days, but with all the tests we had to refactor it ended up closer to two weeks. Tests are NOT completely free.
I'm not sure, but from your description I sense that there might be a misunderstanding on what test-first actually means. It does not mean that you write all your tests first. It does mean that you have a very tight cycle of
write a single, minimal test
make the test pass by writing the minimal production code necessary
write the next test that will fail
make all the existing tests pass by changing the existing production code in the simplest possible way
refactor the code (both test and production!) so that it doesn't contain duplication and is expressive
continue with 3. until you can't think of another sensible test
One cycle (3-5) typically just takes a couple of minutes. Using this technique, you actually evolve the design while you write your tests and production code in parallel. There is not much up front design involved at all.
On the question of it being "necessary" - no, it obviously isn't. There have been uncountable projects successfull without doing TDD. But there is some strong evidence out there that using TDD typically leads to significantly higher quality, often without negative impact on productivity. And it's fun, too!
Oh, and regarding it not feeling "natural", it's just a matter of what you are used to. I know people who are quite addicted to getting a green bar (the typical xUnit sign for "all tests passing") every couple of minutes.
There are so many answers now and they are all different. This perfectly resembles the reality out there. Everyone is doing it differently. I think there is a huge misunderstanding about unit testing. It seems to me as if people heard about TDD and they said it's good. Then they started to write unit tests without really understanding what TDD really is. They just got the part "oh yeah we have to write tests" and they agree with it. They also heard about this "you should write your tests first" but they do not take this serious.
I think it's because they do not understand the benefits of test-first which in turn you can only understand once you've done it this way for some time. And they always seem to find 1.000.000 excuses why they don't like writing the tests first. Because it's too difficult when figuring out how everything will fit together etc. etc. In my opinion, it's all excuses for them to hide away from their inability to once discipline themselve, try the test-first approach and start to see the benefits.
The most ridicoulous thing if they start to argue "I'm not conviced about this test-first thing but I've never done it this way" ... great ...
I wonder where unit testing originally comes from. Because if the concept really originates from TDD then it's just ridicoulous how people get it wrong.
Writing the tests first defines how your code will look like - i.e. it tends to make your code more modular and testable, so you do not create a "bloat" methods with very complex and overlapping functionality. This also helps to isolate all core functionality in separate methods for easier testing.
Personally, I believe unit tests lose a lot of their effectiveness if not done before writing the code.
The age old problem with testing is that no matter how hard we think about it, we will never come up with every possibly scenario to write a test to cover.
Obviously unit testing itself doesn't prevent this completely, as it restrictive testing, looking at only one unit of code not covering the interactions between this code and everything else, but it provides a good basis for writing clean code in the first place that should at least restrict the chances for issues of interaction between modules. I've always worked to the principle of keeping code as simple as it possibly can be - infact I believe this is one of the key principles of TDD.
So starting off with a test that basically says you can create a class of this type and build it up, in theory, writing a test for every line of code or at least covering every route through a particular piece of code. Designing as you go! Obviously based on a rough-up-front design produced initially, to give you a framework to work to.
As you say it is very unnatural to start with and can seem like a waste of time, but I've seen myself first hand that it pays off in the long run when defects stats come through and show the modules that were fully written using TDD have far lower defects over time than others.
Before, during and after.
Before is part of the spec, the contract, the definition of the work
During is when special cases, bad data, exceptions are uncovered while implementing.
After is maintenance, evolution, change, new requirements.
I don't write the actual unit tests first, but I do make a test matrix before I start coding listing all the possible scenarios that will have to be tested. I also make a list of cases that will have to be tested when a change is made to any part of the program as part of regression testing that will cover most of the basic scenarios in the application in addition to fully testing the bit of code that changed.
Remember with Extreme programming your tests effectly are you documenation. So if you don't know what you're testing, then you don't know what you want your application is going to do?
You can start off with "Stories" which might be something like
"Users can Get list of Questions"
Then as you start writing code to solve the unit tests. To solve the above you'll need at least a User and question class. So then you can start thinking about the fields:
"User Class Has Name DOB Address TelNo Locked Fields"
etc.
Hope it helps.
Crafty
Yes, if you are using true TDD principles. Otherwise, as long as you're writing the unit-tests, you're doing better than most.
In my experience, it is usually easier to write the tests before the code, because by doing it that way you give yourself a simple debugging tool to use as you write the code.
I write them at the same time. I create the skeleton code for the new class and the test class, and then I write a test for some functionality (which then helps me to see how I want the new object to be called), and implement it in the code.
Usually, I don't end up with elegant code the first time around, it's normally quite hacky. But once all the tests are working, you can refactor away until you end up with something pretty neat, tidy and proveable to be rock solid.
It helps when you are writing something that you are used writing to write first all the thing you would regularly check for and then write those features. More times then not those features are the most important for the piece of software you are writing. Now , on the other side there are not silver bullets and thing should never be followed to the letter. Developer judgment plays a big role in the decision of using test driven development versus test latter development.